<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2407.03227] Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning</title><meta property="og:description" content="We focus on Text-to-SQL semantic parsing from the perspective of Large Language Models. Motivated by challenges related to the size of commercial database schemata and the deployability of business intelligence solutio…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2407.03227">

<!--Generated on Mon Aug  5 16:49:33 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document">
<h1 class="ltx_title ltx_title_document">Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Zhili Shen<sup id="id5.4.id1" class="ltx_sup"><span id="id5.4.id1.1" class="ltx_text ltx_font_italic">†</span></sup>  Pavlos Vougiouklis<sup id="id6.5.id2" class="ltx_sup"><span id="id6.5.id2.1" class="ltx_text ltx_font_italic">†</span></sup>  Chenxin Diao<sup id="id7.6.id3" class="ltx_sup"><span id="id7.6.id3.1" class="ltx_text ltx_font_italic">†</span></sup>  <span id="id8.7.id4" class="ltx_text ltx_font_bold">Kaustubh Vyas
<br class="ltx_break"></span> <span id="id9.8.id5" class="ltx_text ltx_font_bold">Yuanyi Ji</span>  <span id="id10.9.id6" class="ltx_text ltx_font_bold">Jeff Z. Pan
<br class="ltx_break"></span>Huawei Technologies
<br class="ltx_break">Edinburgh RC, CSI

<br class="ltx_break">Edinburgh, United Kingdom
<br class="ltx_break"><span id="id11.10.id7" class="ltx_text ltx_font_typewriter">{<a href="mailto:zhilishen@huawei.com" title="" class="ltx_ref ltx_href">zhilishen</a>,</span> <a href="mailto:pavlos.vougiouklis@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">pavlos.vougiouklis</a><span id="id12.11.id8" class="ltx_text ltx_font_typewriter">,</span> <a href="mailto:chenxindiao@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">chenxindiao</a><span id="id13.12.id9" class="ltx_text ltx_font_typewriter">,</span> <a href="mailto:kaustubh.vyas@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">kaustubh.vyas</a><span id="id14.13.id10" class="ltx_text ltx_font_typewriter">}@huawei.com
<br class="ltx_break">{<a href="mailto:jiyuanyi@huawei.com" title="" class="ltx_ref ltx_href">jiyuanyi</a>,</span> <a href="mailto:jeff.pan@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">jeff.pan</a><span id="id15.14.id11" class="ltx_text ltx_font_typewriter">}@huawei.com
<br class="ltx_break"></span>
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id16.id1" class="ltx_p">We focus on Text-to-SQL semantic parsing from the perspective of Large Language Models. Motivated by challenges related to the size of commercial database schemata and the deployability of business intelligence solutions, we propose an approach that dynamically retrieves input database information and uses abstract syntax trees to select few-shot examples for in-context learning.</p>
<p id="id4.1" class="ltx_p">Furthermore, we investigate the extent to which an in-parallel semantic parser can be leveraged for generating <span id="id4.1.1" class="ltx_text ltx_font_italic">approximated</span> versions of the expected SQL queries, to support our retrieval. We take this approach to the extreme—we adapt a model consisting of less than <math id="id4.1.m1.1" class="ltx_Math" alttext="500" display="inline"><semantics id="id4.1.m1.1a"><mn id="id4.1.m1.1.1" xref="id4.1.m1.1.1.cmml">500</mn><annotation-xml encoding="MathML-Content" id="id4.1.m1.1b"><cn type="integer" id="id4.1.m1.1.1.cmml" xref="id4.1.m1.1.1">500</cn></annotation-xml><annotation encoding="application/x-tex" id="id4.1.m1.1c">500</annotation></semantics></math>M parameters, to act as an extremely efficient approximator, enhancing it with the ability to process schemata in a parallelised manner. We apply our approach to monolingual and cross-lingual benchmarks for semantic parsing, showing improvements over state-of-the-art baselines. Comprehensive experiments highlight the contribution of modules involved in this retrieval-augmented generation setting, revealing interesting directions for future work.</p>
</div>
<div id="p1" class="ltx_para ltx_noindent">
<div id="p1.3" class="ltx_block ltx_align_bottom">
<p id="p1.3.4" class="ltx_p"><span id="p1.3.4.1" class="ltx_text ltx_font_bold">Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning</span></p>
<br class="ltx_break ltx_centering">
<p id="p1.3.3" class="ltx_p ltx_align_center" style="width:433.6pt;"><span id="p1.3.3.3" class="ltx_text ltx_inline-block" style="width:0.0pt;">
<span id="p1.3.3.3.3" class="ltx_tabular ltx_align_top">
<span class="ltx_tbody">
<span id="p1.3.3.3.3.3" class="ltx_tr">
<span id="p1.3.3.3.3.3.3" class="ltx_td ltx_align_center"><span id="p1.3.3.3.3.3.3.3" class="ltx_text ltx_font_bold">Zhili Shen<sup id="p1.3.3.3.3.3.3.3.1" class="ltx_sup"><span id="p1.3.3.3.3.3.3.3.1.1" class="ltx_text ltx_font_medium ltx_font_italic">†</span></sup>  Pavlos Vougiouklis<sup id="p1.3.3.3.3.3.3.3.2" class="ltx_sup"><span id="p1.3.3.3.3.3.3.3.2.1" class="ltx_text ltx_font_medium ltx_font_italic">†</span></sup>  Chenxin Diao<sup id="p1.3.3.3.3.3.3.3.3" class="ltx_sup"><span id="p1.3.3.3.3.3.3.3.3.1" class="ltx_text ltx_font_medium ltx_font_italic">†</span></sup>  Kaustubh Vyas</span></span></span>
<span id="p1.3.3.3.3.4.1" class="ltx_tr">
<span id="p1.3.3.3.3.4.1.1" class="ltx_td ltx_align_center"> <span id="p1.3.3.3.3.4.1.1.1" class="ltx_text ltx_font_bold">Yuanyi Ji</span>  <span id="p1.3.3.3.3.4.1.1.2" class="ltx_text ltx_font_bold">Jeff Z. Pan</span></span></span>
<span id="p1.3.3.3.3.5.2" class="ltx_tr">
<span id="p1.3.3.3.3.5.2.1" class="ltx_td ltx_align_center">Huawei Technologies</span></span>
<span id="p1.3.3.3.3.6.3" class="ltx_tr">
<span id="p1.3.3.3.3.6.3.1" class="ltx_td ltx_align_center">Edinburgh RC, CSI</span></span>
<span id="p1.3.3.3.3.7.4" class="ltx_tr">
<span id="p1.3.3.3.3.7.4.1" class="ltx_td ltx_align_center">Edinburgh, United Kingdom</span></span>
<span id="p1.3.3.3.3.8.5" class="ltx_tr">
<span id="p1.3.3.3.3.8.5.1" class="ltx_td ltx_align_center"><span id="p1.3.3.3.3.8.5.1.1" class="ltx_text ltx_font_typewriter">{<a href="mailto:zhilishen@huawei.com" title="" class="ltx_ref ltx_href">zhilishen</a>,</span> <a href="mailto:pavlos.vougiouklis@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">pavlos.vougiouklis</a><span id="p1.3.3.3.3.8.5.1.2" class="ltx_text ltx_font_typewriter">,</span> <a href="mailto:chenxindiao@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">chenxindiao</a><span id="p1.3.3.3.3.8.5.1.3" class="ltx_text ltx_font_typewriter">,</span> <a href="mailto:kaustubh.vyas@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">kaustubh.vyas</a><span id="p1.3.3.3.3.8.5.1.4" class="ltx_text ltx_font_typewriter">}@huawei.com</span></span></span>
<span id="p1.3.3.3.3.9.6" class="ltx_tr">
<span id="p1.3.3.3.3.9.6.1" class="ltx_td ltx_align_center"><span id="p1.3.3.3.3.9.6.1.1" class="ltx_text ltx_font_typewriter">{<a href="mailto:jiyuanyi@huawei.com" title="" class="ltx_ref ltx_href">jiyuanyi</a>,</span> <a href="mailto:jeff.pan@huawei.com" title="" class="ltx_ref ltx_href ltx_font_typewriter">jeff.pan</a><span id="p1.3.3.3.3.9.6.1.2" class="ltx_text ltx_font_typewriter">}@huawei.com</span></span></span>
</span>
</span></span></p>
<br class="ltx_break ltx_centering">
</div>
</div>
<span id="footnotex1" class="ltx_note ltx_role_footnotetext"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotetext: </span>The authors contributed equally to this work.</span></span></span>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Text-to-SQL semantic parsing aims at translating natural language questions into SQL, to facilitate querying relational databases by non-experts <cite class="ltx_cite ltx_citemacro_cite">Zelle and Mooney (<a href="#bib.bib33" title="" class="ltx_ref">1996</a>)</cite>.
Given their accessibility benefits, Text-to-SQL applications have become popular recently, with many corporations developing Business Intelligence platforms.</p>
</div>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">The success of Large Language Models (LLMs) in generalising across diverse Natural Language Processing tasks <cite class="ltx_cite ltx_citemacro_cite">Ye et al. (<a href="#bib.bib30" title="" class="ltx_ref">2023</a>); OpenAI et al. (<a href="#bib.bib17" title="" class="ltx_ref">2024</a>)</cite> has fuelled works that looked at how these multi-billion parameter models can be best employed for Text-to-SQL <cite class="ltx_cite ltx_citemacro_cite">Liu et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>); Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>)</cite>. Recent works in this space have focused on the in-context learning ability of LLMs, demonstrating that significant improvements can be achieved by selecting suitable (question, SQL) example pairs <cite class="ltx_cite ltx_citemacro_cite">Nan et al. (<a href="#bib.bib16" title="" class="ltx_ref">2023</a>); Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>); Guo et al. (<a href="#bib.bib11" title="" class="ltx_ref">2024</a>); Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>.
In spite of its underlying benefits, conventional solutions for example selection are usually limited to retrieving examples based solely on the similarity of questions <cite class="ltx_cite ltx_citemacro_cite">Nan et al. (<a href="#bib.bib16" title="" class="ltx_ref">2023</a>); An et al. (<a href="#bib.bib1" title="" class="ltx_ref">2023</a>); Guo et al. (<a href="#bib.bib11" title="" class="ltx_ref">2024</a>)</cite>.
Other approaches resort to a preliminary round of parsing which <span id="S1.p2.1.1" class="ltx_text ltx_font_italic">approximates</span> expected SQL queries, and directly use these approximations in few-shot prompting <cite class="ltx_cite ltx_citemacro_cite">Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>, or to subsequently select (question, SQL) pairs by comparing the approximated query to queries within candidate examples <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>.
The approach proposed by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib10" title="" class="ltx_ref">Gao et al.</a></cite> transforms SQL queries into SQL skeletons <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib12" title="" class="ltx_ref">2023a</a>)</cite> and then filters examples by considering overlap token ratio as the similarity between two skeletons. While incorporating SQL skeleton similarity improves over conventional example selection for Text-to-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>, it can result in structural information loss as exemplified in Table <a href="#S1.T1" title="Table 1 ‣ 1 Introduction ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, where two dissimilar SQL queries are treated as identical. In this paper, we propose a novel approach that selects examples using similarity of normalised SQL Abstract Syntax Trees (ASTs). We argue that considering the similarity of such hierarchical structures can significantly enhance LLMs’ performance for Text-to-SQL parsing.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.1" class="ltx_p">Apart from example selection, we refine database context input to LLMs by dynamically pruning schemata and selecting values.
From the perspective of LLMs, existing studies achieve improvements by including the full database schema in the prompt and additionally <span id="S1.p3.1.1" class="ltx_text ltx_font_italic">hinting</span> the importance of particular schema elements or values <cite class="ltx_cite ltx_citemacro_cite">Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>); Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>.
In this paper, we show that the performance can be boosted with schemata of reduced size.</p>
</div>
<div id="S1.p4" class="ltx_para">
<p id="S1.p4.1" class="ltx_p">Inspired by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib10" title="" class="ltx_ref">Gao et al.</a></cite> that compute an approximated query for a given input question, we further explore how combinations of a sparse retriever with such an in-parallel semantic parser (we would refer to it as <span id="S1.p4.1.1" class="ltx_text ltx_font_italic">approximator</span>) can be used to retrieve relevant database context input to LLMs. For efficiency, we adapt the semantic parser (a decoder-free model with <math id="S1.p4.1.m1.1" class="ltx_Math" alttext="&lt;500" display="inline"><semantics id="S1.p4.1.m1.1a"><mrow id="S1.p4.1.m1.1.1" xref="S1.p4.1.m1.1.1.cmml"><mi id="S1.p4.1.m1.1.1.2" xref="S1.p4.1.m1.1.1.2.cmml"></mi><mo id="S1.p4.1.m1.1.1.1" xref="S1.p4.1.m1.1.1.1.cmml">&lt;</mo><mn id="S1.p4.1.m1.1.1.3" xref="S1.p4.1.m1.1.1.3.cmml">500</mn></mrow><annotation-xml encoding="MathML-Content" id="S1.p4.1.m1.1b"><apply id="S1.p4.1.m1.1.1.cmml" xref="S1.p4.1.m1.1.1"><lt id="S1.p4.1.m1.1.1.1.cmml" xref="S1.p4.1.m1.1.1.1"></lt><csymbol cd="latexml" id="S1.p4.1.m1.1.1.2.cmml" xref="S1.p4.1.m1.1.1.2">absent</csymbol><cn type="integer" id="S1.p4.1.m1.1.1.3.cmml" xref="S1.p4.1.m1.1.1.3">500</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p4.1.m1.1c">&lt;500</annotation></semantics></math>M parameters) proposed by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib27" title="" class="ltx_ref">Vougiouklis et al.</a></cite> to process schemata in a parallelised manner.
Using this efficient approximator, our schema pruning strategy selects a relevant sub-schema in order to simplify the task for LLMs and reduce the relevant computational workload. Furthermore, it enables LLM-based Text-to-SQL solutions to handle longer schemata (usually associated with commercial use-cases) exceeding their context window size.</p>
</div>
<div id="S1.p5" class="ltx_para">
<p id="S1.p5.1" class="ltx_p">We apply our approach on monolingual (S<span id="S1.p5.1.1" class="ltx_text ltx_font_smallcaps">pider</span>, S<span id="S1.p5.1.2" class="ltx_text ltx_font_smallcaps">pider</span>-DK, S<span id="S1.p5.1.3" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="S1.p5.1.4" class="ltx_text ltx_font_smallcaps">eal</span> and S<span id="S1.p5.1.5" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="S1.p5.1.6" class="ltx_text ltx_font_smallcaps">yn</span>) and cross-lingual (CS<span id="S1.p5.1.7" class="ltx_text ltx_font_smallcaps">pider</span>) benchmarks of different generalisation challenges. We evaluate the applicability of our framework across both closed- and open-source LLMs. Our framework, comprising only a single round of prompting, achieves state-of-the-art performance, outperforming other
baselines which may comprise complex prompting and multiple iterations, when LLMs of equal capacity are involved. Through comprehensive experiments, we highlight strengths and limitations. Our contributions can be summarised as follows:</p>
<ul id="S1.I1" class="ltx_itemize">
<li id="S1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i1.p1" class="ltx_para">
<p id="S1.I1.i1.p1.1" class="ltx_p">We propose a novel approach for selecting (question, SQL) examples using similarity of normalised SQL ASTs.</p>
</div>
</li>
<li id="S1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i2.p1" class="ltx_para">
<p id="S1.I1.i2.p1.1" class="ltx_p">We take efficient approximation to the extreme, presenting a schema-parallelisable adaptation of the fastest semantic parser to date.</p>
</div>
</li>
<li id="S1.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i3.p1" class="ltx_para">
<p id="S1.I1.i3.p1.1" class="ltx_p">We introduce a framework for dynamically selecting schema elements and database values, offering substantial execution accuracy improvements over prior works while significantly reducing the computational workload of LLMs.</p>
</div>
</li>
<li id="S1.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i4.p1" class="ltx_para">
<p id="S1.I1.i4.p1.1" class="ltx_p">We shed light on the benefits of database value selection and its symbiotic relation to schema pruning for Text-to-SQL LLM prompting.</p>
</div>
</li>
</ul>
</div>
<figure id="S1.T1" class="ltx_table">
<table id="S1.T1.2" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S1.T1.1.1" class="ltx_tr">
<th id="S1.T1.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" rowspan="2"><span id="S1.T1.1.1.1.1" class="ltx_text"><math id="S1.T1.1.1.1.1.m1.1" class="ltx_Math" alttext="\textbf{SQL}_{1}" display="inline"><semantics id="S1.T1.1.1.1.1.m1.1a"><msub id="S1.T1.1.1.1.1.m1.1.1" xref="S1.T1.1.1.1.1.m1.1.1.cmml"><mtext class="ltx_mathvariant_bold" id="S1.T1.1.1.1.1.m1.1.1.2" xref="S1.T1.1.1.1.1.m1.1.1.2a.cmml">SQL</mtext><mn id="S1.T1.1.1.1.1.m1.1.1.3" xref="S1.T1.1.1.1.1.m1.1.1.3.cmml">1</mn></msub><annotation-xml encoding="MathML-Content" id="S1.T1.1.1.1.1.m1.1b"><apply id="S1.T1.1.1.1.1.m1.1.1.cmml" xref="S1.T1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S1.T1.1.1.1.1.m1.1.1.1.cmml" xref="S1.T1.1.1.1.1.m1.1.1">subscript</csymbol><ci id="S1.T1.1.1.1.1.m1.1.1.2a.cmml" xref="S1.T1.1.1.1.1.m1.1.1.2"><mtext class="ltx_mathvariant_bold" id="S1.T1.1.1.1.1.m1.1.1.2.cmml" xref="S1.T1.1.1.1.1.m1.1.1.2">SQL</mtext></ci><cn type="integer" id="S1.T1.1.1.1.1.m1.1.1.3.cmml" xref="S1.T1.1.1.1.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.T1.1.1.1.1.m1.1c">\textbf{SQL}_{1}</annotation></semantics></math></span></th>
<th id="S1.T1.1.1.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_th ltx_th_column ltx_border_tt" style="width:170.7pt;">
<span id="S1.T1.1.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="S1.T1.1.1.2.1.1" class="ltx_p"><span id="S1.T1.1.1.2.1.1.1" class="ltx_text ltx_font_typewriter">SELECT T2.name, T2.capacity FROM concert AS T1 JOIN stadium AS T2 ON T1.stadium_id = T2.stadium_id WHERE T1.year &gt;= 2014</span></span>
</span>
</th>
</tr>
<tr id="S1.T1.2.3.1" class="ltx_tr">
<td id="S1.T1.2.3.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:170.7pt;">
<span id="S1.T1.2.3.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="S1.T1.2.3.1.1.1.1" class="ltx_p"><span id="S1.T1.2.3.1.1.1.1.1" class="ltx_text ltx_font_bold">Skeleton:</span> <span id="S1.T1.2.3.1.1.1.1.2" class="ltx_text ltx_font_typewriter">select _ from _ where _</span></span>
</span>
</td>
</tr>
<tr id="S1.T1.2.2" class="ltx_tr">
<th id="S1.T1.2.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb ltx_border_t" rowspan="2"><span id="S1.T1.2.2.1.1" class="ltx_text"><math id="S1.T1.2.2.1.1.m1.1" class="ltx_Math" alttext="\textbf{SQL}_{2}" display="inline"><semantics id="S1.T1.2.2.1.1.m1.1a"><msub id="S1.T1.2.2.1.1.m1.1.1" xref="S1.T1.2.2.1.1.m1.1.1.cmml"><mtext class="ltx_mathvariant_bold" id="S1.T1.2.2.1.1.m1.1.1.2" xref="S1.T1.2.2.1.1.m1.1.1.2a.cmml">SQL</mtext><mn id="S1.T1.2.2.1.1.m1.1.1.3" xref="S1.T1.2.2.1.1.m1.1.1.3.cmml">2</mn></msub><annotation-xml encoding="MathML-Content" id="S1.T1.2.2.1.1.m1.1b"><apply id="S1.T1.2.2.1.1.m1.1.1.cmml" xref="S1.T1.2.2.1.1.m1.1.1"><csymbol cd="ambiguous" id="S1.T1.2.2.1.1.m1.1.1.1.cmml" xref="S1.T1.2.2.1.1.m1.1.1">subscript</csymbol><ci id="S1.T1.2.2.1.1.m1.1.1.2a.cmml" xref="S1.T1.2.2.1.1.m1.1.1.2"><mtext class="ltx_mathvariant_bold" id="S1.T1.2.2.1.1.m1.1.1.2.cmml" xref="S1.T1.2.2.1.1.m1.1.1.2">SQL</mtext></ci><cn type="integer" id="S1.T1.2.2.1.1.m1.1.1.3.cmml" xref="S1.T1.2.2.1.1.m1.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.T1.2.2.1.1.m1.1c">\textbf{SQL}_{2}</annotation></semantics></math></span></th>
<td id="S1.T1.2.2.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:170.7pt;">
<span id="S1.T1.2.2.2.1" class="ltx_inline-block ltx_align_top">
<span id="S1.T1.2.2.2.1.1" class="ltx_p"><span id="S1.T1.2.2.2.1.1.1" class="ltx_text ltx_font_typewriter">SELECT name FROM highschooler WHERE grade = 10</span></span>
</span>
</td>
</tr>
<tr id="S1.T1.2.4.2" class="ltx_tr">
<td id="S1.T1.2.4.2.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb ltx_border_t" style="width:170.7pt;">
<span id="S1.T1.2.4.2.1.1" class="ltx_inline-block ltx_align_top">
<span id="S1.T1.2.4.2.1.1.1" class="ltx_p"><span id="S1.T1.2.4.2.1.1.1.1" class="ltx_text ltx_font_bold">Skeleton:</span> <span id="S1.T1.2.4.2.1.1.1.2" class="ltx_text ltx_font_typewriter">select _ from _ where _</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 1: </span>Two SQL queries with identical SQL skeletons.</figcaption>
</figure>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Preliminaries</h2>

<div id="S2.p1" class="ltx_para">
<p id="S2.p1.15" class="ltx_p">Let <math id="S2.p1.1.m1.1" class="ltx_Math" alttext="\mathbf{q}" display="inline"><semantics id="S2.p1.1.m1.1a"><mi id="S2.p1.1.m1.1.1" xref="S2.p1.1.m1.1.1.cmml">𝐪</mi><annotation-xml encoding="MathML-Content" id="S2.p1.1.m1.1b"><ci id="S2.p1.1.m1.1.1.cmml" xref="S2.p1.1.m1.1.1">𝐪</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.1.m1.1c">\mathbf{q}</annotation></semantics></math> be the sequence of tokens of a natural language question for database <math id="S2.p1.2.m2.1" class="ltx_Math" alttext="\mathbf{D}" display="inline"><semantics id="S2.p1.2.m2.1a"><mi id="S2.p1.2.m2.1.1" xref="S2.p1.2.m2.1.1.cmml">𝐃</mi><annotation-xml encoding="MathML-Content" id="S2.p1.2.m2.1b"><ci id="S2.p1.2.m2.1.1.cmml" xref="S2.p1.2.m2.1.1">𝐃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.2.m2.1c">\mathbf{D}</annotation></semantics></math> with tables <math id="S2.p1.3.m3.4" class="ltx_Math" alttext="\mathbf{t}=t_{1},t_{2},\ldots,t_{T}" display="inline"><semantics id="S2.p1.3.m3.4a"><mrow id="S2.p1.3.m3.4.4" xref="S2.p1.3.m3.4.4.cmml"><mi id="S2.p1.3.m3.4.4.5" xref="S2.p1.3.m3.4.4.5.cmml">𝐭</mi><mo id="S2.p1.3.m3.4.4.4" xref="S2.p1.3.m3.4.4.4.cmml">=</mo><mrow id="S2.p1.3.m3.4.4.3.3" xref="S2.p1.3.m3.4.4.3.4.cmml"><msub id="S2.p1.3.m3.2.2.1.1.1" xref="S2.p1.3.m3.2.2.1.1.1.cmml"><mi id="S2.p1.3.m3.2.2.1.1.1.2" xref="S2.p1.3.m3.2.2.1.1.1.2.cmml">t</mi><mn id="S2.p1.3.m3.2.2.1.1.1.3" xref="S2.p1.3.m3.2.2.1.1.1.3.cmml">1</mn></msub><mo id="S2.p1.3.m3.4.4.3.3.4" xref="S2.p1.3.m3.4.4.3.4.cmml">,</mo><msub id="S2.p1.3.m3.3.3.2.2.2" xref="S2.p1.3.m3.3.3.2.2.2.cmml"><mi id="S2.p1.3.m3.3.3.2.2.2.2" xref="S2.p1.3.m3.3.3.2.2.2.2.cmml">t</mi><mn id="S2.p1.3.m3.3.3.2.2.2.3" xref="S2.p1.3.m3.3.3.2.2.2.3.cmml">2</mn></msub><mo id="S2.p1.3.m3.4.4.3.3.5" xref="S2.p1.3.m3.4.4.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.p1.3.m3.1.1" xref="S2.p1.3.m3.1.1.cmml">…</mi><mo id="S2.p1.3.m3.4.4.3.3.6" xref="S2.p1.3.m3.4.4.3.4.cmml">,</mo><msub id="S2.p1.3.m3.4.4.3.3.3" xref="S2.p1.3.m3.4.4.3.3.3.cmml"><mi id="S2.p1.3.m3.4.4.3.3.3.2" xref="S2.p1.3.m3.4.4.3.3.3.2.cmml">t</mi><mi id="S2.p1.3.m3.4.4.3.3.3.3" xref="S2.p1.3.m3.4.4.3.3.3.3.cmml">T</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.3.m3.4b"><apply id="S2.p1.3.m3.4.4.cmml" xref="S2.p1.3.m3.4.4"><eq id="S2.p1.3.m3.4.4.4.cmml" xref="S2.p1.3.m3.4.4.4"></eq><ci id="S2.p1.3.m3.4.4.5.cmml" xref="S2.p1.3.m3.4.4.5">𝐭</ci><list id="S2.p1.3.m3.4.4.3.4.cmml" xref="S2.p1.3.m3.4.4.3.3"><apply id="S2.p1.3.m3.2.2.1.1.1.cmml" xref="S2.p1.3.m3.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.p1.3.m3.2.2.1.1.1.1.cmml" xref="S2.p1.3.m3.2.2.1.1.1">subscript</csymbol><ci id="S2.p1.3.m3.2.2.1.1.1.2.cmml" xref="S2.p1.3.m3.2.2.1.1.1.2">𝑡</ci><cn type="integer" id="S2.p1.3.m3.2.2.1.1.1.3.cmml" xref="S2.p1.3.m3.2.2.1.1.1.3">1</cn></apply><apply id="S2.p1.3.m3.3.3.2.2.2.cmml" xref="S2.p1.3.m3.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.p1.3.m3.3.3.2.2.2.1.cmml" xref="S2.p1.3.m3.3.3.2.2.2">subscript</csymbol><ci id="S2.p1.3.m3.3.3.2.2.2.2.cmml" xref="S2.p1.3.m3.3.3.2.2.2.2">𝑡</ci><cn type="integer" id="S2.p1.3.m3.3.3.2.2.2.3.cmml" xref="S2.p1.3.m3.3.3.2.2.2.3">2</cn></apply><ci id="S2.p1.3.m3.1.1.cmml" xref="S2.p1.3.m3.1.1">…</ci><apply id="S2.p1.3.m3.4.4.3.3.3.cmml" xref="S2.p1.3.m3.4.4.3.3.3"><csymbol cd="ambiguous" id="S2.p1.3.m3.4.4.3.3.3.1.cmml" xref="S2.p1.3.m3.4.4.3.3.3">subscript</csymbol><ci id="S2.p1.3.m3.4.4.3.3.3.2.cmml" xref="S2.p1.3.m3.4.4.3.3.3.2">𝑡</ci><ci id="S2.p1.3.m3.4.4.3.3.3.3.cmml" xref="S2.p1.3.m3.4.4.3.3.3.3">𝑇</ci></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.3.m3.4c">\mathbf{t}=t_{1},t_{2},\ldots,t_{T}</annotation></semantics></math> and columns <math id="S2.p1.4.m4.6" class="ltx_Math" alttext="\mathbf{c}=c_{1}^{1},c_{2}^{1},\ldots,c_{j}^{i},\ldots,\allowbreak c_{C_{T}}^{T}" display="inline"><semantics id="S2.p1.4.m4.6a"><mrow id="S2.p1.4.m4.6.6" xref="S2.p1.4.m4.6.6.cmml"><mi id="S2.p1.4.m4.6.6.6" xref="S2.p1.4.m4.6.6.6.cmml">𝐜</mi><mo id="S2.p1.4.m4.6.6.5" xref="S2.p1.4.m4.6.6.5.cmml">=</mo><mrow id="S2.p1.4.m4.6.6.4.4" xref="S2.p1.4.m4.6.6.4.5.cmml"><msubsup id="S2.p1.4.m4.3.3.1.1.1" xref="S2.p1.4.m4.3.3.1.1.1.cmml"><mi id="S2.p1.4.m4.3.3.1.1.1.2.2" xref="S2.p1.4.m4.3.3.1.1.1.2.2.cmml">c</mi><mn id="S2.p1.4.m4.3.3.1.1.1.2.3" xref="S2.p1.4.m4.3.3.1.1.1.2.3.cmml">1</mn><mn id="S2.p1.4.m4.3.3.1.1.1.3" xref="S2.p1.4.m4.3.3.1.1.1.3.cmml">1</mn></msubsup><mo id="S2.p1.4.m4.6.6.4.4.5" xref="S2.p1.4.m4.6.6.4.5.cmml">,</mo><msubsup id="S2.p1.4.m4.4.4.2.2.2" xref="S2.p1.4.m4.4.4.2.2.2.cmml"><mi id="S2.p1.4.m4.4.4.2.2.2.2.2" xref="S2.p1.4.m4.4.4.2.2.2.2.2.cmml">c</mi><mn id="S2.p1.4.m4.4.4.2.2.2.2.3" xref="S2.p1.4.m4.4.4.2.2.2.2.3.cmml">2</mn><mn id="S2.p1.4.m4.4.4.2.2.2.3" xref="S2.p1.4.m4.4.4.2.2.2.3.cmml">1</mn></msubsup><mo id="S2.p1.4.m4.6.6.4.4.6" xref="S2.p1.4.m4.6.6.4.5.cmml">,</mo><mi mathvariant="normal" id="S2.p1.4.m4.1.1" xref="S2.p1.4.m4.1.1.cmml">…</mi><mo id="S2.p1.4.m4.6.6.4.4.7" xref="S2.p1.4.m4.6.6.4.5.cmml">,</mo><msubsup id="S2.p1.4.m4.5.5.3.3.3" xref="S2.p1.4.m4.5.5.3.3.3.cmml"><mi id="S2.p1.4.m4.5.5.3.3.3.2.2" xref="S2.p1.4.m4.5.5.3.3.3.2.2.cmml">c</mi><mi id="S2.p1.4.m4.5.5.3.3.3.2.3" xref="S2.p1.4.m4.5.5.3.3.3.2.3.cmml">j</mi><mi id="S2.p1.4.m4.5.5.3.3.3.3" xref="S2.p1.4.m4.5.5.3.3.3.3.cmml">i</mi></msubsup><mo id="S2.p1.4.m4.6.6.4.4.8" xref="S2.p1.4.m4.6.6.4.5.cmml">,</mo><mi mathvariant="normal" id="S2.p1.4.m4.2.2" xref="S2.p1.4.m4.2.2.cmml">…</mi><mo id="S2.p1.4.m4.6.6.4.4.9" xref="S2.p1.4.m4.6.6.4.5.cmml">,</mo><msubsup id="S2.p1.4.m4.6.6.4.4.4" xref="S2.p1.4.m4.6.6.4.4.4.cmml"><mi id="S2.p1.4.m4.6.6.4.4.4.2.2" xref="S2.p1.4.m4.6.6.4.4.4.2.2.cmml">c</mi><msub id="S2.p1.4.m4.6.6.4.4.4.2.3" xref="S2.p1.4.m4.6.6.4.4.4.2.3.cmml"><mi id="S2.p1.4.m4.6.6.4.4.4.2.3.2" xref="S2.p1.4.m4.6.6.4.4.4.2.3.2.cmml">C</mi><mi id="S2.p1.4.m4.6.6.4.4.4.2.3.3" xref="S2.p1.4.m4.6.6.4.4.4.2.3.3.cmml">T</mi></msub><mi id="S2.p1.4.m4.6.6.4.4.4.3" xref="S2.p1.4.m4.6.6.4.4.4.3.cmml">T</mi></msubsup></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.4.m4.6b"><apply id="S2.p1.4.m4.6.6.cmml" xref="S2.p1.4.m4.6.6"><eq id="S2.p1.4.m4.6.6.5.cmml" xref="S2.p1.4.m4.6.6.5"></eq><ci id="S2.p1.4.m4.6.6.6.cmml" xref="S2.p1.4.m4.6.6.6">𝐜</ci><list id="S2.p1.4.m4.6.6.4.5.cmml" xref="S2.p1.4.m4.6.6.4.4"><apply id="S2.p1.4.m4.3.3.1.1.1.cmml" xref="S2.p1.4.m4.3.3.1.1.1"><csymbol cd="ambiguous" id="S2.p1.4.m4.3.3.1.1.1.1.cmml" xref="S2.p1.4.m4.3.3.1.1.1">superscript</csymbol><apply id="S2.p1.4.m4.3.3.1.1.1.2.cmml" xref="S2.p1.4.m4.3.3.1.1.1"><csymbol cd="ambiguous" id="S2.p1.4.m4.3.3.1.1.1.2.1.cmml" xref="S2.p1.4.m4.3.3.1.1.1">subscript</csymbol><ci id="S2.p1.4.m4.3.3.1.1.1.2.2.cmml" xref="S2.p1.4.m4.3.3.1.1.1.2.2">𝑐</ci><cn type="integer" id="S2.p1.4.m4.3.3.1.1.1.2.3.cmml" xref="S2.p1.4.m4.3.3.1.1.1.2.3">1</cn></apply><cn type="integer" id="S2.p1.4.m4.3.3.1.1.1.3.cmml" xref="S2.p1.4.m4.3.3.1.1.1.3">1</cn></apply><apply id="S2.p1.4.m4.4.4.2.2.2.cmml" xref="S2.p1.4.m4.4.4.2.2.2"><csymbol cd="ambiguous" id="S2.p1.4.m4.4.4.2.2.2.1.cmml" xref="S2.p1.4.m4.4.4.2.2.2">superscript</csymbol><apply id="S2.p1.4.m4.4.4.2.2.2.2.cmml" xref="S2.p1.4.m4.4.4.2.2.2"><csymbol cd="ambiguous" id="S2.p1.4.m4.4.4.2.2.2.2.1.cmml" xref="S2.p1.4.m4.4.4.2.2.2">subscript</csymbol><ci id="S2.p1.4.m4.4.4.2.2.2.2.2.cmml" xref="S2.p1.4.m4.4.4.2.2.2.2.2">𝑐</ci><cn type="integer" id="S2.p1.4.m4.4.4.2.2.2.2.3.cmml" xref="S2.p1.4.m4.4.4.2.2.2.2.3">2</cn></apply><cn type="integer" id="S2.p1.4.m4.4.4.2.2.2.3.cmml" xref="S2.p1.4.m4.4.4.2.2.2.3">1</cn></apply><ci id="S2.p1.4.m4.1.1.cmml" xref="S2.p1.4.m4.1.1">…</ci><apply id="S2.p1.4.m4.5.5.3.3.3.cmml" xref="S2.p1.4.m4.5.5.3.3.3"><csymbol cd="ambiguous" id="S2.p1.4.m4.5.5.3.3.3.1.cmml" xref="S2.p1.4.m4.5.5.3.3.3">superscript</csymbol><apply id="S2.p1.4.m4.5.5.3.3.3.2.cmml" xref="S2.p1.4.m4.5.5.3.3.3"><csymbol cd="ambiguous" id="S2.p1.4.m4.5.5.3.3.3.2.1.cmml" xref="S2.p1.4.m4.5.5.3.3.3">subscript</csymbol><ci id="S2.p1.4.m4.5.5.3.3.3.2.2.cmml" xref="S2.p1.4.m4.5.5.3.3.3.2.2">𝑐</ci><ci id="S2.p1.4.m4.5.5.3.3.3.2.3.cmml" xref="S2.p1.4.m4.5.5.3.3.3.2.3">𝑗</ci></apply><ci id="S2.p1.4.m4.5.5.3.3.3.3.cmml" xref="S2.p1.4.m4.5.5.3.3.3.3">𝑖</ci></apply><ci id="S2.p1.4.m4.2.2.cmml" xref="S2.p1.4.m4.2.2">…</ci><apply id="S2.p1.4.m4.6.6.4.4.4.cmml" xref="S2.p1.4.m4.6.6.4.4.4"><csymbol cd="ambiguous" id="S2.p1.4.m4.6.6.4.4.4.1.cmml" xref="S2.p1.4.m4.6.6.4.4.4">superscript</csymbol><apply id="S2.p1.4.m4.6.6.4.4.4.2.cmml" xref="S2.p1.4.m4.6.6.4.4.4"><csymbol cd="ambiguous" id="S2.p1.4.m4.6.6.4.4.4.2.1.cmml" xref="S2.p1.4.m4.6.6.4.4.4">subscript</csymbol><ci id="S2.p1.4.m4.6.6.4.4.4.2.2.cmml" xref="S2.p1.4.m4.6.6.4.4.4.2.2">𝑐</ci><apply id="S2.p1.4.m4.6.6.4.4.4.2.3.cmml" xref="S2.p1.4.m4.6.6.4.4.4.2.3"><csymbol cd="ambiguous" id="S2.p1.4.m4.6.6.4.4.4.2.3.1.cmml" xref="S2.p1.4.m4.6.6.4.4.4.2.3">subscript</csymbol><ci id="S2.p1.4.m4.6.6.4.4.4.2.3.2.cmml" xref="S2.p1.4.m4.6.6.4.4.4.2.3.2">𝐶</ci><ci id="S2.p1.4.m4.6.6.4.4.4.2.3.3.cmml" xref="S2.p1.4.m4.6.6.4.4.4.2.3.3">𝑇</ci></apply></apply><ci id="S2.p1.4.m4.6.6.4.4.4.3.cmml" xref="S2.p1.4.m4.6.6.4.4.4.3">𝑇</ci></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.4.m4.6c">\mathbf{c}=c_{1}^{1},c_{2}^{1},\ldots,c_{j}^{i},\ldots,\allowbreak c_{C_{T}}^{T}</annotation></semantics></math>, where <math id="S2.p1.5.m5.1" class="ltx_Math" alttext="c^{i}_{j}" display="inline"><semantics id="S2.p1.5.m5.1a"><msubsup id="S2.p1.5.m5.1.1" xref="S2.p1.5.m5.1.1.cmml"><mi id="S2.p1.5.m5.1.1.2.2" xref="S2.p1.5.m5.1.1.2.2.cmml">c</mi><mi id="S2.p1.5.m5.1.1.3" xref="S2.p1.5.m5.1.1.3.cmml">j</mi><mi id="S2.p1.5.m5.1.1.2.3" xref="S2.p1.5.m5.1.1.2.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.p1.5.m5.1b"><apply id="S2.p1.5.m5.1.1.cmml" xref="S2.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S2.p1.5.m5.1.1.1.cmml" xref="S2.p1.5.m5.1.1">subscript</csymbol><apply id="S2.p1.5.m5.1.1.2.cmml" xref="S2.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S2.p1.5.m5.1.1.2.1.cmml" xref="S2.p1.5.m5.1.1">superscript</csymbol><ci id="S2.p1.5.m5.1.1.2.2.cmml" xref="S2.p1.5.m5.1.1.2.2">𝑐</ci><ci id="S2.p1.5.m5.1.1.2.3.cmml" xref="S2.p1.5.m5.1.1.2.3">𝑖</ci></apply><ci id="S2.p1.5.m5.1.1.3.cmml" xref="S2.p1.5.m5.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.5.m5.1c">c^{i}_{j}</annotation></semantics></math> is the <math id="S2.p1.6.m6.1" class="ltx_Math" alttext="j" display="inline"><semantics id="S2.p1.6.m6.1a"><mi id="S2.p1.6.m6.1.1" xref="S2.p1.6.m6.1.1.cmml">j</mi><annotation-xml encoding="MathML-Content" id="S2.p1.6.m6.1b"><ci id="S2.p1.6.m6.1.1.cmml" xref="S2.p1.6.m6.1.1">𝑗</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.6.m6.1c">j</annotation></semantics></math>-th column of table <math id="S2.p1.7.m7.1" class="ltx_Math" alttext="t_{i}" display="inline"><semantics id="S2.p1.7.m7.1a"><msub id="S2.p1.7.m7.1.1" xref="S2.p1.7.m7.1.1.cmml"><mi id="S2.p1.7.m7.1.1.2" xref="S2.p1.7.m7.1.1.2.cmml">t</mi><mi id="S2.p1.7.m7.1.1.3" xref="S2.p1.7.m7.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.p1.7.m7.1b"><apply id="S2.p1.7.m7.1.1.cmml" xref="S2.p1.7.m7.1.1"><csymbol cd="ambiguous" id="S2.p1.7.m7.1.1.1.cmml" xref="S2.p1.7.m7.1.1">subscript</csymbol><ci id="S2.p1.7.m7.1.1.2.cmml" xref="S2.p1.7.m7.1.1.2">𝑡</ci><ci id="S2.p1.7.m7.1.1.3.cmml" xref="S2.p1.7.m7.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.7.m7.1c">t_{i}</annotation></semantics></math> and <math id="S2.p1.8.m8.1" class="ltx_Math" alttext="C_{i}\in\mathbb{N}" display="inline"><semantics id="S2.p1.8.m8.1a"><mrow id="S2.p1.8.m8.1.1" xref="S2.p1.8.m8.1.1.cmml"><msub id="S2.p1.8.m8.1.1.2" xref="S2.p1.8.m8.1.1.2.cmml"><mi id="S2.p1.8.m8.1.1.2.2" xref="S2.p1.8.m8.1.1.2.2.cmml">C</mi><mi id="S2.p1.8.m8.1.1.2.3" xref="S2.p1.8.m8.1.1.2.3.cmml">i</mi></msub><mo id="S2.p1.8.m8.1.1.1" xref="S2.p1.8.m8.1.1.1.cmml">∈</mo><mi id="S2.p1.8.m8.1.1.3" xref="S2.p1.8.m8.1.1.3.cmml">ℕ</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.8.m8.1b"><apply id="S2.p1.8.m8.1.1.cmml" xref="S2.p1.8.m8.1.1"><in id="S2.p1.8.m8.1.1.1.cmml" xref="S2.p1.8.m8.1.1.1"></in><apply id="S2.p1.8.m8.1.1.2.cmml" xref="S2.p1.8.m8.1.1.2"><csymbol cd="ambiguous" id="S2.p1.8.m8.1.1.2.1.cmml" xref="S2.p1.8.m8.1.1.2">subscript</csymbol><ci id="S2.p1.8.m8.1.1.2.2.cmml" xref="S2.p1.8.m8.1.1.2.2">𝐶</ci><ci id="S2.p1.8.m8.1.1.2.3.cmml" xref="S2.p1.8.m8.1.1.2.3">𝑖</ci></apply><ci id="S2.p1.8.m8.1.1.3.cmml" xref="S2.p1.8.m8.1.1.3">ℕ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.8.m8.1c">C_{i}\in\mathbb{N}</annotation></semantics></math> is the total number of columns in table <math id="S2.p1.9.m9.1" class="ltx_Math" alttext="t_{i}" display="inline"><semantics id="S2.p1.9.m9.1a"><msub id="S2.p1.9.m9.1.1" xref="S2.p1.9.m9.1.1.cmml"><mi id="S2.p1.9.m9.1.1.2" xref="S2.p1.9.m9.1.1.2.cmml">t</mi><mi id="S2.p1.9.m9.1.1.3" xref="S2.p1.9.m9.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.p1.9.m9.1b"><apply id="S2.p1.9.m9.1.1.cmml" xref="S2.p1.9.m9.1.1"><csymbol cd="ambiguous" id="S2.p1.9.m9.1.1.1.cmml" xref="S2.p1.9.m9.1.1">subscript</csymbol><ci id="S2.p1.9.m9.1.1.2.cmml" xref="S2.p1.9.m9.1.1.2">𝑡</ci><ci id="S2.p1.9.m9.1.1.3.cmml" xref="S2.p1.9.m9.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.9.m9.1c">t_{i}</annotation></semantics></math>. Furthermore, let <math id="S2.p1.10.m10.4" class="ltx_Math" alttext="\mathbf{v_{\mathbf{D}}}=\left\{v_{c_{1}^{1}},v_{c_{2}^{1}},\ldots,v_{c_{C_{T}}^{T}}\right\}" display="inline"><semantics id="S2.p1.10.m10.4a"><mrow id="S2.p1.10.m10.4.4" xref="S2.p1.10.m10.4.4.cmml"><msub id="S2.p1.10.m10.4.4.5" xref="S2.p1.10.m10.4.4.5.cmml"><mi id="S2.p1.10.m10.4.4.5.2" xref="S2.p1.10.m10.4.4.5.2.cmml">𝐯</mi><mi id="S2.p1.10.m10.4.4.5.3" xref="S2.p1.10.m10.4.4.5.3.cmml">𝐃</mi></msub><mo id="S2.p1.10.m10.4.4.4" xref="S2.p1.10.m10.4.4.4.cmml">=</mo><mrow id="S2.p1.10.m10.4.4.3.3" xref="S2.p1.10.m10.4.4.3.4.cmml"><mo id="S2.p1.10.m10.4.4.3.3.4" xref="S2.p1.10.m10.4.4.3.4.cmml">{</mo><msub id="S2.p1.10.m10.2.2.1.1.1" xref="S2.p1.10.m10.2.2.1.1.1.cmml"><mi id="S2.p1.10.m10.2.2.1.1.1.2" xref="S2.p1.10.m10.2.2.1.1.1.2.cmml">v</mi><msubsup id="S2.p1.10.m10.2.2.1.1.1.3" xref="S2.p1.10.m10.2.2.1.1.1.3.cmml"><mi id="S2.p1.10.m10.2.2.1.1.1.3.2.2" xref="S2.p1.10.m10.2.2.1.1.1.3.2.2.cmml">c</mi><mn id="S2.p1.10.m10.2.2.1.1.1.3.2.3" xref="S2.p1.10.m10.2.2.1.1.1.3.2.3.cmml">1</mn><mn id="S2.p1.10.m10.2.2.1.1.1.3.3" xref="S2.p1.10.m10.2.2.1.1.1.3.3.cmml">1</mn></msubsup></msub><mo id="S2.p1.10.m10.4.4.3.3.5" xref="S2.p1.10.m10.4.4.3.4.cmml">,</mo><msub id="S2.p1.10.m10.3.3.2.2.2" xref="S2.p1.10.m10.3.3.2.2.2.cmml"><mi id="S2.p1.10.m10.3.3.2.2.2.2" xref="S2.p1.10.m10.3.3.2.2.2.2.cmml">v</mi><msubsup id="S2.p1.10.m10.3.3.2.2.2.3" xref="S2.p1.10.m10.3.3.2.2.2.3.cmml"><mi id="S2.p1.10.m10.3.3.2.2.2.3.2.2" xref="S2.p1.10.m10.3.3.2.2.2.3.2.2.cmml">c</mi><mn id="S2.p1.10.m10.3.3.2.2.2.3.2.3" xref="S2.p1.10.m10.3.3.2.2.2.3.2.3.cmml">2</mn><mn id="S2.p1.10.m10.3.3.2.2.2.3.3" xref="S2.p1.10.m10.3.3.2.2.2.3.3.cmml">1</mn></msubsup></msub><mo id="S2.p1.10.m10.4.4.3.3.6" xref="S2.p1.10.m10.4.4.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.p1.10.m10.1.1" xref="S2.p1.10.m10.1.1.cmml">…</mi><mo id="S2.p1.10.m10.4.4.3.3.7" xref="S2.p1.10.m10.4.4.3.4.cmml">,</mo><msub id="S2.p1.10.m10.4.4.3.3.3" xref="S2.p1.10.m10.4.4.3.3.3.cmml"><mi id="S2.p1.10.m10.4.4.3.3.3.2" xref="S2.p1.10.m10.4.4.3.3.3.2.cmml">v</mi><msubsup id="S2.p1.10.m10.4.4.3.3.3.3" xref="S2.p1.10.m10.4.4.3.3.3.3.cmml"><mi id="S2.p1.10.m10.4.4.3.3.3.3.2.2" xref="S2.p1.10.m10.4.4.3.3.3.3.2.2.cmml">c</mi><msub id="S2.p1.10.m10.4.4.3.3.3.3.2.3" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3.cmml"><mi id="S2.p1.10.m10.4.4.3.3.3.3.2.3.2" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3.2.cmml">C</mi><mi id="S2.p1.10.m10.4.4.3.3.3.3.2.3.3" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3.3.cmml">T</mi></msub><mi id="S2.p1.10.m10.4.4.3.3.3.3.3" xref="S2.p1.10.m10.4.4.3.3.3.3.3.cmml">T</mi></msubsup></msub><mo id="S2.p1.10.m10.4.4.3.3.8" xref="S2.p1.10.m10.4.4.3.4.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.10.m10.4b"><apply id="S2.p1.10.m10.4.4.cmml" xref="S2.p1.10.m10.4.4"><eq id="S2.p1.10.m10.4.4.4.cmml" xref="S2.p1.10.m10.4.4.4"></eq><apply id="S2.p1.10.m10.4.4.5.cmml" xref="S2.p1.10.m10.4.4.5"><csymbol cd="ambiguous" id="S2.p1.10.m10.4.4.5.1.cmml" xref="S2.p1.10.m10.4.4.5">subscript</csymbol><ci id="S2.p1.10.m10.4.4.5.2.cmml" xref="S2.p1.10.m10.4.4.5.2">𝐯</ci><ci id="S2.p1.10.m10.4.4.5.3.cmml" xref="S2.p1.10.m10.4.4.5.3">𝐃</ci></apply><set id="S2.p1.10.m10.4.4.3.4.cmml" xref="S2.p1.10.m10.4.4.3.3"><apply id="S2.p1.10.m10.2.2.1.1.1.cmml" xref="S2.p1.10.m10.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.p1.10.m10.2.2.1.1.1.1.cmml" xref="S2.p1.10.m10.2.2.1.1.1">subscript</csymbol><ci id="S2.p1.10.m10.2.2.1.1.1.2.cmml" xref="S2.p1.10.m10.2.2.1.1.1.2">𝑣</ci><apply id="S2.p1.10.m10.2.2.1.1.1.3.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.2.2.1.1.1.3.1.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3">superscript</csymbol><apply id="S2.p1.10.m10.2.2.1.1.1.3.2.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.2.2.1.1.1.3.2.1.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3">subscript</csymbol><ci id="S2.p1.10.m10.2.2.1.1.1.3.2.2.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3.2.2">𝑐</ci><cn type="integer" id="S2.p1.10.m10.2.2.1.1.1.3.2.3.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3.2.3">1</cn></apply><cn type="integer" id="S2.p1.10.m10.2.2.1.1.1.3.3.cmml" xref="S2.p1.10.m10.2.2.1.1.1.3.3">1</cn></apply></apply><apply id="S2.p1.10.m10.3.3.2.2.2.cmml" xref="S2.p1.10.m10.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.p1.10.m10.3.3.2.2.2.1.cmml" xref="S2.p1.10.m10.3.3.2.2.2">subscript</csymbol><ci id="S2.p1.10.m10.3.3.2.2.2.2.cmml" xref="S2.p1.10.m10.3.3.2.2.2.2">𝑣</ci><apply id="S2.p1.10.m10.3.3.2.2.2.3.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.3.3.2.2.2.3.1.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3">superscript</csymbol><apply id="S2.p1.10.m10.3.3.2.2.2.3.2.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.3.3.2.2.2.3.2.1.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3">subscript</csymbol><ci id="S2.p1.10.m10.3.3.2.2.2.3.2.2.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3.2.2">𝑐</ci><cn type="integer" id="S2.p1.10.m10.3.3.2.2.2.3.2.3.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3.2.3">2</cn></apply><cn type="integer" id="S2.p1.10.m10.3.3.2.2.2.3.3.cmml" xref="S2.p1.10.m10.3.3.2.2.2.3.3">1</cn></apply></apply><ci id="S2.p1.10.m10.1.1.cmml" xref="S2.p1.10.m10.1.1">…</ci><apply id="S2.p1.10.m10.4.4.3.3.3.cmml" xref="S2.p1.10.m10.4.4.3.3.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.4.4.3.3.3.1.cmml" xref="S2.p1.10.m10.4.4.3.3.3">subscript</csymbol><ci id="S2.p1.10.m10.4.4.3.3.3.2.cmml" xref="S2.p1.10.m10.4.4.3.3.3.2">𝑣</ci><apply id="S2.p1.10.m10.4.4.3.3.3.3.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.4.4.3.3.3.3.1.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3">superscript</csymbol><apply id="S2.p1.10.m10.4.4.3.3.3.3.2.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.4.4.3.3.3.3.2.1.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3">subscript</csymbol><ci id="S2.p1.10.m10.4.4.3.3.3.3.2.2.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3.2.2">𝑐</ci><apply id="S2.p1.10.m10.4.4.3.3.3.3.2.3.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3"><csymbol cd="ambiguous" id="S2.p1.10.m10.4.4.3.3.3.3.2.3.1.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3">subscript</csymbol><ci id="S2.p1.10.m10.4.4.3.3.3.3.2.3.2.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3.2">𝐶</ci><ci id="S2.p1.10.m10.4.4.3.3.3.3.2.3.3.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3.2.3.3">𝑇</ci></apply></apply><ci id="S2.p1.10.m10.4.4.3.3.3.3.3.cmml" xref="S2.p1.10.m10.4.4.3.3.3.3.3">𝑇</ci></apply></apply></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.10.m10.4c">\mathbf{v_{\mathbf{D}}}=\left\{v_{c_{1}^{1}},v_{c_{2}^{1}},\ldots,v_{c_{C_{T}}^{T}}\right\}</annotation></semantics></math> be the set of all values associated with the database <math id="S2.p1.11.m11.1" class="ltx_Math" alttext="\mathbf{D}" display="inline"><semantics id="S2.p1.11.m11.1a"><mi id="S2.p1.11.m11.1.1" xref="S2.p1.11.m11.1.1.cmml">𝐃</mi><annotation-xml encoding="MathML-Content" id="S2.p1.11.m11.1b"><ci id="S2.p1.11.m11.1.1.cmml" xref="S2.p1.11.m11.1.1">𝐃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.11.m11.1c">\mathbf{D}</annotation></semantics></math> s.t. <math id="S2.p1.12.m12.3" class="ltx_Math" alttext="v_{c_{1}^{1}},\ldots,v_{c_{C_{T}}^{T}}" display="inline"><semantics id="S2.p1.12.m12.3a"><mrow id="S2.p1.12.m12.3.3.2" xref="S2.p1.12.m12.3.3.3.cmml"><msub id="S2.p1.12.m12.2.2.1.1" xref="S2.p1.12.m12.2.2.1.1.cmml"><mi id="S2.p1.12.m12.2.2.1.1.2" xref="S2.p1.12.m12.2.2.1.1.2.cmml">v</mi><msubsup id="S2.p1.12.m12.2.2.1.1.3" xref="S2.p1.12.m12.2.2.1.1.3.cmml"><mi id="S2.p1.12.m12.2.2.1.1.3.2.2" xref="S2.p1.12.m12.2.2.1.1.3.2.2.cmml">c</mi><mn id="S2.p1.12.m12.2.2.1.1.3.2.3" xref="S2.p1.12.m12.2.2.1.1.3.2.3.cmml">1</mn><mn id="S2.p1.12.m12.2.2.1.1.3.3" xref="S2.p1.12.m12.2.2.1.1.3.3.cmml">1</mn></msubsup></msub><mo id="S2.p1.12.m12.3.3.2.3" xref="S2.p1.12.m12.3.3.3.cmml">,</mo><mi mathvariant="normal" id="S2.p1.12.m12.1.1" xref="S2.p1.12.m12.1.1.cmml">…</mi><mo id="S2.p1.12.m12.3.3.2.4" xref="S2.p1.12.m12.3.3.3.cmml">,</mo><msub id="S2.p1.12.m12.3.3.2.2" xref="S2.p1.12.m12.3.3.2.2.cmml"><mi id="S2.p1.12.m12.3.3.2.2.2" xref="S2.p1.12.m12.3.3.2.2.2.cmml">v</mi><msubsup id="S2.p1.12.m12.3.3.2.2.3" xref="S2.p1.12.m12.3.3.2.2.3.cmml"><mi id="S2.p1.12.m12.3.3.2.2.3.2.2" xref="S2.p1.12.m12.3.3.2.2.3.2.2.cmml">c</mi><msub id="S2.p1.12.m12.3.3.2.2.3.2.3" xref="S2.p1.12.m12.3.3.2.2.3.2.3.cmml"><mi id="S2.p1.12.m12.3.3.2.2.3.2.3.2" xref="S2.p1.12.m12.3.3.2.2.3.2.3.2.cmml">C</mi><mi id="S2.p1.12.m12.3.3.2.2.3.2.3.3" xref="S2.p1.12.m12.3.3.2.2.3.2.3.3.cmml">T</mi></msub><mi id="S2.p1.12.m12.3.3.2.2.3.3" xref="S2.p1.12.m12.3.3.2.2.3.3.cmml">T</mi></msubsup></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.12.m12.3b"><list id="S2.p1.12.m12.3.3.3.cmml" xref="S2.p1.12.m12.3.3.2"><apply id="S2.p1.12.m12.2.2.1.1.cmml" xref="S2.p1.12.m12.2.2.1.1"><csymbol cd="ambiguous" id="S2.p1.12.m12.2.2.1.1.1.cmml" xref="S2.p1.12.m12.2.2.1.1">subscript</csymbol><ci id="S2.p1.12.m12.2.2.1.1.2.cmml" xref="S2.p1.12.m12.2.2.1.1.2">𝑣</ci><apply id="S2.p1.12.m12.2.2.1.1.3.cmml" xref="S2.p1.12.m12.2.2.1.1.3"><csymbol cd="ambiguous" id="S2.p1.12.m12.2.2.1.1.3.1.cmml" xref="S2.p1.12.m12.2.2.1.1.3">superscript</csymbol><apply id="S2.p1.12.m12.2.2.1.1.3.2.cmml" xref="S2.p1.12.m12.2.2.1.1.3"><csymbol cd="ambiguous" id="S2.p1.12.m12.2.2.1.1.3.2.1.cmml" xref="S2.p1.12.m12.2.2.1.1.3">subscript</csymbol><ci id="S2.p1.12.m12.2.2.1.1.3.2.2.cmml" xref="S2.p1.12.m12.2.2.1.1.3.2.2">𝑐</ci><cn type="integer" id="S2.p1.12.m12.2.2.1.1.3.2.3.cmml" xref="S2.p1.12.m12.2.2.1.1.3.2.3">1</cn></apply><cn type="integer" id="S2.p1.12.m12.2.2.1.1.3.3.cmml" xref="S2.p1.12.m12.2.2.1.1.3.3">1</cn></apply></apply><ci id="S2.p1.12.m12.1.1.cmml" xref="S2.p1.12.m12.1.1">…</ci><apply id="S2.p1.12.m12.3.3.2.2.cmml" xref="S2.p1.12.m12.3.3.2.2"><csymbol cd="ambiguous" id="S2.p1.12.m12.3.3.2.2.1.cmml" xref="S2.p1.12.m12.3.3.2.2">subscript</csymbol><ci id="S2.p1.12.m12.3.3.2.2.2.cmml" xref="S2.p1.12.m12.3.3.2.2.2">𝑣</ci><apply id="S2.p1.12.m12.3.3.2.2.3.cmml" xref="S2.p1.12.m12.3.3.2.2.3"><csymbol cd="ambiguous" id="S2.p1.12.m12.3.3.2.2.3.1.cmml" xref="S2.p1.12.m12.3.3.2.2.3">superscript</csymbol><apply id="S2.p1.12.m12.3.3.2.2.3.2.cmml" xref="S2.p1.12.m12.3.3.2.2.3"><csymbol cd="ambiguous" id="S2.p1.12.m12.3.3.2.2.3.2.1.cmml" xref="S2.p1.12.m12.3.3.2.2.3">subscript</csymbol><ci id="S2.p1.12.m12.3.3.2.2.3.2.2.cmml" xref="S2.p1.12.m12.3.3.2.2.3.2.2">𝑐</ci><apply id="S2.p1.12.m12.3.3.2.2.3.2.3.cmml" xref="S2.p1.12.m12.3.3.2.2.3.2.3"><csymbol cd="ambiguous" id="S2.p1.12.m12.3.3.2.2.3.2.3.1.cmml" xref="S2.p1.12.m12.3.3.2.2.3.2.3">subscript</csymbol><ci id="S2.p1.12.m12.3.3.2.2.3.2.3.2.cmml" xref="S2.p1.12.m12.3.3.2.2.3.2.3.2">𝐶</ci><ci id="S2.p1.12.m12.3.3.2.2.3.2.3.3.cmml" xref="S2.p1.12.m12.3.3.2.2.3.2.3.3">𝑇</ci></apply></apply><ci id="S2.p1.12.m12.3.3.2.2.3.3.cmml" xref="S2.p1.12.m12.3.3.2.2.3.3">𝑇</ci></apply></apply></list></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.12.m12.3c">v_{c_{1}^{1}},\ldots,v_{c_{C_{T}}^{T}}</annotation></semantics></math> are the DB value sets associated with respective columns <math id="S2.p1.13.m13.3" class="ltx_Math" alttext="c_{1}^{1},\ldots,\allowbreak c_{C_{T}}^{T}\in\mathbf{c}" display="inline"><semantics id="S2.p1.13.m13.3a"><mrow id="S2.p1.13.m13.3.3" xref="S2.p1.13.m13.3.3.cmml"><mrow id="S2.p1.13.m13.3.3.2.2" xref="S2.p1.13.m13.3.3.2.3.cmml"><msubsup id="S2.p1.13.m13.2.2.1.1.1" xref="S2.p1.13.m13.2.2.1.1.1.cmml"><mi id="S2.p1.13.m13.2.2.1.1.1.2.2" xref="S2.p1.13.m13.2.2.1.1.1.2.2.cmml">c</mi><mn id="S2.p1.13.m13.2.2.1.1.1.2.3" xref="S2.p1.13.m13.2.2.1.1.1.2.3.cmml">1</mn><mn id="S2.p1.13.m13.2.2.1.1.1.3" xref="S2.p1.13.m13.2.2.1.1.1.3.cmml">1</mn></msubsup><mo id="S2.p1.13.m13.3.3.2.2.3" xref="S2.p1.13.m13.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.p1.13.m13.1.1" xref="S2.p1.13.m13.1.1.cmml">…</mi><mo id="S2.p1.13.m13.3.3.2.2.4" xref="S2.p1.13.m13.3.3.2.3.cmml">,</mo><msubsup id="S2.p1.13.m13.3.3.2.2.2" xref="S2.p1.13.m13.3.3.2.2.2.cmml"><mi id="S2.p1.13.m13.3.3.2.2.2.2.2" xref="S2.p1.13.m13.3.3.2.2.2.2.2.cmml">c</mi><msub id="S2.p1.13.m13.3.3.2.2.2.2.3" xref="S2.p1.13.m13.3.3.2.2.2.2.3.cmml"><mi id="S2.p1.13.m13.3.3.2.2.2.2.3.2" xref="S2.p1.13.m13.3.3.2.2.2.2.3.2.cmml">C</mi><mi id="S2.p1.13.m13.3.3.2.2.2.2.3.3" xref="S2.p1.13.m13.3.3.2.2.2.2.3.3.cmml">T</mi></msub><mi id="S2.p1.13.m13.3.3.2.2.2.3" xref="S2.p1.13.m13.3.3.2.2.2.3.cmml">T</mi></msubsup></mrow><mo id="S2.p1.13.m13.3.3.3" xref="S2.p1.13.m13.3.3.3.cmml">∈</mo><mi id="S2.p1.13.m13.3.3.4" xref="S2.p1.13.m13.3.3.4.cmml">𝐜</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.13.m13.3b"><apply id="S2.p1.13.m13.3.3.cmml" xref="S2.p1.13.m13.3.3"><in id="S2.p1.13.m13.3.3.3.cmml" xref="S2.p1.13.m13.3.3.3"></in><list id="S2.p1.13.m13.3.3.2.3.cmml" xref="S2.p1.13.m13.3.3.2.2"><apply id="S2.p1.13.m13.2.2.1.1.1.cmml" xref="S2.p1.13.m13.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.p1.13.m13.2.2.1.1.1.1.cmml" xref="S2.p1.13.m13.2.2.1.1.1">superscript</csymbol><apply id="S2.p1.13.m13.2.2.1.1.1.2.cmml" xref="S2.p1.13.m13.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.p1.13.m13.2.2.1.1.1.2.1.cmml" xref="S2.p1.13.m13.2.2.1.1.1">subscript</csymbol><ci id="S2.p1.13.m13.2.2.1.1.1.2.2.cmml" xref="S2.p1.13.m13.2.2.1.1.1.2.2">𝑐</ci><cn type="integer" id="S2.p1.13.m13.2.2.1.1.1.2.3.cmml" xref="S2.p1.13.m13.2.2.1.1.1.2.3">1</cn></apply><cn type="integer" id="S2.p1.13.m13.2.2.1.1.1.3.cmml" xref="S2.p1.13.m13.2.2.1.1.1.3">1</cn></apply><ci id="S2.p1.13.m13.1.1.cmml" xref="S2.p1.13.m13.1.1">…</ci><apply id="S2.p1.13.m13.3.3.2.2.2.cmml" xref="S2.p1.13.m13.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.p1.13.m13.3.3.2.2.2.1.cmml" xref="S2.p1.13.m13.3.3.2.2.2">superscript</csymbol><apply id="S2.p1.13.m13.3.3.2.2.2.2.cmml" xref="S2.p1.13.m13.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.p1.13.m13.3.3.2.2.2.2.1.cmml" xref="S2.p1.13.m13.3.3.2.2.2">subscript</csymbol><ci id="S2.p1.13.m13.3.3.2.2.2.2.2.cmml" xref="S2.p1.13.m13.3.3.2.2.2.2.2">𝑐</ci><apply id="S2.p1.13.m13.3.3.2.2.2.2.3.cmml" xref="S2.p1.13.m13.3.3.2.2.2.2.3"><csymbol cd="ambiguous" id="S2.p1.13.m13.3.3.2.2.2.2.3.1.cmml" xref="S2.p1.13.m13.3.3.2.2.2.2.3">subscript</csymbol><ci id="S2.p1.13.m13.3.3.2.2.2.2.3.2.cmml" xref="S2.p1.13.m13.3.3.2.2.2.2.3.2">𝐶</ci><ci id="S2.p1.13.m13.3.3.2.2.2.2.3.3.cmml" xref="S2.p1.13.m13.3.3.2.2.2.2.3.3">𝑇</ci></apply></apply><ci id="S2.p1.13.m13.3.3.2.2.2.3.cmml" xref="S2.p1.13.m13.3.3.2.2.2.3">𝑇</ci></apply></list><ci id="S2.p1.13.m13.3.3.4.cmml" xref="S2.p1.13.m13.3.3.4">𝐜</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.13.m13.3c">c_{1}^{1},\ldots,\allowbreak c_{C_{T}}^{T}\in\mathbf{c}</annotation></semantics></math>.
The goal of Text-to-SQL semantic parsing is to predict the SQL query <math id="S2.p1.14.m14.1" class="ltx_Math" alttext="\mathbf{s}" display="inline"><semantics id="S2.p1.14.m14.1a"><mi id="S2.p1.14.m14.1.1" xref="S2.p1.14.m14.1.1.cmml">𝐬</mi><annotation-xml encoding="MathML-Content" id="S2.p1.14.m14.1b"><ci id="S2.p1.14.m14.1.1.cmml" xref="S2.p1.14.m14.1.1">𝐬</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.14.m14.1c">\mathbf{s}</annotation></semantics></math> given the <math id="S2.p1.15.m15.2" class="ltx_Math" alttext="(\mathbf{q},\mathbf{D})" display="inline"><semantics id="S2.p1.15.m15.2a"><mrow id="S2.p1.15.m15.2.3.2" xref="S2.p1.15.m15.2.3.1.cmml"><mo stretchy="false" id="S2.p1.15.m15.2.3.2.1" xref="S2.p1.15.m15.2.3.1.cmml">(</mo><mi id="S2.p1.15.m15.1.1" xref="S2.p1.15.m15.1.1.cmml">𝐪</mi><mo id="S2.p1.15.m15.2.3.2.2" xref="S2.p1.15.m15.2.3.1.cmml">,</mo><mi id="S2.p1.15.m15.2.2" xref="S2.p1.15.m15.2.2.cmml">𝐃</mi><mo stretchy="false" id="S2.p1.15.m15.2.3.2.3" xref="S2.p1.15.m15.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.p1.15.m15.2b"><interval closure="open" id="S2.p1.15.m15.2.3.1.cmml" xref="S2.p1.15.m15.2.3.2"><ci id="S2.p1.15.m15.1.1.cmml" xref="S2.p1.15.m15.1.1">𝐪</ci><ci id="S2.p1.15.m15.2.2.cmml" xref="S2.p1.15.m15.2.2">𝐃</ci></interval></annotation-xml><annotation encoding="application/x-tex" id="S2.p1.15.m15.2c">(\mathbf{q},\mathbf{D})</annotation></semantics></math> combination, as follows:</p>
<table id="A5.EGx1" class="ltx_equationgroup ltx_eqn_align ltx_eqn_table">

<tbody id="S2.E1"><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S2.E1.m1.3" class="ltx_Math" alttext="\displaystyle\mathbf{s}=\operatorname*{\arg\max}_{\mathbf{s}}p\left(\mathbf{s}\mid\mathbf{q},\mathbf{D}\right)" display="inline"><semantics id="S2.E1.m1.3a"><mrow id="S2.E1.m1.3.3" xref="S2.E1.m1.3.3.cmml"><mi id="S2.E1.m1.3.3.3" xref="S2.E1.m1.3.3.3.cmml">𝐬</mi><mo id="S2.E1.m1.3.3.2" xref="S2.E1.m1.3.3.2.cmml">=</mo><mrow id="S2.E1.m1.3.3.1" xref="S2.E1.m1.3.3.1.cmml"><mrow id="S2.E1.m1.3.3.1.3" xref="S2.E1.m1.3.3.1.3.cmml"><munder id="S2.E1.m1.3.3.1.3.1" xref="S2.E1.m1.3.3.1.3.1.cmml"><mrow id="S2.E1.m1.3.3.1.3.1.2" xref="S2.E1.m1.3.3.1.3.1.2.cmml"><mi id="S2.E1.m1.3.3.1.3.1.2.1" xref="S2.E1.m1.3.3.1.3.1.2.1.cmml">arg</mi><mo lspace="0.167em" id="S2.E1.m1.3.3.1.3.1.2a" xref="S2.E1.m1.3.3.1.3.1.2.cmml">⁡</mo><mi id="S2.E1.m1.3.3.1.3.1.2.2" xref="S2.E1.m1.3.3.1.3.1.2.2.cmml">max</mi></mrow><mi id="S2.E1.m1.3.3.1.3.1.3" xref="S2.E1.m1.3.3.1.3.1.3.cmml">𝐬</mi></munder><mo lspace="0.167em" id="S2.E1.m1.3.3.1.3a" xref="S2.E1.m1.3.3.1.3.cmml">⁡</mo><mi id="S2.E1.m1.3.3.1.3.2" xref="S2.E1.m1.3.3.1.3.2.cmml">p</mi></mrow><mo lspace="0em" rspace="0em" id="S2.E1.m1.3.3.1.2" xref="S2.E1.m1.3.3.1.2.cmml">​</mo><mrow id="S2.E1.m1.3.3.1.1.1" xref="S2.E1.m1.3.3.1.1.1.1.cmml"><mo id="S2.E1.m1.3.3.1.1.1.2" xref="S2.E1.m1.3.3.1.1.1.1.cmml">(</mo><mrow id="S2.E1.m1.3.3.1.1.1.1" xref="S2.E1.m1.3.3.1.1.1.1.cmml"><mi id="S2.E1.m1.3.3.1.1.1.1.2" xref="S2.E1.m1.3.3.1.1.1.1.2.cmml">𝐬</mi><mo id="S2.E1.m1.3.3.1.1.1.1.1" xref="S2.E1.m1.3.3.1.1.1.1.1.cmml">∣</mo><mrow id="S2.E1.m1.3.3.1.1.1.1.3.2" xref="S2.E1.m1.3.3.1.1.1.1.3.1.cmml"><mi id="S2.E1.m1.1.1" xref="S2.E1.m1.1.1.cmml">𝐪</mi><mo id="S2.E1.m1.3.3.1.1.1.1.3.2.1" xref="S2.E1.m1.3.3.1.1.1.1.3.1.cmml">,</mo><mi id="S2.E1.m1.2.2" xref="S2.E1.m1.2.2.cmml">𝐃</mi></mrow></mrow><mo id="S2.E1.m1.3.3.1.1.1.3" xref="S2.E1.m1.3.3.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.E1.m1.3b"><apply id="S2.E1.m1.3.3.cmml" xref="S2.E1.m1.3.3"><eq id="S2.E1.m1.3.3.2.cmml" xref="S2.E1.m1.3.3.2"></eq><ci id="S2.E1.m1.3.3.3.cmml" xref="S2.E1.m1.3.3.3">𝐬</ci><apply id="S2.E1.m1.3.3.1.cmml" xref="S2.E1.m1.3.3.1"><times id="S2.E1.m1.3.3.1.2.cmml" xref="S2.E1.m1.3.3.1.2"></times><apply id="S2.E1.m1.3.3.1.3.cmml" xref="S2.E1.m1.3.3.1.3"><apply id="S2.E1.m1.3.3.1.3.1.cmml" xref="S2.E1.m1.3.3.1.3.1"><csymbol cd="ambiguous" id="S2.E1.m1.3.3.1.3.1.1.cmml" xref="S2.E1.m1.3.3.1.3.1">subscript</csymbol><apply id="S2.E1.m1.3.3.1.3.1.2.cmml" xref="S2.E1.m1.3.3.1.3.1.2"><arg id="S2.E1.m1.3.3.1.3.1.2.1.cmml" xref="S2.E1.m1.3.3.1.3.1.2.1"></arg><max id="S2.E1.m1.3.3.1.3.1.2.2.cmml" xref="S2.E1.m1.3.3.1.3.1.2.2"></max></apply><ci id="S2.E1.m1.3.3.1.3.1.3.cmml" xref="S2.E1.m1.3.3.1.3.1.3">𝐬</ci></apply><ci id="S2.E1.m1.3.3.1.3.2.cmml" xref="S2.E1.m1.3.3.1.3.2">𝑝</ci></apply><apply id="S2.E1.m1.3.3.1.1.1.1.cmml" xref="S2.E1.m1.3.3.1.1.1"><csymbol cd="latexml" id="S2.E1.m1.3.3.1.1.1.1.1.cmml" xref="S2.E1.m1.3.3.1.1.1.1.1">conditional</csymbol><ci id="S2.E1.m1.3.3.1.1.1.1.2.cmml" xref="S2.E1.m1.3.3.1.1.1.1.2">𝐬</ci><list id="S2.E1.m1.3.3.1.1.1.1.3.1.cmml" xref="S2.E1.m1.3.3.1.1.1.1.3.2"><ci id="S2.E1.m1.1.1.cmml" xref="S2.E1.m1.1.1">𝐪</ci><ci id="S2.E1.m1.2.2.cmml" xref="S2.E1.m1.2.2">𝐃</ci></list></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E1.m1.3c">\displaystyle\mathbf{s}=\operatorname*{\arg\max}_{\mathbf{s}}p\left(\mathbf{s}\mid\mathbf{q},\mathbf{D}\right)</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(1)</span></td>
</tr></tbody>
</table>
<p id="S2.p1.16" class="ltx_p">For in-context learning, we seek to select pertinent input context including few-shot examples, schema, and database values to simplify the task for LLMs.</p>
</div>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Example Selection using Abstract Syntax Trees</h2>

<div id="S3.p1" class="ltx_para">
<p id="S3.p1.4" class="ltx_p">Our goal is to identify the most suitable set of <math id="S3.p1.1.m1.3" class="ltx_Math" alttext="\mathbb{X^{\star}}=\left\{\left(\mathbf{q}^{\star}_{1},\mathbf{s}^{\star}_{1}\right),\ldots,\left(\mathbf{q}^{\star}_{e},\mathbf{s}^{\star}_{e}\right)\right\}" display="inline"><semantics id="S3.p1.1.m1.3a"><mrow id="S3.p1.1.m1.3.3" xref="S3.p1.1.m1.3.3.cmml"><msup id="S3.p1.1.m1.3.3.4" xref="S3.p1.1.m1.3.3.4.cmml"><mi id="S3.p1.1.m1.3.3.4.2" xref="S3.p1.1.m1.3.3.4.2.cmml">𝕏</mi><mo id="S3.p1.1.m1.3.3.4.3" xref="S3.p1.1.m1.3.3.4.3.cmml">⋆</mo></msup><mo id="S3.p1.1.m1.3.3.3" xref="S3.p1.1.m1.3.3.3.cmml">=</mo><mrow id="S3.p1.1.m1.3.3.2.2" xref="S3.p1.1.m1.3.3.2.3.cmml"><mo id="S3.p1.1.m1.3.3.2.2.3" xref="S3.p1.1.m1.3.3.2.3.cmml">{</mo><mrow id="S3.p1.1.m1.2.2.1.1.1.2" xref="S3.p1.1.m1.2.2.1.1.1.3.cmml"><mo id="S3.p1.1.m1.2.2.1.1.1.2.3" xref="S3.p1.1.m1.2.2.1.1.1.3.cmml">(</mo><msubsup id="S3.p1.1.m1.2.2.1.1.1.1.1" xref="S3.p1.1.m1.2.2.1.1.1.1.1.cmml"><mi id="S3.p1.1.m1.2.2.1.1.1.1.1.2.2" xref="S3.p1.1.m1.2.2.1.1.1.1.1.2.2.cmml">𝐪</mi><mn id="S3.p1.1.m1.2.2.1.1.1.1.1.3" xref="S3.p1.1.m1.2.2.1.1.1.1.1.3.cmml">1</mn><mo id="S3.p1.1.m1.2.2.1.1.1.1.1.2.3" xref="S3.p1.1.m1.2.2.1.1.1.1.1.2.3.cmml">⋆</mo></msubsup><mo id="S3.p1.1.m1.2.2.1.1.1.2.4" xref="S3.p1.1.m1.2.2.1.1.1.3.cmml">,</mo><msubsup id="S3.p1.1.m1.2.2.1.1.1.2.2" xref="S3.p1.1.m1.2.2.1.1.1.2.2.cmml"><mi id="S3.p1.1.m1.2.2.1.1.1.2.2.2.2" xref="S3.p1.1.m1.2.2.1.1.1.2.2.2.2.cmml">𝐬</mi><mn id="S3.p1.1.m1.2.2.1.1.1.2.2.3" xref="S3.p1.1.m1.2.2.1.1.1.2.2.3.cmml">1</mn><mo id="S3.p1.1.m1.2.2.1.1.1.2.2.2.3" xref="S3.p1.1.m1.2.2.1.1.1.2.2.2.3.cmml">⋆</mo></msubsup><mo id="S3.p1.1.m1.2.2.1.1.1.2.5" xref="S3.p1.1.m1.2.2.1.1.1.3.cmml">)</mo></mrow><mo id="S3.p1.1.m1.3.3.2.2.4" xref="S3.p1.1.m1.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="S3.p1.1.m1.1.1" xref="S3.p1.1.m1.1.1.cmml">…</mi><mo id="S3.p1.1.m1.3.3.2.2.5" xref="S3.p1.1.m1.3.3.2.3.cmml">,</mo><mrow id="S3.p1.1.m1.3.3.2.2.2.2" xref="S3.p1.1.m1.3.3.2.2.2.3.cmml"><mo id="S3.p1.1.m1.3.3.2.2.2.2.3" xref="S3.p1.1.m1.3.3.2.2.2.3.cmml">(</mo><msubsup id="S3.p1.1.m1.3.3.2.2.2.1.1" xref="S3.p1.1.m1.3.3.2.2.2.1.1.cmml"><mi id="S3.p1.1.m1.3.3.2.2.2.1.1.2.2" xref="S3.p1.1.m1.3.3.2.2.2.1.1.2.2.cmml">𝐪</mi><mi id="S3.p1.1.m1.3.3.2.2.2.1.1.3" xref="S3.p1.1.m1.3.3.2.2.2.1.1.3.cmml">e</mi><mo id="S3.p1.1.m1.3.3.2.2.2.1.1.2.3" xref="S3.p1.1.m1.3.3.2.2.2.1.1.2.3.cmml">⋆</mo></msubsup><mo id="S3.p1.1.m1.3.3.2.2.2.2.4" xref="S3.p1.1.m1.3.3.2.2.2.3.cmml">,</mo><msubsup id="S3.p1.1.m1.3.3.2.2.2.2.2" xref="S3.p1.1.m1.3.3.2.2.2.2.2.cmml"><mi id="S3.p1.1.m1.3.3.2.2.2.2.2.2.2" xref="S3.p1.1.m1.3.3.2.2.2.2.2.2.2.cmml">𝐬</mi><mi id="S3.p1.1.m1.3.3.2.2.2.2.2.3" xref="S3.p1.1.m1.3.3.2.2.2.2.2.3.cmml">e</mi><mo id="S3.p1.1.m1.3.3.2.2.2.2.2.2.3" xref="S3.p1.1.m1.3.3.2.2.2.2.2.2.3.cmml">⋆</mo></msubsup><mo id="S3.p1.1.m1.3.3.2.2.2.2.5" xref="S3.p1.1.m1.3.3.2.2.2.3.cmml">)</mo></mrow><mo id="S3.p1.1.m1.3.3.2.2.6" xref="S3.p1.1.m1.3.3.2.3.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.1.m1.3b"><apply id="S3.p1.1.m1.3.3.cmml" xref="S3.p1.1.m1.3.3"><eq id="S3.p1.1.m1.3.3.3.cmml" xref="S3.p1.1.m1.3.3.3"></eq><apply id="S3.p1.1.m1.3.3.4.cmml" xref="S3.p1.1.m1.3.3.4"><csymbol cd="ambiguous" id="S3.p1.1.m1.3.3.4.1.cmml" xref="S3.p1.1.m1.3.3.4">superscript</csymbol><ci id="S3.p1.1.m1.3.3.4.2.cmml" xref="S3.p1.1.m1.3.3.4.2">𝕏</ci><ci id="S3.p1.1.m1.3.3.4.3.cmml" xref="S3.p1.1.m1.3.3.4.3">⋆</ci></apply><set id="S3.p1.1.m1.3.3.2.3.cmml" xref="S3.p1.1.m1.3.3.2.2"><interval closure="open" id="S3.p1.1.m1.2.2.1.1.1.3.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2"><apply id="S3.p1.1.m1.2.2.1.1.1.1.1.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.p1.1.m1.2.2.1.1.1.1.1.1.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1">subscript</csymbol><apply id="S3.p1.1.m1.2.2.1.1.1.1.1.2.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.p1.1.m1.2.2.1.1.1.1.1.2.1.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1">superscript</csymbol><ci id="S3.p1.1.m1.2.2.1.1.1.1.1.2.2.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1.2.2">𝐪</ci><ci id="S3.p1.1.m1.2.2.1.1.1.1.1.2.3.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1.2.3">⋆</ci></apply><cn type="integer" id="S3.p1.1.m1.2.2.1.1.1.1.1.3.cmml" xref="S3.p1.1.m1.2.2.1.1.1.1.1.3">1</cn></apply><apply id="S3.p1.1.m1.2.2.1.1.1.2.2.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2"><csymbol cd="ambiguous" id="S3.p1.1.m1.2.2.1.1.1.2.2.1.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2">subscript</csymbol><apply id="S3.p1.1.m1.2.2.1.1.1.2.2.2.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2"><csymbol cd="ambiguous" id="S3.p1.1.m1.2.2.1.1.1.2.2.2.1.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2">superscript</csymbol><ci id="S3.p1.1.m1.2.2.1.1.1.2.2.2.2.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2.2.2">𝐬</ci><ci id="S3.p1.1.m1.2.2.1.1.1.2.2.2.3.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2.2.3">⋆</ci></apply><cn type="integer" id="S3.p1.1.m1.2.2.1.1.1.2.2.3.cmml" xref="S3.p1.1.m1.2.2.1.1.1.2.2.3">1</cn></apply></interval><ci id="S3.p1.1.m1.1.1.cmml" xref="S3.p1.1.m1.1.1">…</ci><interval closure="open" id="S3.p1.1.m1.3.3.2.2.2.3.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2"><apply id="S3.p1.1.m1.3.3.2.2.2.1.1.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1"><csymbol cd="ambiguous" id="S3.p1.1.m1.3.3.2.2.2.1.1.1.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1">subscript</csymbol><apply id="S3.p1.1.m1.3.3.2.2.2.1.1.2.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1"><csymbol cd="ambiguous" id="S3.p1.1.m1.3.3.2.2.2.1.1.2.1.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1">superscript</csymbol><ci id="S3.p1.1.m1.3.3.2.2.2.1.1.2.2.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1.2.2">𝐪</ci><ci id="S3.p1.1.m1.3.3.2.2.2.1.1.2.3.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1.2.3">⋆</ci></apply><ci id="S3.p1.1.m1.3.3.2.2.2.1.1.3.cmml" xref="S3.p1.1.m1.3.3.2.2.2.1.1.3">𝑒</ci></apply><apply id="S3.p1.1.m1.3.3.2.2.2.2.2.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2"><csymbol cd="ambiguous" id="S3.p1.1.m1.3.3.2.2.2.2.2.1.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2">subscript</csymbol><apply id="S3.p1.1.m1.3.3.2.2.2.2.2.2.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2"><csymbol cd="ambiguous" id="S3.p1.1.m1.3.3.2.2.2.2.2.2.1.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2">superscript</csymbol><ci id="S3.p1.1.m1.3.3.2.2.2.2.2.2.2.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2.2.2">𝐬</ci><ci id="S3.p1.1.m1.3.3.2.2.2.2.2.2.3.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2.2.3">⋆</ci></apply><ci id="S3.p1.1.m1.3.3.2.2.2.2.2.3.cmml" xref="S3.p1.1.m1.3.3.2.2.2.2.2.3">𝑒</ci></apply></interval></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.1.m1.3c">\mathbb{X^{\star}}=\left\{\left(\mathbf{q}^{\star}_{1},\mathbf{s}^{\star}_{1}\right),\ldots,\left(\mathbf{q}^{\star}_{e},\mathbf{s}^{\star}_{e}\right)\right\}</annotation></semantics></math> question-SQL pairs from an index of examples, <math id="S3.p1.2.m2.1" class="ltx_Math" alttext="\mathbb{X}" display="inline"><semantics id="S3.p1.2.m2.1a"><mi id="S3.p1.2.m2.1.1" xref="S3.p1.2.m2.1.1.cmml">𝕏</mi><annotation-xml encoding="MathML-Content" id="S3.p1.2.m2.1b"><ci id="S3.p1.2.m2.1.1.cmml" xref="S3.p1.2.m2.1.1">𝕏</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.2.m2.1c">\mathbb{X}</annotation></semantics></math>, s.t. <math id="S3.p1.3.m3.1" class="ltx_Math" alttext="\mathbb{X^{\star}}\subseteq\mathbb{X}" display="inline"><semantics id="S3.p1.3.m3.1a"><mrow id="S3.p1.3.m3.1.1" xref="S3.p1.3.m3.1.1.cmml"><msup id="S3.p1.3.m3.1.1.2" xref="S3.p1.3.m3.1.1.2.cmml"><mi id="S3.p1.3.m3.1.1.2.2" xref="S3.p1.3.m3.1.1.2.2.cmml">𝕏</mi><mo id="S3.p1.3.m3.1.1.2.3" xref="S3.p1.3.m3.1.1.2.3.cmml">⋆</mo></msup><mo id="S3.p1.3.m3.1.1.1" xref="S3.p1.3.m3.1.1.1.cmml">⊆</mo><mi id="S3.p1.3.m3.1.1.3" xref="S3.p1.3.m3.1.1.3.cmml">𝕏</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.3.m3.1b"><apply id="S3.p1.3.m3.1.1.cmml" xref="S3.p1.3.m3.1.1"><subset id="S3.p1.3.m3.1.1.1.cmml" xref="S3.p1.3.m3.1.1.1"></subset><apply id="S3.p1.3.m3.1.1.2.cmml" xref="S3.p1.3.m3.1.1.2"><csymbol cd="ambiguous" id="S3.p1.3.m3.1.1.2.1.cmml" xref="S3.p1.3.m3.1.1.2">superscript</csymbol><ci id="S3.p1.3.m3.1.1.2.2.cmml" xref="S3.p1.3.m3.1.1.2.2">𝕏</ci><ci id="S3.p1.3.m3.1.1.2.3.cmml" xref="S3.p1.3.m3.1.1.2.3">⋆</ci></apply><ci id="S3.p1.3.m3.1.1.3.cmml" xref="S3.p1.3.m3.1.1.3">𝕏</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.3.m3.1c">\mathbb{X^{\star}}\subseteq\mathbb{X}</annotation></semantics></math>, for maximising the probability of an LLM to predict the correct SQL given <math id="S3.p1.4.m4.2" class="ltx_Math" alttext="(\mathbf{q},\mathbf{D})" display="inline"><semantics id="S3.p1.4.m4.2a"><mrow id="S3.p1.4.m4.2.3.2" xref="S3.p1.4.m4.2.3.1.cmml"><mo stretchy="false" id="S3.p1.4.m4.2.3.2.1" xref="S3.p1.4.m4.2.3.1.cmml">(</mo><mi id="S3.p1.4.m4.1.1" xref="S3.p1.4.m4.1.1.cmml">𝐪</mi><mo id="S3.p1.4.m4.2.3.2.2" xref="S3.p1.4.m4.2.3.1.cmml">,</mo><mi id="S3.p1.4.m4.2.2" xref="S3.p1.4.m4.2.2.cmml">𝐃</mi><mo stretchy="false" id="S3.p1.4.m4.2.3.2.3" xref="S3.p1.4.m4.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.4.m4.2b"><interval closure="open" id="S3.p1.4.m4.2.3.1.cmml" xref="S3.p1.4.m4.2.3.2"><ci id="S3.p1.4.m4.1.1.cmml" xref="S3.p1.4.m4.1.1">𝐪</ci><ci id="S3.p1.4.m4.2.2.cmml" xref="S3.p1.4.m4.2.2">𝐃</ci></interval></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.4.m4.2c">(\mathbf{q},\mathbf{D})</annotation></semantics></math>:</p>
</div>
<div id="S3.p2" class="ltx_para">
<table id="A5.EGx2" class="ltx_equationgroup ltx_eqn_align ltx_eqn_table">

<tbody id="S3.E2"><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S3.E2.m1.4" class="ltx_Math" alttext="\displaystyle\mathbb{X^{\star}}=\operatorname*{\arg\max}_{\mathbb{X}}p\left(\mathbf{s}|\mathbf{q},\mathbf{D},\mathbb{X}\right)" display="inline"><semantics id="S3.E2.m1.4a"><mrow id="S3.E2.m1.4.4" xref="S3.E2.m1.4.4.cmml"><msup id="S3.E2.m1.4.4.3" xref="S3.E2.m1.4.4.3.cmml"><mi id="S3.E2.m1.4.4.3.2" xref="S3.E2.m1.4.4.3.2.cmml">𝕏</mi><mo id="S3.E2.m1.4.4.3.3" xref="S3.E2.m1.4.4.3.3.cmml">⋆</mo></msup><mo id="S3.E2.m1.4.4.2" xref="S3.E2.m1.4.4.2.cmml">=</mo><mrow id="S3.E2.m1.4.4.1" xref="S3.E2.m1.4.4.1.cmml"><mrow id="S3.E2.m1.4.4.1.3" xref="S3.E2.m1.4.4.1.3.cmml"><munder id="S3.E2.m1.4.4.1.3.1" xref="S3.E2.m1.4.4.1.3.1.cmml"><mrow id="S3.E2.m1.4.4.1.3.1.2" xref="S3.E2.m1.4.4.1.3.1.2.cmml"><mi id="S3.E2.m1.4.4.1.3.1.2.1" xref="S3.E2.m1.4.4.1.3.1.2.1.cmml">arg</mi><mo lspace="0.167em" id="S3.E2.m1.4.4.1.3.1.2a" xref="S3.E2.m1.4.4.1.3.1.2.cmml">⁡</mo><mi id="S3.E2.m1.4.4.1.3.1.2.2" xref="S3.E2.m1.4.4.1.3.1.2.2.cmml">max</mi></mrow><mi id="S3.E2.m1.4.4.1.3.1.3" xref="S3.E2.m1.4.4.1.3.1.3.cmml">𝕏</mi></munder><mo lspace="0.167em" id="S3.E2.m1.4.4.1.3a" xref="S3.E2.m1.4.4.1.3.cmml">⁡</mo><mi id="S3.E2.m1.4.4.1.3.2" xref="S3.E2.m1.4.4.1.3.2.cmml">p</mi></mrow><mo lspace="0em" rspace="0em" id="S3.E2.m1.4.4.1.2" xref="S3.E2.m1.4.4.1.2.cmml">​</mo><mrow id="S3.E2.m1.4.4.1.1.1" xref="S3.E2.m1.4.4.1.1.1.1.cmml"><mo id="S3.E2.m1.4.4.1.1.1.2" xref="S3.E2.m1.4.4.1.1.1.1.cmml">(</mo><mrow id="S3.E2.m1.4.4.1.1.1.1" xref="S3.E2.m1.4.4.1.1.1.1.cmml"><mi id="S3.E2.m1.4.4.1.1.1.1.2" xref="S3.E2.m1.4.4.1.1.1.1.2.cmml">𝐬</mi><mo fence="false" id="S3.E2.m1.4.4.1.1.1.1.1" xref="S3.E2.m1.4.4.1.1.1.1.1.cmml">|</mo><mrow id="S3.E2.m1.4.4.1.1.1.1.3.2" xref="S3.E2.m1.4.4.1.1.1.1.3.1.cmml"><mi id="S3.E2.m1.1.1" xref="S3.E2.m1.1.1.cmml">𝐪</mi><mo id="S3.E2.m1.4.4.1.1.1.1.3.2.1" xref="S3.E2.m1.4.4.1.1.1.1.3.1.cmml">,</mo><mi id="S3.E2.m1.2.2" xref="S3.E2.m1.2.2.cmml">𝐃</mi><mo id="S3.E2.m1.4.4.1.1.1.1.3.2.2" xref="S3.E2.m1.4.4.1.1.1.1.3.1.cmml">,</mo><mi id="S3.E2.m1.3.3" xref="S3.E2.m1.3.3.cmml">𝕏</mi></mrow></mrow><mo id="S3.E2.m1.4.4.1.1.1.3" xref="S3.E2.m1.4.4.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.E2.m1.4b"><apply id="S3.E2.m1.4.4.cmml" xref="S3.E2.m1.4.4"><eq id="S3.E2.m1.4.4.2.cmml" xref="S3.E2.m1.4.4.2"></eq><apply id="S3.E2.m1.4.4.3.cmml" xref="S3.E2.m1.4.4.3"><csymbol cd="ambiguous" id="S3.E2.m1.4.4.3.1.cmml" xref="S3.E2.m1.4.4.3">superscript</csymbol><ci id="S3.E2.m1.4.4.3.2.cmml" xref="S3.E2.m1.4.4.3.2">𝕏</ci><ci id="S3.E2.m1.4.4.3.3.cmml" xref="S3.E2.m1.4.4.3.3">⋆</ci></apply><apply id="S3.E2.m1.4.4.1.cmml" xref="S3.E2.m1.4.4.1"><times id="S3.E2.m1.4.4.1.2.cmml" xref="S3.E2.m1.4.4.1.2"></times><apply id="S3.E2.m1.4.4.1.3.cmml" xref="S3.E2.m1.4.4.1.3"><apply id="S3.E2.m1.4.4.1.3.1.cmml" xref="S3.E2.m1.4.4.1.3.1"><csymbol cd="ambiguous" id="S3.E2.m1.4.4.1.3.1.1.cmml" xref="S3.E2.m1.4.4.1.3.1">subscript</csymbol><apply id="S3.E2.m1.4.4.1.3.1.2.cmml" xref="S3.E2.m1.4.4.1.3.1.2"><arg id="S3.E2.m1.4.4.1.3.1.2.1.cmml" xref="S3.E2.m1.4.4.1.3.1.2.1"></arg><max id="S3.E2.m1.4.4.1.3.1.2.2.cmml" xref="S3.E2.m1.4.4.1.3.1.2.2"></max></apply><ci id="S3.E2.m1.4.4.1.3.1.3.cmml" xref="S3.E2.m1.4.4.1.3.1.3">𝕏</ci></apply><ci id="S3.E2.m1.4.4.1.3.2.cmml" xref="S3.E2.m1.4.4.1.3.2">𝑝</ci></apply><apply id="S3.E2.m1.4.4.1.1.1.1.cmml" xref="S3.E2.m1.4.4.1.1.1"><csymbol cd="latexml" id="S3.E2.m1.4.4.1.1.1.1.1.cmml" xref="S3.E2.m1.4.4.1.1.1.1.1">conditional</csymbol><ci id="S3.E2.m1.4.4.1.1.1.1.2.cmml" xref="S3.E2.m1.4.4.1.1.1.1.2">𝐬</ci><list id="S3.E2.m1.4.4.1.1.1.1.3.1.cmml" xref="S3.E2.m1.4.4.1.1.1.1.3.2"><ci id="S3.E2.m1.1.1.cmml" xref="S3.E2.m1.1.1">𝐪</ci><ci id="S3.E2.m1.2.2.cmml" xref="S3.E2.m1.2.2">𝐃</ci><ci id="S3.E2.m1.3.3.cmml" xref="S3.E2.m1.3.3">𝕏</ci></list></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E2.m1.4c">\displaystyle\mathbb{X^{\star}}=\operatorname*{\arg\max}_{\mathbb{X}}p\left(\mathbf{s}|\mathbf{q},\mathbf{D},\mathbb{X}\right)</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(2)</span></td>
</tr></tbody>
</table>
</div>
<div id="S3.p3" class="ltx_para">
<p id="S3.p3.2" class="ltx_p">From the perspective of ranking, we consider the relevance score between a candidate example <math id="S3.p3.1.m1.2" class="ltx_Math" alttext="(\mathbf{q}_{j},\mathbf{s}_{j})\in\mathbb{X}" display="inline"><semantics id="S3.p3.1.m1.2a"><mrow id="S3.p3.1.m1.2.2" xref="S3.p3.1.m1.2.2.cmml"><mrow id="S3.p3.1.m1.2.2.2.2" xref="S3.p3.1.m1.2.2.2.3.cmml"><mo stretchy="false" id="S3.p3.1.m1.2.2.2.2.3" xref="S3.p3.1.m1.2.2.2.3.cmml">(</mo><msub id="S3.p3.1.m1.1.1.1.1.1" xref="S3.p3.1.m1.1.1.1.1.1.cmml"><mi id="S3.p3.1.m1.1.1.1.1.1.2" xref="S3.p3.1.m1.1.1.1.1.1.2.cmml">𝐪</mi><mi id="S3.p3.1.m1.1.1.1.1.1.3" xref="S3.p3.1.m1.1.1.1.1.1.3.cmml">j</mi></msub><mo id="S3.p3.1.m1.2.2.2.2.4" xref="S3.p3.1.m1.2.2.2.3.cmml">,</mo><msub id="S3.p3.1.m1.2.2.2.2.2" xref="S3.p3.1.m1.2.2.2.2.2.cmml"><mi id="S3.p3.1.m1.2.2.2.2.2.2" xref="S3.p3.1.m1.2.2.2.2.2.2.cmml">𝐬</mi><mi id="S3.p3.1.m1.2.2.2.2.2.3" xref="S3.p3.1.m1.2.2.2.2.2.3.cmml">j</mi></msub><mo stretchy="false" id="S3.p3.1.m1.2.2.2.2.5" xref="S3.p3.1.m1.2.2.2.3.cmml">)</mo></mrow><mo id="S3.p3.1.m1.2.2.3" xref="S3.p3.1.m1.2.2.3.cmml">∈</mo><mi id="S3.p3.1.m1.2.2.4" xref="S3.p3.1.m1.2.2.4.cmml">𝕏</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p3.1.m1.2b"><apply id="S3.p3.1.m1.2.2.cmml" xref="S3.p3.1.m1.2.2"><in id="S3.p3.1.m1.2.2.3.cmml" xref="S3.p3.1.m1.2.2.3"></in><interval closure="open" id="S3.p3.1.m1.2.2.2.3.cmml" xref="S3.p3.1.m1.2.2.2.2"><apply id="S3.p3.1.m1.1.1.1.1.1.cmml" xref="S3.p3.1.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.p3.1.m1.1.1.1.1.1.1.cmml" xref="S3.p3.1.m1.1.1.1.1.1">subscript</csymbol><ci id="S3.p3.1.m1.1.1.1.1.1.2.cmml" xref="S3.p3.1.m1.1.1.1.1.1.2">𝐪</ci><ci id="S3.p3.1.m1.1.1.1.1.1.3.cmml" xref="S3.p3.1.m1.1.1.1.1.1.3">𝑗</ci></apply><apply id="S3.p3.1.m1.2.2.2.2.2.cmml" xref="S3.p3.1.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="S3.p3.1.m1.2.2.2.2.2.1.cmml" xref="S3.p3.1.m1.2.2.2.2.2">subscript</csymbol><ci id="S3.p3.1.m1.2.2.2.2.2.2.cmml" xref="S3.p3.1.m1.2.2.2.2.2.2">𝐬</ci><ci id="S3.p3.1.m1.2.2.2.2.2.3.cmml" xref="S3.p3.1.m1.2.2.2.2.2.3">𝑗</ci></apply></interval><ci id="S3.p3.1.m1.2.2.4.cmml" xref="S3.p3.1.m1.2.2.4">𝕏</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.1.m1.2c">(\mathbf{q}_{j},\mathbf{s}_{j})\in\mathbb{X}</annotation></semantics></math> and the input <math id="S3.p3.2.m2.2" class="ltx_Math" alttext="(\mathbf{q},\mathbf{D})" display="inline"><semantics id="S3.p3.2.m2.2a"><mrow id="S3.p3.2.m2.2.3.2" xref="S3.p3.2.m2.2.3.1.cmml"><mo stretchy="false" id="S3.p3.2.m2.2.3.2.1" xref="S3.p3.2.m2.2.3.1.cmml">(</mo><mi id="S3.p3.2.m2.1.1" xref="S3.p3.2.m2.1.1.cmml">𝐪</mi><mo id="S3.p3.2.m2.2.3.2.2" xref="S3.p3.2.m2.2.3.1.cmml">,</mo><mi id="S3.p3.2.m2.2.2" xref="S3.p3.2.m2.2.2.cmml">𝐃</mi><mo stretchy="false" id="S3.p3.2.m2.2.3.2.3" xref="S3.p3.2.m2.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.p3.2.m2.2b"><interval closure="open" id="S3.p3.2.m2.2.3.1.cmml" xref="S3.p3.2.m2.2.3.2"><ci id="S3.p3.2.m2.1.1.cmml" xref="S3.p3.2.m2.1.1">𝐪</ci><ci id="S3.p3.2.m2.2.2.cmml" xref="S3.p3.2.m2.2.2">𝐃</ci></interval></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.2.m2.2c">(\mathbf{q},\mathbf{D})</annotation></semantics></math>. <span id="S3.p3.2.1" class="ltx_text ltx_font_italic">Vanilla</span> semantic search is usually based solely on question embeddings, whereas the structure of SQL queries for similar questions is subject to target databases and can thus differ significantly.</p>
</div>
<div id="S3.p4" class="ltx_para">
<p id="S3.p4.5" class="ltx_p">To incorporate database context for selecting examples, we propose to re-rank examples retrieved by question embeddings based on normalised SQL ASTs.
Inspired by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib10" title="" class="ltx_ref">Gao et al.</a></cite>, our framework utilises a preliminary model to compute an approximated SQL query <math id="S3.p4.1.m1.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}" display="inline"><semantics id="S3.p4.1.m1.1a"><msup id="S3.p4.1.m1.1.1" xref="S3.p4.1.m1.1.1.cmml"><mi id="S3.p4.1.m1.1.1.2" xref="S3.p4.1.m1.1.1.2.cmml">𝐬</mi><mo id="S3.p4.1.m1.1.1.3" xref="S3.p4.1.m1.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S3.p4.1.m1.1b"><apply id="S3.p4.1.m1.1.1.cmml" xref="S3.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.p4.1.m1.1.1.1.cmml" xref="S3.p4.1.m1.1.1">superscript</csymbol><ci id="S3.p4.1.m1.1.1.2.cmml" xref="S3.p4.1.m1.1.1.2">𝐬</ci><ci id="S3.p4.1.m1.1.1.3.cmml" xref="S3.p4.1.m1.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.1.m1.1c">\mathbf{s}^{\prime}</annotation></semantics></math>, structurally similar to the ground truth, given <math id="S3.p4.2.m2.2" class="ltx_Math" alttext="(\mathbf{q},\mathbf{D})" display="inline"><semantics id="S3.p4.2.m2.2a"><mrow id="S3.p4.2.m2.2.3.2" xref="S3.p4.2.m2.2.3.1.cmml"><mo stretchy="false" id="S3.p4.2.m2.2.3.2.1" xref="S3.p4.2.m2.2.3.1.cmml">(</mo><mi id="S3.p4.2.m2.1.1" xref="S3.p4.2.m2.1.1.cmml">𝐪</mi><mo id="S3.p4.2.m2.2.3.2.2" xref="S3.p4.2.m2.2.3.1.cmml">,</mo><mi id="S3.p4.2.m2.2.2" xref="S3.p4.2.m2.2.2.cmml">𝐃</mi><mo stretchy="false" id="S3.p4.2.m2.2.3.2.3" xref="S3.p4.2.m2.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.p4.2.m2.2b"><interval closure="open" id="S3.p4.2.m2.2.3.1.cmml" xref="S3.p4.2.m2.2.3.2"><ci id="S3.p4.2.m2.1.1.cmml" xref="S3.p4.2.m2.1.1">𝐪</ci><ci id="S3.p4.2.m2.2.2.cmml" xref="S3.p4.2.m2.2.2">𝐃</ci></interval></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.2.m2.2c">(\mathbf{q},\mathbf{D})</annotation></semantics></math> s.t. <math id="S3.p4.3.m3.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}\sim\mathbf{s}" display="inline"><semantics id="S3.p4.3.m3.1a"><mrow id="S3.p4.3.m3.1.1" xref="S3.p4.3.m3.1.1.cmml"><msup id="S3.p4.3.m3.1.1.2" xref="S3.p4.3.m3.1.1.2.cmml"><mi id="S3.p4.3.m3.1.1.2.2" xref="S3.p4.3.m3.1.1.2.2.cmml">𝐬</mi><mo id="S3.p4.3.m3.1.1.2.3" xref="S3.p4.3.m3.1.1.2.3.cmml">′</mo></msup><mo id="S3.p4.3.m3.1.1.1" xref="S3.p4.3.m3.1.1.1.cmml">∼</mo><mi id="S3.p4.3.m3.1.1.3" xref="S3.p4.3.m3.1.1.3.cmml">𝐬</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p4.3.m3.1b"><apply id="S3.p4.3.m3.1.1.cmml" xref="S3.p4.3.m3.1.1"><csymbol cd="latexml" id="S3.p4.3.m3.1.1.1.cmml" xref="S3.p4.3.m3.1.1.1">similar-to</csymbol><apply id="S3.p4.3.m3.1.1.2.cmml" xref="S3.p4.3.m3.1.1.2"><csymbol cd="ambiguous" id="S3.p4.3.m3.1.1.2.1.cmml" xref="S3.p4.3.m3.1.1.2">superscript</csymbol><ci id="S3.p4.3.m3.1.1.2.2.cmml" xref="S3.p4.3.m3.1.1.2.2">𝐬</ci><ci id="S3.p4.3.m3.1.1.2.3.cmml" xref="S3.p4.3.m3.1.1.2.3">′</ci></apply><ci id="S3.p4.3.m3.1.1.3.cmml" xref="S3.p4.3.m3.1.1.3">𝐬</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.3.m3.1c">\mathbf{s}^{\prime}\sim\mathbf{s}</annotation></semantics></math>.
Examples are then re-ranked by <math id="S3.p4.4.m4.2" class="ltx_Math" alttext="\text{score}_{\text{AST}}\left(\mathbf{s}^{\prime},\mathbf{s}_{j}\right)" display="inline"><semantics id="S3.p4.4.m4.2a"><mrow id="S3.p4.4.m4.2.2" xref="S3.p4.4.m4.2.2.cmml"><msub id="S3.p4.4.m4.2.2.4" xref="S3.p4.4.m4.2.2.4.cmml"><mtext id="S3.p4.4.m4.2.2.4.2" xref="S3.p4.4.m4.2.2.4.2a.cmml">score</mtext><mtext id="S3.p4.4.m4.2.2.4.3" xref="S3.p4.4.m4.2.2.4.3a.cmml">AST</mtext></msub><mo lspace="0em" rspace="0em" id="S3.p4.4.m4.2.2.3" xref="S3.p4.4.m4.2.2.3.cmml">​</mo><mrow id="S3.p4.4.m4.2.2.2.2" xref="S3.p4.4.m4.2.2.2.3.cmml"><mo id="S3.p4.4.m4.2.2.2.2.3" xref="S3.p4.4.m4.2.2.2.3.cmml">(</mo><msup id="S3.p4.4.m4.1.1.1.1.1" xref="S3.p4.4.m4.1.1.1.1.1.cmml"><mi id="S3.p4.4.m4.1.1.1.1.1.2" xref="S3.p4.4.m4.1.1.1.1.1.2.cmml">𝐬</mi><mo id="S3.p4.4.m4.1.1.1.1.1.3" xref="S3.p4.4.m4.1.1.1.1.1.3.cmml">′</mo></msup><mo id="S3.p4.4.m4.2.2.2.2.4" xref="S3.p4.4.m4.2.2.2.3.cmml">,</mo><msub id="S3.p4.4.m4.2.2.2.2.2" xref="S3.p4.4.m4.2.2.2.2.2.cmml"><mi id="S3.p4.4.m4.2.2.2.2.2.2" xref="S3.p4.4.m4.2.2.2.2.2.2.cmml">𝐬</mi><mi id="S3.p4.4.m4.2.2.2.2.2.3" xref="S3.p4.4.m4.2.2.2.2.2.3.cmml">j</mi></msub><mo id="S3.p4.4.m4.2.2.2.2.5" xref="S3.p4.4.m4.2.2.2.3.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p4.4.m4.2b"><apply id="S3.p4.4.m4.2.2.cmml" xref="S3.p4.4.m4.2.2"><times id="S3.p4.4.m4.2.2.3.cmml" xref="S3.p4.4.m4.2.2.3"></times><apply id="S3.p4.4.m4.2.2.4.cmml" xref="S3.p4.4.m4.2.2.4"><csymbol cd="ambiguous" id="S3.p4.4.m4.2.2.4.1.cmml" xref="S3.p4.4.m4.2.2.4">subscript</csymbol><ci id="S3.p4.4.m4.2.2.4.2a.cmml" xref="S3.p4.4.m4.2.2.4.2"><mtext id="S3.p4.4.m4.2.2.4.2.cmml" xref="S3.p4.4.m4.2.2.4.2">score</mtext></ci><ci id="S3.p4.4.m4.2.2.4.3a.cmml" xref="S3.p4.4.m4.2.2.4.3"><mtext mathsize="70%" id="S3.p4.4.m4.2.2.4.3.cmml" xref="S3.p4.4.m4.2.2.4.3">AST</mtext></ci></apply><interval closure="open" id="S3.p4.4.m4.2.2.2.3.cmml" xref="S3.p4.4.m4.2.2.2.2"><apply id="S3.p4.4.m4.1.1.1.1.1.cmml" xref="S3.p4.4.m4.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.p4.4.m4.1.1.1.1.1.1.cmml" xref="S3.p4.4.m4.1.1.1.1.1">superscript</csymbol><ci id="S3.p4.4.m4.1.1.1.1.1.2.cmml" xref="S3.p4.4.m4.1.1.1.1.1.2">𝐬</ci><ci id="S3.p4.4.m4.1.1.1.1.1.3.cmml" xref="S3.p4.4.m4.1.1.1.1.1.3">′</ci></apply><apply id="S3.p4.4.m4.2.2.2.2.2.cmml" xref="S3.p4.4.m4.2.2.2.2.2"><csymbol cd="ambiguous" id="S3.p4.4.m4.2.2.2.2.2.1.cmml" xref="S3.p4.4.m4.2.2.2.2.2">subscript</csymbol><ci id="S3.p4.4.m4.2.2.2.2.2.2.cmml" xref="S3.p4.4.m4.2.2.2.2.2.2">𝐬</ci><ci id="S3.p4.4.m4.2.2.2.2.2.3.cmml" xref="S3.p4.4.m4.2.2.2.2.2.3">𝑗</ci></apply></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.4.m4.2c">\text{score}_{\text{AST}}\left(\mathbf{s}^{\prime},\mathbf{s}_{j}\right)</annotation></semantics></math> for each candidate <math id="S3.p4.5.m5.1" class="ltx_Math" alttext="\mathbf{s}_{j}" display="inline"><semantics id="S3.p4.5.m5.1a"><msub id="S3.p4.5.m5.1.1" xref="S3.p4.5.m5.1.1.cmml"><mi id="S3.p4.5.m5.1.1.2" xref="S3.p4.5.m5.1.1.2.cmml">𝐬</mi><mi id="S3.p4.5.m5.1.1.3" xref="S3.p4.5.m5.1.1.3.cmml">j</mi></msub><annotation-xml encoding="MathML-Content" id="S3.p4.5.m5.1b"><apply id="S3.p4.5.m5.1.1.cmml" xref="S3.p4.5.m5.1.1"><csymbol cd="ambiguous" id="S3.p4.5.m5.1.1.1.cmml" xref="S3.p4.5.m5.1.1">subscript</csymbol><ci id="S3.p4.5.m5.1.1.2.cmml" xref="S3.p4.5.m5.1.1.2">𝐬</ci><ci id="S3.p4.5.m5.1.1.3.cmml" xref="S3.p4.5.m5.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.5.m5.1c">\mathbf{s}_{j}</annotation></semantics></math>.</p>
</div>
<div id="S3.p5" class="ltx_para">
<p id="S3.p5.1" class="ltx_p">AST represents the hierarchical structure of code in a tree form and can be applied to evaluation metrics for code generation <cite class="ltx_cite ltx_citemacro_cite">Tran et al. (<a href="#bib.bib26" title="" class="ltx_ref">2019</a>); Ren et al. (<a href="#bib.bib22" title="" class="ltx_ref">2020</a>)</cite>. The fact that SQL queries sharing identical abstract meanings may not align with the same syntactic structure poses a challenge for measuring similarity through AST differencing.</p>
</div>
<section id="S3.SS0.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">AST Normalisation</h5>

<div id="S3.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S3.SS0.SSS0.Px1.p1.1" class="ltx_p">Although it is infeasible to exhaustively transform a SQL to another equivalent form, we can normalise ASTs to reduce undesired mismatch. Firstly, nodes of identifiers are lowercased and unnecessary references are removed (e.g. <code id="S3.SS0.SSS0.Px1.p1.1.1" class="ltx_verbatim ltx_font_typewriter">&lt;table&gt;.&lt;column&gt;</code> is substituted with <code id="S3.SS0.SSS0.Px1.p1.1.2" class="ltx_verbatim ltx_font_typewriter">&lt;column&gt;</code> if possible). We then delete nodes that create aliases and map each alias to a copy of the subtree to which it references. For cross-domain settings wherein databases at inference time are unseen in the train set, we mask out nodes of values and identifiers after resolving aliases. Otherwise for in-domain settings we further sort nodes associated with JOIN operations(s) to ensure the ordering of tables and keys is consistent.</p>
</div>
</section>
<section id="S3.SS0.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">AST Similarity</h5>

<div id="S3.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S3.SS0.SSS0.Px2.p1.1" class="ltx_p">Given two normalised ASTs, we adopt the Change Distilling algorithm <cite class="ltx_cite ltx_citemacro_cite">Fluri et al. (<a href="#bib.bib7" title="" class="ltx_ref">2007</a>)</cite> that computes a list of tree edit operations to transform the source AST to the target AST. Types of tree edit operations include: <code id="S3.SS0.SSS0.Px2.p1.1.1" class="ltx_verbatim ltx_font_typewriter">insert</code>, <code id="S3.SS0.SSS0.Px2.p1.1.2" class="ltx_verbatim ltx_font_typewriter">delete</code>, <code id="S3.SS0.SSS0.Px2.p1.1.3" class="ltx_verbatim ltx_font_typewriter">alignment</code>, <code id="S3.SS0.SSS0.Px2.p1.1.4" class="ltx_verbatim ltx_font_typewriter">move</code> and <code id="S3.SS0.SSS0.Px2.p1.1.5" class="ltx_verbatim ltx_font_typewriter">update</code>. It is essential to note that <code id="S3.SS0.SSS0.Px2.p1.1.6" class="ltx_verbatim ltx_font_typewriter">move</code> operation relocates a node to a different parent while moving a node within the same parent is an <code id="S3.SS0.SSS0.Px2.p1.1.7" class="ltx_verbatim ltx_font_typewriter">alignment</code>. Therefore, we calculate the similarity between ASTs simply as the ratio of alignments to the total number of operations within the list. Examples of our normalisation and AST similarity are provided in Appendix <a href="#A1" title="Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A</span></a>.</p>
</div>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Database Context Selection</h2>

<div id="S4.p1" class="ltx_para">
<p id="S4.p1.1" class="ltx_p">Apart from relevant question-SQL pairs, prompting for Text-to-SQL parsing requires the context of database schema and values.</p>
</div>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Schema Selection</h3>

<div id="S4.SS1.p1" class="ltx_para">
<p id="S4.SS1.p1.1" class="ltx_p">We present a hybrid search strategy that selects a sub-schema given a test question to minimise lengthy and potentially irrelevant schema elements
input to LLMs, while maintaining high recall.</p>
</div>
<div id="S4.SS1.p2" class="ltx_para">
<p id="S4.SS1.p2.7" class="ltx_p">Let <math id="S4.SS1.p2.1.m1.1" class="ltx_Math" alttext="r_{j}^{i}" display="inline"><semantics id="S4.SS1.p2.1.m1.1a"><msubsup id="S4.SS1.p2.1.m1.1.1" xref="S4.SS1.p2.1.m1.1.1.cmml"><mi id="S4.SS1.p2.1.m1.1.1.2.2" xref="S4.SS1.p2.1.m1.1.1.2.2.cmml">r</mi><mi id="S4.SS1.p2.1.m1.1.1.2.3" xref="S4.SS1.p2.1.m1.1.1.2.3.cmml">j</mi><mi id="S4.SS1.p2.1.m1.1.1.3" xref="S4.SS1.p2.1.m1.1.1.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.1.m1.1b"><apply id="S4.SS1.p2.1.m1.1.1.cmml" xref="S4.SS1.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.1.m1.1.1.1.cmml" xref="S4.SS1.p2.1.m1.1.1">superscript</csymbol><apply id="S4.SS1.p2.1.m1.1.1.2.cmml" xref="S4.SS1.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.1.m1.1.1.2.1.cmml" xref="S4.SS1.p2.1.m1.1.1">subscript</csymbol><ci id="S4.SS1.p2.1.m1.1.1.2.2.cmml" xref="S4.SS1.p2.1.m1.1.1.2.2">𝑟</ci><ci id="S4.SS1.p2.1.m1.1.1.2.3.cmml" xref="S4.SS1.p2.1.m1.1.1.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.1.m1.1.1.3.cmml" xref="S4.SS1.p2.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.1.m1.1c">r_{j}^{i}</annotation></semantics></math> be a semantic representation of column <math id="S4.SS1.p2.2.m2.1" class="ltx_Math" alttext="c_{j}^{i}" display="inline"><semantics id="S4.SS1.p2.2.m2.1a"><msubsup id="S4.SS1.p2.2.m2.1.1" xref="S4.SS1.p2.2.m2.1.1.cmml"><mi id="S4.SS1.p2.2.m2.1.1.2.2" xref="S4.SS1.p2.2.m2.1.1.2.2.cmml">c</mi><mi id="S4.SS1.p2.2.m2.1.1.2.3" xref="S4.SS1.p2.2.m2.1.1.2.3.cmml">j</mi><mi id="S4.SS1.p2.2.m2.1.1.3" xref="S4.SS1.p2.2.m2.1.1.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.2.m2.1b"><apply id="S4.SS1.p2.2.m2.1.1.cmml" xref="S4.SS1.p2.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.2.m2.1.1.1.cmml" xref="S4.SS1.p2.2.m2.1.1">superscript</csymbol><apply id="S4.SS1.p2.2.m2.1.1.2.cmml" xref="S4.SS1.p2.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.2.m2.1.1.2.1.cmml" xref="S4.SS1.p2.2.m2.1.1">subscript</csymbol><ci id="S4.SS1.p2.2.m2.1.1.2.2.cmml" xref="S4.SS1.p2.2.m2.1.1.2.2">𝑐</ci><ci id="S4.SS1.p2.2.m2.1.1.2.3.cmml" xref="S4.SS1.p2.2.m2.1.1.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.2.m2.1.1.3.cmml" xref="S4.SS1.p2.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.2.m2.1c">c_{j}^{i}</annotation></semantics></math>. We aggregate the semantic names<span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>Semantic name can refer to simply to the name of a particular or to a concatenation of its name and description.</span></span></span> of
<math id="S4.SS1.p2.3.m3.1" class="ltx_Math" alttext="c_{j}^{i}" display="inline"><semantics id="S4.SS1.p2.3.m3.1a"><msubsup id="S4.SS1.p2.3.m3.1.1" xref="S4.SS1.p2.3.m3.1.1.cmml"><mi id="S4.SS1.p2.3.m3.1.1.2.2" xref="S4.SS1.p2.3.m3.1.1.2.2.cmml">c</mi><mi id="S4.SS1.p2.3.m3.1.1.2.3" xref="S4.SS1.p2.3.m3.1.1.2.3.cmml">j</mi><mi id="S4.SS1.p2.3.m3.1.1.3" xref="S4.SS1.p2.3.m3.1.1.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.3.m3.1b"><apply id="S4.SS1.p2.3.m3.1.1.cmml" xref="S4.SS1.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.3.m3.1.1.1.cmml" xref="S4.SS1.p2.3.m3.1.1">superscript</csymbol><apply id="S4.SS1.p2.3.m3.1.1.2.cmml" xref="S4.SS1.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.3.m3.1.1.2.1.cmml" xref="S4.SS1.p2.3.m3.1.1">subscript</csymbol><ci id="S4.SS1.p2.3.m3.1.1.2.2.cmml" xref="S4.SS1.p2.3.m3.1.1.2.2">𝑐</ci><ci id="S4.SS1.p2.3.m3.1.1.2.3.cmml" xref="S4.SS1.p2.3.m3.1.1.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.3.m3.1.1.3.cmml" xref="S4.SS1.p2.3.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.3.m3.1c">c_{j}^{i}</annotation></semantics></math> and the table it belongs to, <math id="S4.SS1.p2.4.m4.1" class="ltx_Math" alttext="t_{i}" display="inline"><semantics id="S4.SS1.p2.4.m4.1a"><msub id="S4.SS1.p2.4.m4.1.1" xref="S4.SS1.p2.4.m4.1.1.cmml"><mi id="S4.SS1.p2.4.m4.1.1.2" xref="S4.SS1.p2.4.m4.1.1.2.cmml">t</mi><mi id="S4.SS1.p2.4.m4.1.1.3" xref="S4.SS1.p2.4.m4.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.4.m4.1b"><apply id="S4.SS1.p2.4.m4.1.1.cmml" xref="S4.SS1.p2.4.m4.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.4.m4.1.1.1.cmml" xref="S4.SS1.p2.4.m4.1.1">subscript</csymbol><ci id="S4.SS1.p2.4.m4.1.1.2.cmml" xref="S4.SS1.p2.4.m4.1.1.2">𝑡</ci><ci id="S4.SS1.p2.4.m4.1.1.3.cmml" xref="S4.SS1.p2.4.m4.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.4.m4.1c">t_{i}</annotation></semantics></math>, and its corresponding value set in <math id="S4.SS1.p2.5.m5.1" class="ltx_Math" alttext="\mathbf{D}" display="inline"><semantics id="S4.SS1.p2.5.m5.1a"><mi id="S4.SS1.p2.5.m5.1.1" xref="S4.SS1.p2.5.m5.1.1.cmml">𝐃</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.5.m5.1b"><ci id="S4.SS1.p2.5.m5.1.1.cmml" xref="S4.SS1.p2.5.m5.1.1">𝐃</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.5.m5.1c">\mathbf{D}</annotation></semantics></math>, <math id="S4.SS1.p2.6.m6.1" class="ltx_Math" alttext="v_{c_{j}^{i}}" display="inline"><semantics id="S4.SS1.p2.6.m6.1a"><msub id="S4.SS1.p2.6.m6.1.1" xref="S4.SS1.p2.6.m6.1.1.cmml"><mi id="S4.SS1.p2.6.m6.1.1.2" xref="S4.SS1.p2.6.m6.1.1.2.cmml">v</mi><msubsup id="S4.SS1.p2.6.m6.1.1.3" xref="S4.SS1.p2.6.m6.1.1.3.cmml"><mi id="S4.SS1.p2.6.m6.1.1.3.2.2" xref="S4.SS1.p2.6.m6.1.1.3.2.2.cmml">c</mi><mi id="S4.SS1.p2.6.m6.1.1.3.2.3" xref="S4.SS1.p2.6.m6.1.1.3.2.3.cmml">j</mi><mi id="S4.SS1.p2.6.m6.1.1.3.3" xref="S4.SS1.p2.6.m6.1.1.3.3.cmml">i</mi></msubsup></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.6.m6.1b"><apply id="S4.SS1.p2.6.m6.1.1.cmml" xref="S4.SS1.p2.6.m6.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.6.m6.1.1.1.cmml" xref="S4.SS1.p2.6.m6.1.1">subscript</csymbol><ci id="S4.SS1.p2.6.m6.1.1.2.cmml" xref="S4.SS1.p2.6.m6.1.1.2">𝑣</ci><apply id="S4.SS1.p2.6.m6.1.1.3.cmml" xref="S4.SS1.p2.6.m6.1.1.3"><csymbol cd="ambiguous" id="S4.SS1.p2.6.m6.1.1.3.1.cmml" xref="S4.SS1.p2.6.m6.1.1.3">superscript</csymbol><apply id="S4.SS1.p2.6.m6.1.1.3.2.cmml" xref="S4.SS1.p2.6.m6.1.1.3"><csymbol cd="ambiguous" id="S4.SS1.p2.6.m6.1.1.3.2.1.cmml" xref="S4.SS1.p2.6.m6.1.1.3">subscript</csymbol><ci id="S4.SS1.p2.6.m6.1.1.3.2.2.cmml" xref="S4.SS1.p2.6.m6.1.1.3.2.2">𝑐</ci><ci id="S4.SS1.p2.6.m6.1.1.3.2.3.cmml" xref="S4.SS1.p2.6.m6.1.1.3.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.6.m6.1.1.3.3.cmml" xref="S4.SS1.p2.6.m6.1.1.3.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.6.m6.1c">v_{c_{j}^{i}}</annotation></semantics></math>, as follows:
<br class="ltx_break"><math id="S4.SS1.p2.7.m7.5" class="ltx_Math" alttext="r_{j}^{i}=\left\{t_{i}\cup c_{j}^{i}\cup v_{c_{j}^{i}}\mid i\in\left[1,T\right]\text{ and }j\in\left[1,C_{i}\right]\right\}" display="inline"><semantics id="S4.SS1.p2.7.m7.5a"><mrow id="S4.SS1.p2.7.m7.5.5" xref="S4.SS1.p2.7.m7.5.5.cmml"><msubsup id="S4.SS1.p2.7.m7.5.5.4" xref="S4.SS1.p2.7.m7.5.5.4.cmml"><mi id="S4.SS1.p2.7.m7.5.5.4.2.2" xref="S4.SS1.p2.7.m7.5.5.4.2.2.cmml">r</mi><mi id="S4.SS1.p2.7.m7.5.5.4.2.3" xref="S4.SS1.p2.7.m7.5.5.4.2.3.cmml">j</mi><mi id="S4.SS1.p2.7.m7.5.5.4.3" xref="S4.SS1.p2.7.m7.5.5.4.3.cmml">i</mi></msubsup><mo id="S4.SS1.p2.7.m7.5.5.3" xref="S4.SS1.p2.7.m7.5.5.3.cmml">=</mo><mrow id="S4.SS1.p2.7.m7.5.5.2.2" xref="S4.SS1.p2.7.m7.5.5.2.3.cmml"><mo id="S4.SS1.p2.7.m7.5.5.2.2.3" xref="S4.SS1.p2.7.m7.5.5.2.3.1.cmml">{</mo><mrow id="S4.SS1.p2.7.m7.4.4.1.1.1" xref="S4.SS1.p2.7.m7.4.4.1.1.1.cmml"><msub id="S4.SS1.p2.7.m7.4.4.1.1.1.2" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2.cmml"><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.2.2" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2.2.cmml">t</mi><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.2.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2.3.cmml">i</mi></msub><mo id="S4.SS1.p2.7.m7.4.4.1.1.1.1" xref="S4.SS1.p2.7.m7.4.4.1.1.1.1.cmml">∪</mo><msubsup id="S4.SS1.p2.7.m7.4.4.1.1.1.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.cmml"><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.2" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.2.cmml">c</mi><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.3.cmml">j</mi><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.3.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.3.cmml">i</mi></msubsup><mo id="S4.SS1.p2.7.m7.4.4.1.1.1.1a" xref="S4.SS1.p2.7.m7.4.4.1.1.1.1.cmml">∪</mo><msub id="S4.SS1.p2.7.m7.4.4.1.1.1.4" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.cmml"><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.4.2" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.2.cmml">v</mi><msubsup id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.cmml"><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.2" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.2.cmml">c</mi><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.3.cmml">j</mi><mi id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.3" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.3.cmml">i</mi></msubsup></msub></mrow><mo fence="true" lspace="0em" rspace="0em" id="S4.SS1.p2.7.m7.5.5.2.2.4" xref="S4.SS1.p2.7.m7.5.5.2.3.1.cmml">∣</mo><mrow id="S4.SS1.p2.7.m7.5.5.2.2.2" xref="S4.SS1.p2.7.m7.5.5.2.2.2.cmml"><mi id="S4.SS1.p2.7.m7.5.5.2.2.2.3" xref="S4.SS1.p2.7.m7.5.5.2.2.2.3.cmml">i</mi><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.4" xref="S4.SS1.p2.7.m7.5.5.2.2.2.4.cmml">∈</mo><mrow id="S4.SS1.p2.7.m7.5.5.2.2.2.5" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.cmml"><mrow id="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.2" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.1.cmml"><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.2.1" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.1.cmml">[</mo><mn id="S4.SS1.p2.7.m7.1.1" xref="S4.SS1.p2.7.m7.1.1.cmml">1</mn><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.2.2" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.1.cmml">,</mo><mi id="S4.SS1.p2.7.m7.2.2" xref="S4.SS1.p2.7.m7.2.2.cmml">T</mi><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.2.3" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.1.cmml">]</mo></mrow><mo lspace="0em" rspace="0em" id="S4.SS1.p2.7.m7.5.5.2.2.2.5.1" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.1.cmml">​</mo><mtext id="S4.SS1.p2.7.m7.5.5.2.2.2.5.3" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.3a.cmml"> and </mtext><mo lspace="0em" rspace="0em" id="S4.SS1.p2.7.m7.5.5.2.2.2.5.1a" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.1.cmml">​</mo><mi id="S4.SS1.p2.7.m7.5.5.2.2.2.5.4" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.4.cmml">j</mi></mrow><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.6" xref="S4.SS1.p2.7.m7.5.5.2.2.2.6.cmml">∈</mo><mrow id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.2.cmml"><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.2" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.2.cmml">[</mo><mn id="S4.SS1.p2.7.m7.3.3" xref="S4.SS1.p2.7.m7.3.3.cmml">1</mn><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.3" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.2.cmml">,</mo><msub id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.cmml"><mi id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.2" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.2.cmml">C</mi><mi id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.3" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.4" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.2.cmml">]</mo></mrow></mrow><mo id="S4.SS1.p2.7.m7.5.5.2.2.5" xref="S4.SS1.p2.7.m7.5.5.2.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p2.7.m7.5b"><apply id="S4.SS1.p2.7.m7.5.5.cmml" xref="S4.SS1.p2.7.m7.5.5"><eq id="S4.SS1.p2.7.m7.5.5.3.cmml" xref="S4.SS1.p2.7.m7.5.5.3"></eq><apply id="S4.SS1.p2.7.m7.5.5.4.cmml" xref="S4.SS1.p2.7.m7.5.5.4"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.5.5.4.1.cmml" xref="S4.SS1.p2.7.m7.5.5.4">superscript</csymbol><apply id="S4.SS1.p2.7.m7.5.5.4.2.cmml" xref="S4.SS1.p2.7.m7.5.5.4"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.5.5.4.2.1.cmml" xref="S4.SS1.p2.7.m7.5.5.4">subscript</csymbol><ci id="S4.SS1.p2.7.m7.5.5.4.2.2.cmml" xref="S4.SS1.p2.7.m7.5.5.4.2.2">𝑟</ci><ci id="S4.SS1.p2.7.m7.5.5.4.2.3.cmml" xref="S4.SS1.p2.7.m7.5.5.4.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.7.m7.5.5.4.3.cmml" xref="S4.SS1.p2.7.m7.5.5.4.3">𝑖</ci></apply><apply id="S4.SS1.p2.7.m7.5.5.2.3.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2"><csymbol cd="latexml" id="S4.SS1.p2.7.m7.5.5.2.3.1.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.3">conditional-set</csymbol><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1"><union id="S4.SS1.p2.7.m7.4.4.1.1.1.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.1"></union><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.4.4.1.1.1.2.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2">subscript</csymbol><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.2.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2.2">𝑡</ci><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.2.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.2.3">𝑖</ci></apply><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.4.4.1.1.1.3.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3">superscript</csymbol><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3">subscript</csymbol><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.2">𝑐</ci><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.3.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.3.3">𝑖</ci></apply><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.4.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.4.4.1.1.1.4.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4">subscript</csymbol><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.4.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.2">𝑣</ci><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3">superscript</csymbol><apply id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.1.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3">subscript</csymbol><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.2.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.2">𝑐</ci><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.2.3">𝑗</ci></apply><ci id="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.3.cmml" xref="S4.SS1.p2.7.m7.4.4.1.1.1.4.3.3">𝑖</ci></apply></apply></apply><apply id="S4.SS1.p2.7.m7.5.5.2.2.2.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2"><and id="S4.SS1.p2.7.m7.5.5.2.2.2a.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2"></and><apply id="S4.SS1.p2.7.m7.5.5.2.2.2b.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2"><in id="S4.SS1.p2.7.m7.5.5.2.2.2.4.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.4"></in><ci id="S4.SS1.p2.7.m7.5.5.2.2.2.3.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.3">𝑖</ci><apply id="S4.SS1.p2.7.m7.5.5.2.2.2.5.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5"><times id="S4.SS1.p2.7.m7.5.5.2.2.2.5.1.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.1"></times><interval closure="closed" id="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.1.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.2.2"><cn type="integer" id="S4.SS1.p2.7.m7.1.1.cmml" xref="S4.SS1.p2.7.m7.1.1">1</cn><ci id="S4.SS1.p2.7.m7.2.2.cmml" xref="S4.SS1.p2.7.m7.2.2">𝑇</ci></interval><ci id="S4.SS1.p2.7.m7.5.5.2.2.2.5.3a.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.3"><mtext id="S4.SS1.p2.7.m7.5.5.2.2.2.5.3.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.3"> and </mtext></ci><ci id="S4.SS1.p2.7.m7.5.5.2.2.2.5.4.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.5.4">𝑗</ci></apply></apply><apply id="S4.SS1.p2.7.m7.5.5.2.2.2c.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2"><in id="S4.SS1.p2.7.m7.5.5.2.2.2.6.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.6"></in><share href="#S4.SS1.p2.7.m7.5.5.2.2.2.5.cmml" id="S4.SS1.p2.7.m7.5.5.2.2.2d.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2"></share><interval closure="closed" id="S4.SS1.p2.7.m7.5.5.2.2.2.1.2.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1"><cn type="integer" id="S4.SS1.p2.7.m7.3.3.cmml" xref="S4.SS1.p2.7.m7.3.3">1</cn><apply id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.1.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1">subscript</csymbol><ci id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.2.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.2">𝐶</ci><ci id="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.3.cmml" xref="S4.SS1.p2.7.m7.5.5.2.2.2.1.1.1.3">𝑖</ci></apply></interval></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p2.7.m7.5c">r_{j}^{i}=\left\{t_{i}\cup c_{j}^{i}\cup v_{c_{j}^{i}}\mid i\in\left[1,T\right]\text{ and }j\in\left[1,C_{i}\right]\right\}</annotation></semantics></math>.</p>
</div>
<div id="S4.SS1.p3" class="ltx_para">
<p id="S4.SS1.p3.4" class="ltx_p">Given question <math id="S4.SS1.p3.1.m1.1" class="ltx_Math" alttext="\mathbf{q}" display="inline"><semantics id="S4.SS1.p3.1.m1.1a"><mi id="S4.SS1.p3.1.m1.1.1" xref="S4.SS1.p3.1.m1.1.1.cmml">𝐪</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p3.1.m1.1b"><ci id="S4.SS1.p3.1.m1.1.1.cmml" xref="S4.SS1.p3.1.m1.1.1">𝐪</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p3.1.m1.1c">\mathbf{q}</annotation></semantics></math>, we retrieve the most relevant columns using <math id="S4.SS1.p3.2.m2.2" class="ltx_Math" alttext="\text{score}_{\text{BM25}}(\mathbf{q},r_{j}^{i})" display="inline"><semantics id="S4.SS1.p3.2.m2.2a"><mrow id="S4.SS1.p3.2.m2.2.2" xref="S4.SS1.p3.2.m2.2.2.cmml"><msub id="S4.SS1.p3.2.m2.2.2.3" xref="S4.SS1.p3.2.m2.2.2.3.cmml"><mtext id="S4.SS1.p3.2.m2.2.2.3.2" xref="S4.SS1.p3.2.m2.2.2.3.2a.cmml">score</mtext><mtext id="S4.SS1.p3.2.m2.2.2.3.3" xref="S4.SS1.p3.2.m2.2.2.3.3a.cmml">BM25</mtext></msub><mo lspace="0em" rspace="0em" id="S4.SS1.p3.2.m2.2.2.2" xref="S4.SS1.p3.2.m2.2.2.2.cmml">​</mo><mrow id="S4.SS1.p3.2.m2.2.2.1.1" xref="S4.SS1.p3.2.m2.2.2.1.2.cmml"><mo stretchy="false" id="S4.SS1.p3.2.m2.2.2.1.1.2" xref="S4.SS1.p3.2.m2.2.2.1.2.cmml">(</mo><mi id="S4.SS1.p3.2.m2.1.1" xref="S4.SS1.p3.2.m2.1.1.cmml">𝐪</mi><mo id="S4.SS1.p3.2.m2.2.2.1.1.3" xref="S4.SS1.p3.2.m2.2.2.1.2.cmml">,</mo><msubsup id="S4.SS1.p3.2.m2.2.2.1.1.1" xref="S4.SS1.p3.2.m2.2.2.1.1.1.cmml"><mi id="S4.SS1.p3.2.m2.2.2.1.1.1.2.2" xref="S4.SS1.p3.2.m2.2.2.1.1.1.2.2.cmml">r</mi><mi id="S4.SS1.p3.2.m2.2.2.1.1.1.2.3" xref="S4.SS1.p3.2.m2.2.2.1.1.1.2.3.cmml">j</mi><mi id="S4.SS1.p3.2.m2.2.2.1.1.1.3" xref="S4.SS1.p3.2.m2.2.2.1.1.1.3.cmml">i</mi></msubsup><mo stretchy="false" id="S4.SS1.p3.2.m2.2.2.1.1.4" xref="S4.SS1.p3.2.m2.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p3.2.m2.2b"><apply id="S4.SS1.p3.2.m2.2.2.cmml" xref="S4.SS1.p3.2.m2.2.2"><times id="S4.SS1.p3.2.m2.2.2.2.cmml" xref="S4.SS1.p3.2.m2.2.2.2"></times><apply id="S4.SS1.p3.2.m2.2.2.3.cmml" xref="S4.SS1.p3.2.m2.2.2.3"><csymbol cd="ambiguous" id="S4.SS1.p3.2.m2.2.2.3.1.cmml" xref="S4.SS1.p3.2.m2.2.2.3">subscript</csymbol><ci id="S4.SS1.p3.2.m2.2.2.3.2a.cmml" xref="S4.SS1.p3.2.m2.2.2.3.2"><mtext id="S4.SS1.p3.2.m2.2.2.3.2.cmml" xref="S4.SS1.p3.2.m2.2.2.3.2">score</mtext></ci><ci id="S4.SS1.p3.2.m2.2.2.3.3a.cmml" xref="S4.SS1.p3.2.m2.2.2.3.3"><mtext mathsize="70%" id="S4.SS1.p3.2.m2.2.2.3.3.cmml" xref="S4.SS1.p3.2.m2.2.2.3.3">BM25</mtext></ci></apply><interval closure="open" id="S4.SS1.p3.2.m2.2.2.1.2.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1"><ci id="S4.SS1.p3.2.m2.1.1.cmml" xref="S4.SS1.p3.2.m2.1.1">𝐪</ci><apply id="S4.SS1.p3.2.m2.2.2.1.1.1.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p3.2.m2.2.2.1.1.1.1.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1">superscript</csymbol><apply id="S4.SS1.p3.2.m2.2.2.1.1.1.2.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p3.2.m2.2.2.1.1.1.2.1.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1">subscript</csymbol><ci id="S4.SS1.p3.2.m2.2.2.1.1.1.2.2.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1.2.2">𝑟</ci><ci id="S4.SS1.p3.2.m2.2.2.1.1.1.2.3.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1.2.3">𝑗</ci></apply><ci id="S4.SS1.p3.2.m2.2.2.1.1.1.3.cmml" xref="S4.SS1.p3.2.m2.2.2.1.1.1.3">𝑖</ci></apply></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p3.2.m2.2c">\text{score}_{\text{BM25}}(\mathbf{q},r_{j}^{i})</annotation></semantics></math> <math id="S4.SS1.p3.3.m3.2" class="ltx_Math" alttext="\forall i\in\left[1,T\right]" display="inline"><semantics id="S4.SS1.p3.3.m3.2a"><mrow id="S4.SS1.p3.3.m3.2.3" xref="S4.SS1.p3.3.m3.2.3.cmml"><mrow id="S4.SS1.p3.3.m3.2.3.2" xref="S4.SS1.p3.3.m3.2.3.2.cmml"><mo rspace="0.167em" id="S4.SS1.p3.3.m3.2.3.2.1" xref="S4.SS1.p3.3.m3.2.3.2.1.cmml">∀</mo><mi id="S4.SS1.p3.3.m3.2.3.2.2" xref="S4.SS1.p3.3.m3.2.3.2.2.cmml">i</mi></mrow><mo id="S4.SS1.p3.3.m3.2.3.1" xref="S4.SS1.p3.3.m3.2.3.1.cmml">∈</mo><mrow id="S4.SS1.p3.3.m3.2.3.3.2" xref="S4.SS1.p3.3.m3.2.3.3.1.cmml"><mo id="S4.SS1.p3.3.m3.2.3.3.2.1" xref="S4.SS1.p3.3.m3.2.3.3.1.cmml">[</mo><mn id="S4.SS1.p3.3.m3.1.1" xref="S4.SS1.p3.3.m3.1.1.cmml">1</mn><mo id="S4.SS1.p3.3.m3.2.3.3.2.2" xref="S4.SS1.p3.3.m3.2.3.3.1.cmml">,</mo><mi id="S4.SS1.p3.3.m3.2.2" xref="S4.SS1.p3.3.m3.2.2.cmml">T</mi><mo id="S4.SS1.p3.3.m3.2.3.3.2.3" xref="S4.SS1.p3.3.m3.2.3.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p3.3.m3.2b"><apply id="S4.SS1.p3.3.m3.2.3.cmml" xref="S4.SS1.p3.3.m3.2.3"><in id="S4.SS1.p3.3.m3.2.3.1.cmml" xref="S4.SS1.p3.3.m3.2.3.1"></in><apply id="S4.SS1.p3.3.m3.2.3.2.cmml" xref="S4.SS1.p3.3.m3.2.3.2"><csymbol cd="latexml" id="S4.SS1.p3.3.m3.2.3.2.1.cmml" xref="S4.SS1.p3.3.m3.2.3.2.1">for-all</csymbol><ci id="S4.SS1.p3.3.m3.2.3.2.2.cmml" xref="S4.SS1.p3.3.m3.2.3.2.2">𝑖</ci></apply><interval closure="closed" id="S4.SS1.p3.3.m3.2.3.3.1.cmml" xref="S4.SS1.p3.3.m3.2.3.3.2"><cn type="integer" id="S4.SS1.p3.3.m3.1.1.cmml" xref="S4.SS1.p3.3.m3.1.1">1</cn><ci id="S4.SS1.p3.3.m3.2.2.cmml" xref="S4.SS1.p3.3.m3.2.2">𝑇</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p3.3.m3.2c">\forall i\in\left[1,T\right]</annotation></semantics></math> and <math id="S4.SS1.p3.4.m4.2" class="ltx_Math" alttext="j\in\left[1,C_{i}\right]" display="inline"><semantics id="S4.SS1.p3.4.m4.2a"><mrow id="S4.SS1.p3.4.m4.2.2" xref="S4.SS1.p3.4.m4.2.2.cmml"><mi id="S4.SS1.p3.4.m4.2.2.3" xref="S4.SS1.p3.4.m4.2.2.3.cmml">j</mi><mo id="S4.SS1.p3.4.m4.2.2.2" xref="S4.SS1.p3.4.m4.2.2.2.cmml">∈</mo><mrow id="S4.SS1.p3.4.m4.2.2.1.1" xref="S4.SS1.p3.4.m4.2.2.1.2.cmml"><mo id="S4.SS1.p3.4.m4.2.2.1.1.2" xref="S4.SS1.p3.4.m4.2.2.1.2.cmml">[</mo><mn id="S4.SS1.p3.4.m4.1.1" xref="S4.SS1.p3.4.m4.1.1.cmml">1</mn><mo id="S4.SS1.p3.4.m4.2.2.1.1.3" xref="S4.SS1.p3.4.m4.2.2.1.2.cmml">,</mo><msub id="S4.SS1.p3.4.m4.2.2.1.1.1" xref="S4.SS1.p3.4.m4.2.2.1.1.1.cmml"><mi id="S4.SS1.p3.4.m4.2.2.1.1.1.2" xref="S4.SS1.p3.4.m4.2.2.1.1.1.2.cmml">C</mi><mi id="S4.SS1.p3.4.m4.2.2.1.1.1.3" xref="S4.SS1.p3.4.m4.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S4.SS1.p3.4.m4.2.2.1.1.4" xref="S4.SS1.p3.4.m4.2.2.1.2.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p3.4.m4.2b"><apply id="S4.SS1.p3.4.m4.2.2.cmml" xref="S4.SS1.p3.4.m4.2.2"><in id="S4.SS1.p3.4.m4.2.2.2.cmml" xref="S4.SS1.p3.4.m4.2.2.2"></in><ci id="S4.SS1.p3.4.m4.2.2.3.cmml" xref="S4.SS1.p3.4.m4.2.2.3">𝑗</ci><interval closure="closed" id="S4.SS1.p3.4.m4.2.2.1.2.cmml" xref="S4.SS1.p3.4.m4.2.2.1.1"><cn type="integer" id="S4.SS1.p3.4.m4.1.1.cmml" xref="S4.SS1.p3.4.m4.1.1">1</cn><apply id="S4.SS1.p3.4.m4.2.2.1.1.1.cmml" xref="S4.SS1.p3.4.m4.2.2.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p3.4.m4.2.2.1.1.1.1.cmml" xref="S4.SS1.p3.4.m4.2.2.1.1.1">subscript</csymbol><ci id="S4.SS1.p3.4.m4.2.2.1.1.1.2.cmml" xref="S4.SS1.p3.4.m4.2.2.1.1.1.2">𝐶</ci><ci id="S4.SS1.p3.4.m4.2.2.1.1.1.3.cmml" xref="S4.SS1.p3.4.m4.2.2.1.1.1.3">𝑖</ci></apply></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p3.4.m4.2c">j\in\left[1,C_{i}\right]</annotation></semantics></math> <cite class="ltx_cite ltx_citemacro_cite">Robertson et al. (<a href="#bib.bib23" title="" class="ltx_ref">1994</a>)</cite>. A table is retrieved if any of its columns are retrieved.</p>
</div>
<section id="S4.SS1.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">4.1.1 </span>Incorporating for Approximated Query</h4>

<div id="S4.SS1.SSS1.p1" class="ltx_para">
<p id="S4.SS1.SSS1.p1.6" class="ltx_p">The semantic search for schema selection requires a comprehension of the relevance between heterogeneous database information and natural language questions, in addition to interactions across schema elements.
To this end, a trained parser can inherently be a semantic search model for retrieving a sub-schema, where columns and tables are extracted from the approximated query <math id="S4.SS1.SSS1.p1.1.m1.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}" display="inline"><semantics id="S4.SS1.SSS1.p1.1.m1.1a"><msup id="S4.SS1.SSS1.p1.1.m1.1.1" xref="S4.SS1.SSS1.p1.1.m1.1.1.cmml"><mi id="S4.SS1.SSS1.p1.1.m1.1.1.2" xref="S4.SS1.SSS1.p1.1.m1.1.1.2.cmml">𝐬</mi><mo id="S4.SS1.SSS1.p1.1.m1.1.1.3" xref="S4.SS1.SSS1.p1.1.m1.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS1.p1.1.m1.1b"><apply id="S4.SS1.SSS1.p1.1.m1.1.1.cmml" xref="S4.SS1.SSS1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS1.p1.1.m1.1.1.1.cmml" xref="S4.SS1.SSS1.p1.1.m1.1.1">superscript</csymbol><ci id="S4.SS1.SSS1.p1.1.m1.1.1.2.cmml" xref="S4.SS1.SSS1.p1.1.m1.1.1.2">𝐬</ci><ci id="S4.SS1.SSS1.p1.1.m1.1.1.3.cmml" xref="S4.SS1.SSS1.p1.1.m1.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS1.p1.1.m1.1c">\mathbf{s}^{\prime}</annotation></semantics></math>.
We argue that a semantic parser which is performing reasonably on the task, can provide us with an <math id="S4.SS1.SSS1.p1.2.m2.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}" display="inline"><semantics id="S4.SS1.SSS1.p1.2.m2.1a"><msup id="S4.SS1.SSS1.p1.2.m2.1.1" xref="S4.SS1.SSS1.p1.2.m2.1.1.cmml"><mi id="S4.SS1.SSS1.p1.2.m2.1.1.2" xref="S4.SS1.SSS1.p1.2.m2.1.1.2.cmml">𝐬</mi><mo id="S4.SS1.SSS1.p1.2.m2.1.1.3" xref="S4.SS1.SSS1.p1.2.m2.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS1.p1.2.m2.1b"><apply id="S4.SS1.SSS1.p1.2.m2.1.1.cmml" xref="S4.SS1.SSS1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS1.p1.2.m2.1.1.1.cmml" xref="S4.SS1.SSS1.p1.2.m2.1.1">superscript</csymbol><ci id="S4.SS1.SSS1.p1.2.m2.1.1.2.cmml" xref="S4.SS1.SSS1.p1.2.m2.1.1.2">𝐬</ci><ci id="S4.SS1.SSS1.p1.2.m2.1.1.3.cmml" xref="S4.SS1.SSS1.p1.2.m2.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS1.p1.2.m2.1c">\mathbf{s}^{\prime}</annotation></semantics></math>, whose structure would assimilate the structure of the expected final query, <math id="S4.SS1.SSS1.p1.3.m3.1" class="ltx_Math" alttext="\mathbf{s}" display="inline"><semantics id="S4.SS1.SSS1.p1.3.m3.1a"><mi id="S4.SS1.SSS1.p1.3.m3.1.1" xref="S4.SS1.SSS1.p1.3.m3.1.1.cmml">𝐬</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS1.p1.3.m3.1b"><ci id="S4.SS1.SSS1.p1.3.m3.1.1.cmml" xref="S4.SS1.SSS1.p1.3.m3.1.1">𝐬</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS1.p1.3.m3.1c">\mathbf{s}</annotation></semantics></math>. Consequently, we opt to dynamically determine the number of columns to be retrieved by <math id="S4.SS1.SSS1.p1.4.m4.1" class="ltx_Math" alttext="\text{score}_{\text{BM25}}" display="inline"><semantics id="S4.SS1.SSS1.p1.4.m4.1a"><msub id="S4.SS1.SSS1.p1.4.m4.1.1" xref="S4.SS1.SSS1.p1.4.m4.1.1.cmml"><mtext id="S4.SS1.SSS1.p1.4.m4.1.1.2" xref="S4.SS1.SSS1.p1.4.m4.1.1.2a.cmml">score</mtext><mtext id="S4.SS1.SSS1.p1.4.m4.1.1.3" xref="S4.SS1.SSS1.p1.4.m4.1.1.3a.cmml">BM25</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS1.p1.4.m4.1b"><apply id="S4.SS1.SSS1.p1.4.m4.1.1.cmml" xref="S4.SS1.SSS1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS1.p1.4.m4.1.1.1.cmml" xref="S4.SS1.SSS1.p1.4.m4.1.1">subscript</csymbol><ci id="S4.SS1.SSS1.p1.4.m4.1.1.2a.cmml" xref="S4.SS1.SSS1.p1.4.m4.1.1.2"><mtext id="S4.SS1.SSS1.p1.4.m4.1.1.2.cmml" xref="S4.SS1.SSS1.p1.4.m4.1.1.2">score</mtext></ci><ci id="S4.SS1.SSS1.p1.4.m4.1.1.3a.cmml" xref="S4.SS1.SSS1.p1.4.m4.1.1.3"><mtext mathsize="70%" id="S4.SS1.SSS1.p1.4.m4.1.1.3.cmml" xref="S4.SS1.SSS1.p1.4.m4.1.1.3">BM25</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS1.p1.4.m4.1c">\text{score}_{\text{BM25}}</annotation></semantics></math> as proportional to the number of unique columns in <math id="S4.SS1.SSS1.p1.5.m5.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}" display="inline"><semantics id="S4.SS1.SSS1.p1.5.m5.1a"><msup id="S4.SS1.SSS1.p1.5.m5.1.1" xref="S4.SS1.SSS1.p1.5.m5.1.1.cmml"><mi id="S4.SS1.SSS1.p1.5.m5.1.1.2" xref="S4.SS1.SSS1.p1.5.m5.1.1.2.cmml">𝐬</mi><mo id="S4.SS1.SSS1.p1.5.m5.1.1.3" xref="S4.SS1.SSS1.p1.5.m5.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS1.p1.5.m5.1b"><apply id="S4.SS1.SSS1.p1.5.m5.1.1.cmml" xref="S4.SS1.SSS1.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS1.p1.5.m5.1.1.1.cmml" xref="S4.SS1.SSS1.p1.5.m5.1.1">superscript</csymbol><ci id="S4.SS1.SSS1.p1.5.m5.1.1.2.cmml" xref="S4.SS1.SSS1.p1.5.m5.1.1.2">𝐬</ci><ci id="S4.SS1.SSS1.p1.5.m5.1.1.3.cmml" xref="S4.SS1.SSS1.p1.5.m5.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS1.p1.5.m5.1c">\mathbf{s}^{\prime}</annotation></semantics></math>, returned by the approximator. A sub-schema is then obtained by merging schema elements selected by <math id="S4.SS1.SSS1.p1.6.m6.1" class="ltx_Math" alttext="\text{score}_{\text{BM25}}" display="inline"><semantics id="S4.SS1.SSS1.p1.6.m6.1a"><msub id="S4.SS1.SSS1.p1.6.m6.1.1" xref="S4.SS1.SSS1.p1.6.m6.1.1.cmml"><mtext id="S4.SS1.SSS1.p1.6.m6.1.1.2" xref="S4.SS1.SSS1.p1.6.m6.1.1.2a.cmml">score</mtext><mtext id="S4.SS1.SSS1.p1.6.m6.1.1.3" xref="S4.SS1.SSS1.p1.6.m6.1.1.3a.cmml">BM25</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS1.p1.6.m6.1b"><apply id="S4.SS1.SSS1.p1.6.m6.1.1.cmml" xref="S4.SS1.SSS1.p1.6.m6.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS1.p1.6.m6.1.1.1.cmml" xref="S4.SS1.SSS1.p1.6.m6.1.1">subscript</csymbol><ci id="S4.SS1.SSS1.p1.6.m6.1.1.2a.cmml" xref="S4.SS1.SSS1.p1.6.m6.1.1.2"><mtext id="S4.SS1.SSS1.p1.6.m6.1.1.2.cmml" xref="S4.SS1.SSS1.p1.6.m6.1.1.2">score</mtext></ci><ci id="S4.SS1.SSS1.p1.6.m6.1.1.3a.cmml" xref="S4.SS1.SSS1.p1.6.m6.1.1.3"><mtext mathsize="70%" id="S4.SS1.SSS1.p1.6.m6.1.1.3.cmml" xref="S4.SS1.SSS1.p1.6.m6.1.1.3">BM25</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS1.p1.6.m6.1c">\text{score}_{\text{BM25}}</annotation></semantics></math> with elements from the approximated query.</p>
</div>
</section>
<section id="S4.SS1.SSS2" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">4.1.2 </span>Approximating for Longer Schemata</h4>

<div id="S4.SS1.SSS2.p1" class="ltx_para">
<p id="S4.SS1.SSS2.p1.2" class="ltx_p">To further reduce the computational workload, we opt for using a <span id="S4.SS1.SSS2.p1.2.1" class="ltx_text ltx_font_italic">smaller</span> model for computing the approximated query, <math id="S4.SS1.SSS2.p1.1.m1.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}" display="inline"><semantics id="S4.SS1.SSS2.p1.1.m1.1a"><msup id="S4.SS1.SSS2.p1.1.m1.1.1" xref="S4.SS1.SSS2.p1.1.m1.1.1.cmml"><mi id="S4.SS1.SSS2.p1.1.m1.1.1.2" xref="S4.SS1.SSS2.p1.1.m1.1.1.2.cmml">𝐬</mi><mo id="S4.SS1.SSS2.p1.1.m1.1.1.3" xref="S4.SS1.SSS2.p1.1.m1.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p1.1.m1.1b"><apply id="S4.SS1.SSS2.p1.1.m1.1.1.cmml" xref="S4.SS1.SSS2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS2.p1.1.m1.1.1.1.cmml" xref="S4.SS1.SSS2.p1.1.m1.1.1">superscript</csymbol><ci id="S4.SS1.SSS2.p1.1.m1.1.1.2.cmml" xref="S4.SS1.SSS2.p1.1.m1.1.1.2">𝐬</ci><ci id="S4.SS1.SSS2.p1.1.m1.1.1.3.cmml" xref="S4.SS1.SSS2.p1.1.m1.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p1.1.m1.1c">\mathbf{s}^{\prime}</annotation></semantics></math>. However, smaller models usually have shorter context windows (i.e. <math id="S4.SS1.SSS2.p1.2.m2.1" class="ltx_Math" alttext="&lt;2" display="inline"><semantics id="S4.SS1.SSS2.p1.2.m2.1a"><mrow id="S4.SS1.SSS2.p1.2.m2.1.1" xref="S4.SS1.SSS2.p1.2.m2.1.1.cmml"><mi id="S4.SS1.SSS2.p1.2.m2.1.1.2" xref="S4.SS1.SSS2.p1.2.m2.1.1.2.cmml"></mi><mo id="S4.SS1.SSS2.p1.2.m2.1.1.1" xref="S4.SS1.SSS2.p1.2.m2.1.1.1.cmml">&lt;</mo><mn id="S4.SS1.SSS2.p1.2.m2.1.1.3" xref="S4.SS1.SSS2.p1.2.m2.1.1.3.cmml">2</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p1.2.m2.1b"><apply id="S4.SS1.SSS2.p1.2.m2.1.1.cmml" xref="S4.SS1.SSS2.p1.2.m2.1.1"><lt id="S4.SS1.SSS2.p1.2.m2.1.1.1.cmml" xref="S4.SS1.SSS2.p1.2.m2.1.1.1"></lt><csymbol cd="latexml" id="S4.SS1.SSS2.p1.2.m2.1.1.2.cmml" xref="S4.SS1.SSS2.p1.2.m2.1.1.2">absent</csymbol><cn type="integer" id="S4.SS1.SSS2.p1.2.m2.1.1.3.cmml" xref="S4.SS1.SSS2.p1.2.m2.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p1.2.m2.1c">&lt;2</annotation></semantics></math>k tokens), and, as such, they cannot be easily scaled to the requirements of larger schemata. To this end, we propose an approach that enables transformer-based encoders to process longer schemata, in a parallelised manner.</p>
</div>
<div id="S4.SS1.SSS2.p2" class="ltx_para">
<p id="S4.SS1.SSS2.p2.1" class="ltx_p">We start with FastRAT <cite class="ltx_cite ltx_citemacro_cite">Vougiouklis et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite>, which exploits a decoder-free architecture for efficient text-to-SQL parsing. Given a concatenation of the input natural language question <math id="S4.SS1.SSS2.p2.1.m1.1" class="ltx_Math" alttext="\mathbf{q}" display="inline"><semantics id="S4.SS1.SSS2.p2.1.m1.1a"><mi id="S4.SS1.SSS2.p2.1.m1.1.1" xref="S4.SS1.SSS2.p2.1.m1.1.1.cmml">𝐪</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p2.1.m1.1b"><ci id="S4.SS1.SSS2.p2.1.m1.1.1.cmml" xref="S4.SS1.SSS2.p2.1.m1.1.1">𝐪</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p2.1.m1.1c">\mathbf{q}</annotation></semantics></math> with the column and table names of a database schema, FastRAT computes the SQL operation in which each element of the input schema would participate in the expected SQL query. We refer to these SQL operations as SQL Semantic Prediction (SSP) labels <cite class="ltx_cite ltx_citemacro_cite">Yu et al. (<a href="#bib.bib31" title="" class="ltx_ref">2021</a>)</cite>. SQL queries are then deterministically constructed from the predicted SSP labels. We introduce a schema splitting strategy to scale the model up to the requirements of schemata comprising several columns.</p>
</div>
<div id="S4.SS1.SSS2.p3" class="ltx_para">
<p id="S4.SS1.SSS2.p3.5" class="ltx_p">We augment the input embedding matrix of the model, with two special schema-completion tokens, <span id="S4.SS1.SSS2.p3.5.1" class="ltx_text ltx_font_typewriter">[full_schema]</span> and <span id="S4.SS1.SSS2.p3.5.2" class="ltx_text ltx_font_typewriter">[part_schema]</span>, which are used for signalling cases in which a full and a partial schema are provided as input respectively. Our goal is to split a schema consisting of <math id="S4.SS1.SSS2.p3.1.m1.1" class="ltx_Math" alttext="\sum_{j=1}^{T}C_{j}" display="inline"><semantics id="S4.SS1.SSS2.p3.1.m1.1a"><mrow id="S4.SS1.SSS2.p3.1.m1.1.1" xref="S4.SS1.SSS2.p3.1.m1.1.1.cmml"><msubsup id="S4.SS1.SSS2.p3.1.m1.1.1.1" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.cmml"><mo id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.2" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.2.cmml">∑</mo><mrow id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.cmml"><mi id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.2" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.2.cmml">j</mi><mo id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.1" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.1.cmml">=</mo><mn id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.3" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="S4.SS1.SSS2.p3.1.m1.1.1.1.3" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.3.cmml">T</mi></msubsup><msub id="S4.SS1.SSS2.p3.1.m1.1.1.2" xref="S4.SS1.SSS2.p3.1.m1.1.1.2.cmml"><mi id="S4.SS1.SSS2.p3.1.m1.1.1.2.2" xref="S4.SS1.SSS2.p3.1.m1.1.1.2.2.cmml">C</mi><mi id="S4.SS1.SSS2.p3.1.m1.1.1.2.3" xref="S4.SS1.SSS2.p3.1.m1.1.1.2.3.cmml">j</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p3.1.m1.1b"><apply id="S4.SS1.SSS2.p3.1.m1.1.1.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1"><apply id="S4.SS1.SSS2.p3.1.m1.1.1.1.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS2.p3.1.m1.1.1.1.1.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1">superscript</csymbol><apply id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.1.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1">subscript</csymbol><sum id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.2.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.2"></sum><apply id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3"><eq id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.1.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.1"></eq><ci id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.2.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.2">𝑗</ci><cn type="integer" id="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.3.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.2.3.3">1</cn></apply></apply><ci id="S4.SS1.SSS2.p3.1.m1.1.1.1.3.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.1.3">𝑇</ci></apply><apply id="S4.SS1.SSS2.p3.1.m1.1.1.2.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.2"><csymbol cd="ambiguous" id="S4.SS1.SSS2.p3.1.m1.1.1.2.1.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.2">subscript</csymbol><ci id="S4.SS1.SSS2.p3.1.m1.1.1.2.2.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.2.2">𝐶</ci><ci id="S4.SS1.SSS2.p3.1.m1.1.1.2.3.cmml" xref="S4.SS1.SSS2.p3.1.m1.1.1.2.3">𝑗</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p3.1.m1.1c">\sum_{j=1}^{T}C_{j}</annotation></semantics></math> columns into <math id="S4.SS1.SSS2.p3.2.m2.1" class="ltx_Math" alttext="r_{m}" display="inline"><semantics id="S4.SS1.SSS2.p3.2.m2.1a"><msub id="S4.SS1.SSS2.p3.2.m2.1.1" xref="S4.SS1.SSS2.p3.2.m2.1.1.cmml"><mi id="S4.SS1.SSS2.p3.2.m2.1.1.2" xref="S4.SS1.SSS2.p3.2.m2.1.1.2.cmml">r</mi><mi id="S4.SS1.SSS2.p3.2.m2.1.1.3" xref="S4.SS1.SSS2.p3.2.m2.1.1.3.cmml">m</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p3.2.m2.1b"><apply id="S4.SS1.SSS2.p3.2.m2.1.1.cmml" xref="S4.SS1.SSS2.p3.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS2.p3.2.m2.1.1.1.cmml" xref="S4.SS1.SSS2.p3.2.m2.1.1">subscript</csymbol><ci id="S4.SS1.SSS2.p3.2.m2.1.1.2.cmml" xref="S4.SS1.SSS2.p3.2.m2.1.1.2">𝑟</ci><ci id="S4.SS1.SSS2.p3.2.m2.1.1.3.cmml" xref="S4.SS1.SSS2.p3.2.m2.1.1.3">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p3.2.m2.1c">r_{m}</annotation></semantics></math> splits s.t. each split includes a maximum, pre-defined number of columns <math id="S4.SS1.SSS2.p3.3.m3.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S4.SS1.SSS2.p3.3.m3.1a"><mi id="S4.SS1.SSS2.p3.3.m3.1.1" xref="S4.SS1.SSS2.p3.3.m3.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p3.3.m3.1b"><ci id="S4.SS1.SSS2.p3.3.m3.1.1.cmml" xref="S4.SS1.SSS2.p3.3.m3.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p3.3.m3.1c">r</annotation></semantics></math>. Each split consists of the question tokens, a single schema-completion token, the table names of the input <math id="S4.SS1.SSS2.p3.4.m4.1" class="ltx_Math" alttext="\mathbf{D}" display="inline"><semantics id="S4.SS1.SSS2.p3.4.m4.1a"><mi id="S4.SS1.SSS2.p3.4.m4.1.1" xref="S4.SS1.SSS2.p3.4.m4.1.1.cmml">𝐃</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p3.4.m4.1b"><ci id="S4.SS1.SSS2.p3.4.m4.1.1.cmml" xref="S4.SS1.SSS2.p3.4.m4.1.1">𝐃</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p3.4.m4.1c">\mathbf{D}</annotation></semantics></math> and up to a maximum <math id="S4.SS1.SSS2.p3.5.m5.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S4.SS1.SSS2.p3.5.m5.1a"><mi id="S4.SS1.SSS2.p3.5.m5.1.1" xref="S4.SS1.SSS2.p3.5.m5.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p3.5.m5.1b"><ci id="S4.SS1.SSS2.p3.5.m5.1.1.cmml" xref="S4.SS1.SSS2.p3.5.m5.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p3.5.m5.1c">r</annotation></semantics></math> number of columns allocated to this particular split (see Algorithm <a href="#algorithm1" title="In 4.1.2 Approximating for Longer Schemata ‣ 4.1 Schema Selection ‣ 4 Database Context Selection ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> for further details).</p>
</div>
<figure id="algorithm1" class="ltx_float ltx_algorithm">
<div id="algorithm1.16" class="ltx_listing ltx_lst_numbers_left ltx_listing">
<div id="algorithm1.4.4" class="ltx_listingline">



<span id="algorithm1.4.4.5" class="ltx_text"><span id="algorithm1.4.4.5.1" class="ltx_text ltx_font_bold">Input:</span> </span>
<span id="algorithm1.3.3.3" class="ltx_text ltx_font_typewriter" style="color:#0000FF;"># col. name tokens concat.
<br class="ltx_break"><math id="algorithm1.1.1.1.m1.3" class="ltx_Math" alttext="\mathbf{c}^{\text{tok}}\leftarrow[c_{1}^{\text{tok}_{1}},\ldots,c_{C_{T}}^{\text{tok}_{T}}]" display="inline"><semantics id="algorithm1.1.1.1.m1.3a"><mrow id="algorithm1.1.1.1.m1.3.3" xref="algorithm1.1.1.1.m1.3.3.cmml"><msup id="algorithm1.1.1.1.m1.3.3.4" xref="algorithm1.1.1.1.m1.3.3.4.cmml"><mi mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.4.2" xref="algorithm1.1.1.1.m1.3.3.4.2.cmml">𝐜</mi><mtext mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.4.3" xref="algorithm1.1.1.1.m1.3.3.4.3a.cmml">tok</mtext></msup><mo mathcolor="#000000" stretchy="false" id="algorithm1.1.1.1.m1.3.3.3" xref="algorithm1.1.1.1.m1.3.3.3.cmml">←</mo><mrow id="algorithm1.1.1.1.m1.3.3.2.2" xref="algorithm1.1.1.1.m1.3.3.2.3.cmml"><mo mathcolor="#000000" stretchy="false" id="algorithm1.1.1.1.m1.3.3.2.2.3" xref="algorithm1.1.1.1.m1.3.3.2.3.cmml">[</mo><msubsup id="algorithm1.1.1.1.m1.2.2.1.1.1" xref="algorithm1.1.1.1.m1.2.2.1.1.1.cmml"><mi mathcolor="#000000" id="algorithm1.1.1.1.m1.2.2.1.1.1.2.2" xref="algorithm1.1.1.1.m1.2.2.1.1.1.2.2.cmml">c</mi><mn mathcolor="#000000" id="algorithm1.1.1.1.m1.2.2.1.1.1.2.3" xref="algorithm1.1.1.1.m1.2.2.1.1.1.2.3.cmml">1</mn><msub id="algorithm1.1.1.1.m1.2.2.1.1.1.3" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3.cmml"><mtext mathcolor="#000000" id="algorithm1.1.1.1.m1.2.2.1.1.1.3.2" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3.2a.cmml">tok</mtext><mn mathcolor="#000000" id="algorithm1.1.1.1.m1.2.2.1.1.1.3.3" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3.3.cmml">1</mn></msub></msubsup><mo mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.4" xref="algorithm1.1.1.1.m1.3.3.2.3.cmml">,</mo><mi mathcolor="#000000" mathvariant="normal" id="algorithm1.1.1.1.m1.1.1" xref="algorithm1.1.1.1.m1.1.1.cmml">…</mi><mo mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.5" xref="algorithm1.1.1.1.m1.3.3.2.3.cmml">,</mo><msubsup id="algorithm1.1.1.1.m1.3.3.2.2.2" xref="algorithm1.1.1.1.m1.3.3.2.2.2.cmml"><mi mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.2.2.2" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.2.cmml">c</mi><msub id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.cmml"><mi mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.2" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.2.cmml">C</mi><mi mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.3" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.3.cmml">T</mi></msub><msub id="algorithm1.1.1.1.m1.3.3.2.2.2.3" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3.cmml"><mtext mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.2.3.2" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3.2a.cmml">tok</mtext><mi mathcolor="#000000" id="algorithm1.1.1.1.m1.3.3.2.2.2.3.3" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3.3.cmml">T</mi></msub></msubsup><mo mathcolor="#000000" stretchy="false" id="algorithm1.1.1.1.m1.3.3.2.2.6" xref="algorithm1.1.1.1.m1.3.3.2.3.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.1.1.1.m1.3b"><apply id="algorithm1.1.1.1.m1.3.3.cmml" xref="algorithm1.1.1.1.m1.3.3"><ci id="algorithm1.1.1.1.m1.3.3.3.cmml" xref="algorithm1.1.1.1.m1.3.3.3">←</ci><apply id="algorithm1.1.1.1.m1.3.3.4.cmml" xref="algorithm1.1.1.1.m1.3.3.4"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.3.3.4.1.cmml" xref="algorithm1.1.1.1.m1.3.3.4">superscript</csymbol><ci id="algorithm1.1.1.1.m1.3.3.4.2.cmml" xref="algorithm1.1.1.1.m1.3.3.4.2">𝐜</ci><ci id="algorithm1.1.1.1.m1.3.3.4.3a.cmml" xref="algorithm1.1.1.1.m1.3.3.4.3"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.1.1.1.m1.3.3.4.3.cmml" xref="algorithm1.1.1.1.m1.3.3.4.3">tok</mtext></ci></apply><list id="algorithm1.1.1.1.m1.3.3.2.3.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2"><apply id="algorithm1.1.1.1.m1.2.2.1.1.1.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.2.2.1.1.1.1.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1">superscript</csymbol><apply id="algorithm1.1.1.1.m1.2.2.1.1.1.2.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.2.2.1.1.1.2.1.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1">subscript</csymbol><ci id="algorithm1.1.1.1.m1.2.2.1.1.1.2.2.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.2.2">𝑐</ci><cn type="integer" id="algorithm1.1.1.1.m1.2.2.1.1.1.2.3.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.2.3">1</cn></apply><apply id="algorithm1.1.1.1.m1.2.2.1.1.1.3.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.2.2.1.1.1.3.1.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3">subscript</csymbol><ci id="algorithm1.1.1.1.m1.2.2.1.1.1.3.2a.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3.2"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.1.1.1.m1.2.2.1.1.1.3.2.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3.2">tok</mtext></ci><cn type="integer" id="algorithm1.1.1.1.m1.2.2.1.1.1.3.3.cmml" xref="algorithm1.1.1.1.m1.2.2.1.1.1.3.3">1</cn></apply></apply><ci id="algorithm1.1.1.1.m1.1.1.cmml" xref="algorithm1.1.1.1.m1.1.1">…</ci><apply id="algorithm1.1.1.1.m1.3.3.2.2.2.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.3.3.2.2.2.1.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2">superscript</csymbol><apply id="algorithm1.1.1.1.m1.3.3.2.2.2.2.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.3.3.2.2.2.2.1.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2">subscript</csymbol><ci id="algorithm1.1.1.1.m1.3.3.2.2.2.2.2.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.2">𝑐</ci><apply id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.1.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3">subscript</csymbol><ci id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.2.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.2">𝐶</ci><ci id="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.3.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.2.3.3">𝑇</ci></apply></apply><apply id="algorithm1.1.1.1.m1.3.3.2.2.2.3.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3"><csymbol cd="ambiguous" id="algorithm1.1.1.1.m1.3.3.2.2.2.3.1.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3">subscript</csymbol><ci id="algorithm1.1.1.1.m1.3.3.2.2.2.3.2a.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3.2"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.1.1.1.m1.3.3.2.2.2.3.2.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3.2">tok</mtext></ci><ci id="algorithm1.1.1.1.m1.3.3.2.2.2.3.3.cmml" xref="algorithm1.1.1.1.m1.3.3.2.2.2.3.3">𝑇</ci></apply></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.1.1.1.m1.3c">\mathbf{c}^{\text{tok}}\leftarrow[c_{1}^{\text{tok}_{1}},\ldots,c_{C_{T}}^{\text{tok}_{T}}]</annotation></semantics></math>
<br class="ltx_break"># flatten concat. of tab. name tokens
<br class="ltx_break"><math id="algorithm1.2.2.2.m2.4" class="ltx_Math" alttext="\mathbf{t}^{\text{tok}}\leftarrow[t_{1}^{\text{tok}},t_{2}^{\text{tok}},\ldots,t_{T}^{\text{tok}}]" display="inline"><semantics id="algorithm1.2.2.2.m2.4a"><mrow id="algorithm1.2.2.2.m2.4.4" xref="algorithm1.2.2.2.m2.4.4.cmml"><msup id="algorithm1.2.2.2.m2.4.4.5" xref="algorithm1.2.2.2.m2.4.4.5.cmml"><mi mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.5.2" xref="algorithm1.2.2.2.m2.4.4.5.2.cmml">𝐭</mi><mtext mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.5.3" xref="algorithm1.2.2.2.m2.4.4.5.3a.cmml">tok</mtext></msup><mo mathcolor="#000000" stretchy="false" id="algorithm1.2.2.2.m2.4.4.4" xref="algorithm1.2.2.2.m2.4.4.4.cmml">←</mo><mrow id="algorithm1.2.2.2.m2.4.4.3.3" xref="algorithm1.2.2.2.m2.4.4.3.4.cmml"><mo mathcolor="#000000" stretchy="false" id="algorithm1.2.2.2.m2.4.4.3.3.4" xref="algorithm1.2.2.2.m2.4.4.3.4.cmml">[</mo><msubsup id="algorithm1.2.2.2.m2.2.2.1.1.1" xref="algorithm1.2.2.2.m2.2.2.1.1.1.cmml"><mi mathcolor="#000000" id="algorithm1.2.2.2.m2.2.2.1.1.1.2.2" xref="algorithm1.2.2.2.m2.2.2.1.1.1.2.2.cmml">t</mi><mn mathcolor="#000000" id="algorithm1.2.2.2.m2.2.2.1.1.1.2.3" xref="algorithm1.2.2.2.m2.2.2.1.1.1.2.3.cmml">1</mn><mtext mathcolor="#000000" id="algorithm1.2.2.2.m2.2.2.1.1.1.3" xref="algorithm1.2.2.2.m2.2.2.1.1.1.3a.cmml">tok</mtext></msubsup><mo mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.3.3.5" xref="algorithm1.2.2.2.m2.4.4.3.4.cmml">,</mo><msubsup id="algorithm1.2.2.2.m2.3.3.2.2.2" xref="algorithm1.2.2.2.m2.3.3.2.2.2.cmml"><mi mathcolor="#000000" id="algorithm1.2.2.2.m2.3.3.2.2.2.2.2" xref="algorithm1.2.2.2.m2.3.3.2.2.2.2.2.cmml">t</mi><mn mathcolor="#000000" id="algorithm1.2.2.2.m2.3.3.2.2.2.2.3" xref="algorithm1.2.2.2.m2.3.3.2.2.2.2.3.cmml">2</mn><mtext mathcolor="#000000" id="algorithm1.2.2.2.m2.3.3.2.2.2.3" xref="algorithm1.2.2.2.m2.3.3.2.2.2.3a.cmml">tok</mtext></msubsup><mo mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.3.3.6" xref="algorithm1.2.2.2.m2.4.4.3.4.cmml">,</mo><mi mathcolor="#000000" mathvariant="normal" id="algorithm1.2.2.2.m2.1.1" xref="algorithm1.2.2.2.m2.1.1.cmml">…</mi><mo mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.3.3.7" xref="algorithm1.2.2.2.m2.4.4.3.4.cmml">,</mo><msubsup id="algorithm1.2.2.2.m2.4.4.3.3.3" xref="algorithm1.2.2.2.m2.4.4.3.3.3.cmml"><mi mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.3.3.3.2.2" xref="algorithm1.2.2.2.m2.4.4.3.3.3.2.2.cmml">t</mi><mi mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.3.3.3.2.3" xref="algorithm1.2.2.2.m2.4.4.3.3.3.2.3.cmml">T</mi><mtext mathcolor="#000000" id="algorithm1.2.2.2.m2.4.4.3.3.3.3" xref="algorithm1.2.2.2.m2.4.4.3.3.3.3a.cmml">tok</mtext></msubsup><mo mathcolor="#000000" stretchy="false" id="algorithm1.2.2.2.m2.4.4.3.3.8" xref="algorithm1.2.2.2.m2.4.4.3.4.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.2.2.2.m2.4b"><apply id="algorithm1.2.2.2.m2.4.4.cmml" xref="algorithm1.2.2.2.m2.4.4"><ci id="algorithm1.2.2.2.m2.4.4.4.cmml" xref="algorithm1.2.2.2.m2.4.4.4">←</ci><apply id="algorithm1.2.2.2.m2.4.4.5.cmml" xref="algorithm1.2.2.2.m2.4.4.5"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.4.4.5.1.cmml" xref="algorithm1.2.2.2.m2.4.4.5">superscript</csymbol><ci id="algorithm1.2.2.2.m2.4.4.5.2.cmml" xref="algorithm1.2.2.2.m2.4.4.5.2">𝐭</ci><ci id="algorithm1.2.2.2.m2.4.4.5.3a.cmml" xref="algorithm1.2.2.2.m2.4.4.5.3"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.2.2.2.m2.4.4.5.3.cmml" xref="algorithm1.2.2.2.m2.4.4.5.3">tok</mtext></ci></apply><list id="algorithm1.2.2.2.m2.4.4.3.4.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3"><apply id="algorithm1.2.2.2.m2.2.2.1.1.1.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.2.2.1.1.1.1.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1">superscript</csymbol><apply id="algorithm1.2.2.2.m2.2.2.1.1.1.2.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.2.2.1.1.1.2.1.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1">subscript</csymbol><ci id="algorithm1.2.2.2.m2.2.2.1.1.1.2.2.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1.2.2">𝑡</ci><cn type="integer" id="algorithm1.2.2.2.m2.2.2.1.1.1.2.3.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1.2.3">1</cn></apply><ci id="algorithm1.2.2.2.m2.2.2.1.1.1.3a.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1.3"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.2.2.2.m2.2.2.1.1.1.3.cmml" xref="algorithm1.2.2.2.m2.2.2.1.1.1.3">tok</mtext></ci></apply><apply id="algorithm1.2.2.2.m2.3.3.2.2.2.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.3.3.2.2.2.1.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2">superscript</csymbol><apply id="algorithm1.2.2.2.m2.3.3.2.2.2.2.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.3.3.2.2.2.2.1.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2">subscript</csymbol><ci id="algorithm1.2.2.2.m2.3.3.2.2.2.2.2.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2.2.2">𝑡</ci><cn type="integer" id="algorithm1.2.2.2.m2.3.3.2.2.2.2.3.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2.2.3">2</cn></apply><ci id="algorithm1.2.2.2.m2.3.3.2.2.2.3a.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2.3"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.2.2.2.m2.3.3.2.2.2.3.cmml" xref="algorithm1.2.2.2.m2.3.3.2.2.2.3">tok</mtext></ci></apply><ci id="algorithm1.2.2.2.m2.1.1.cmml" xref="algorithm1.2.2.2.m2.1.1">…</ci><apply id="algorithm1.2.2.2.m2.4.4.3.3.3.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.4.4.3.3.3.1.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3">superscript</csymbol><apply id="algorithm1.2.2.2.m2.4.4.3.3.3.2.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3"><csymbol cd="ambiguous" id="algorithm1.2.2.2.m2.4.4.3.3.3.2.1.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3">subscript</csymbol><ci id="algorithm1.2.2.2.m2.4.4.3.3.3.2.2.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3.2.2">𝑡</ci><ci id="algorithm1.2.2.2.m2.4.4.3.3.3.2.3.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3.2.3">𝑇</ci></apply><ci id="algorithm1.2.2.2.m2.4.4.3.3.3.3a.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3.3"><mtext mathcolor="#0000FF" mathsize="70%" id="algorithm1.2.2.2.m2.4.4.3.3.3.3.cmml" xref="algorithm1.2.2.2.m2.4.4.3.3.3.3">tok</mtext></ci></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.2.2.2.m2.4c">\mathbf{t}^{\text{tok}}\leftarrow[t_{1}^{\text{tok}},t_{2}^{\text{tok}},\ldots,t_{T}^{\text{tok}}]</annotation></semantics></math>
<br class="ltx_break"><math id="algorithm1.3.3.3.m3.3" class="ltx_Math" alttext="\mathbf{q}\leftarrow[q_{1},\ldots,q_{Q}]" display="inline"><semantics id="algorithm1.3.3.3.m3.3a"><mrow id="algorithm1.3.3.3.m3.3.3" xref="algorithm1.3.3.3.m3.3.3.cmml"><mi mathcolor="#000000" id="algorithm1.3.3.3.m3.3.3.4" xref="algorithm1.3.3.3.m3.3.3.4.cmml">𝐪</mi><mo mathcolor="#000000" stretchy="false" id="algorithm1.3.3.3.m3.3.3.3" xref="algorithm1.3.3.3.m3.3.3.3.cmml">←</mo><mrow id="algorithm1.3.3.3.m3.3.3.2.2" xref="algorithm1.3.3.3.m3.3.3.2.3.cmml"><mo mathcolor="#000000" stretchy="false" id="algorithm1.3.3.3.m3.3.3.2.2.3" xref="algorithm1.3.3.3.m3.3.3.2.3.cmml">[</mo><msub id="algorithm1.3.3.3.m3.2.2.1.1.1" xref="algorithm1.3.3.3.m3.2.2.1.1.1.cmml"><mi mathcolor="#000000" id="algorithm1.3.3.3.m3.2.2.1.1.1.2" xref="algorithm1.3.3.3.m3.2.2.1.1.1.2.cmml">q</mi><mn mathcolor="#000000" id="algorithm1.3.3.3.m3.2.2.1.1.1.3" xref="algorithm1.3.3.3.m3.2.2.1.1.1.3.cmml">1</mn></msub><mo mathcolor="#000000" id="algorithm1.3.3.3.m3.3.3.2.2.4" xref="algorithm1.3.3.3.m3.3.3.2.3.cmml">,</mo><mi mathcolor="#000000" mathvariant="normal" id="algorithm1.3.3.3.m3.1.1" xref="algorithm1.3.3.3.m3.1.1.cmml">…</mi><mo mathcolor="#000000" id="algorithm1.3.3.3.m3.3.3.2.2.5" xref="algorithm1.3.3.3.m3.3.3.2.3.cmml">,</mo><msub id="algorithm1.3.3.3.m3.3.3.2.2.2" xref="algorithm1.3.3.3.m3.3.3.2.2.2.cmml"><mi mathcolor="#000000" id="algorithm1.3.3.3.m3.3.3.2.2.2.2" xref="algorithm1.3.3.3.m3.3.3.2.2.2.2.cmml">q</mi><mi mathcolor="#000000" id="algorithm1.3.3.3.m3.3.3.2.2.2.3" xref="algorithm1.3.3.3.m3.3.3.2.2.2.3.cmml">Q</mi></msub><mo mathcolor="#000000" stretchy="false" id="algorithm1.3.3.3.m3.3.3.2.2.6" xref="algorithm1.3.3.3.m3.3.3.2.3.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.3.3.3.m3.3b"><apply id="algorithm1.3.3.3.m3.3.3.cmml" xref="algorithm1.3.3.3.m3.3.3"><ci id="algorithm1.3.3.3.m3.3.3.3.cmml" xref="algorithm1.3.3.3.m3.3.3.3">←</ci><ci id="algorithm1.3.3.3.m3.3.3.4.cmml" xref="algorithm1.3.3.3.m3.3.3.4">𝐪</ci><list id="algorithm1.3.3.3.m3.3.3.2.3.cmml" xref="algorithm1.3.3.3.m3.3.3.2.2"><apply id="algorithm1.3.3.3.m3.2.2.1.1.1.cmml" xref="algorithm1.3.3.3.m3.2.2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.3.3.3.m3.2.2.1.1.1.1.cmml" xref="algorithm1.3.3.3.m3.2.2.1.1.1">subscript</csymbol><ci id="algorithm1.3.3.3.m3.2.2.1.1.1.2.cmml" xref="algorithm1.3.3.3.m3.2.2.1.1.1.2">𝑞</ci><cn type="integer" id="algorithm1.3.3.3.m3.2.2.1.1.1.3.cmml" xref="algorithm1.3.3.3.m3.2.2.1.1.1.3">1</cn></apply><ci id="algorithm1.3.3.3.m3.1.1.cmml" xref="algorithm1.3.3.3.m3.1.1">…</ci><apply id="algorithm1.3.3.3.m3.3.3.2.2.2.cmml" xref="algorithm1.3.3.3.m3.3.3.2.2.2"><csymbol cd="ambiguous" id="algorithm1.3.3.3.m3.3.3.2.2.2.1.cmml" xref="algorithm1.3.3.3.m3.3.3.2.2.2">subscript</csymbol><ci id="algorithm1.3.3.3.m3.3.3.2.2.2.2.cmml" xref="algorithm1.3.3.3.m3.3.3.2.2.2.2">𝑞</ci><ci id="algorithm1.3.3.3.m3.3.3.2.2.2.3.cmml" xref="algorithm1.3.3.3.m3.3.3.2.2.2.3">𝑄</ci></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.3.3.3.m3.3c">\mathbf{q}\leftarrow[q_{1},\ldots,q_{Q}]</annotation></semantics></math></span> <span id="algorithm1.4.4.4" class="ltx_text ltx_font_typewriter" style="color:#0000FF;"> # q tokens
<br class="ltx_break"><math id="algorithm1.4.4.4.m1.1" class="ltx_Math" alttext="r" display="inline"><semantics id="algorithm1.4.4.4.m1.1a"><mi mathcolor="#000000" id="algorithm1.4.4.4.m1.1.1" xref="algorithm1.4.4.4.m1.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="algorithm1.4.4.4.m1.1b"><ci id="algorithm1.4.4.4.m1.1.1.cmml" xref="algorithm1.4.4.4.m1.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.4.4.4.m1.1c">r</annotation></semantics></math></span> <span id="algorithm1.4.4.6" class="ltx_text ltx_font_typewriter">: int</span>

</div>
<div id="algorithm1.5.5" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">1</span>
<span id="algorithm1.5.5.1" class="ltx_text ltx_font_typewriter">splits</span> <math id="algorithm1.5.5.m1.1" class="ltx_Math" alttext="\leftarrow\left[\,\right]" display="inline"><semantics id="algorithm1.5.5.m1.1a"><mrow id="algorithm1.5.5.m1.1.1" xref="algorithm1.5.5.m1.1.1.cmml"><mi id="algorithm1.5.5.m1.1.1.2" xref="algorithm1.5.5.m1.1.1.2.cmml"></mi><mo stretchy="false" id="algorithm1.5.5.m1.1.1.1" xref="algorithm1.5.5.m1.1.1.1.cmml">←</mo><mrow id="algorithm1.5.5.m1.1.1.3.2" xref="algorithm1.5.5.m1.1.1.cmml"><mo rspace="0.170em" id="algorithm1.5.5.m1.1.1.3.2.1" xref="algorithm1.5.5.m1.1.1.3.1.cmml">[</mo><mo id="algorithm1.5.5.m1.1.1.3.2.2" xref="algorithm1.5.5.m1.1.1.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.5.5.m1.1b"><apply id="algorithm1.5.5.m1.1.1.cmml" xref="algorithm1.5.5.m1.1.1"><ci id="algorithm1.5.5.m1.1.1.1.cmml" xref="algorithm1.5.5.m1.1.1.1">←</ci><csymbol cd="latexml" id="algorithm1.5.5.m1.1.1.2.cmml" xref="algorithm1.5.5.m1.1.1.2">absent</csymbol><list id="algorithm1.5.5.m1.1.1.3.1.cmml" xref="algorithm1.5.5.m1.1.1.3.2.1"></list></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.5.5.m1.1c">\leftarrow\left[\,\right]</annotation></semantics></math>;
</div>
<div id="algorithm1.6.6" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">2</span>
<span id="algorithm1.6.6.2" class="ltx_text ltx_font_bold">if</span> <em id="algorithm1.6.6.1" class="ltx_emph ltx_font_italic"><math id="algorithm1.6.6.1.m1.1" class="ltx_Math" alttext="\texttt{len(}\mathbf{c}\texttt{)}&gt;r" display="inline"><semantics id="algorithm1.6.6.1.m1.1a"><mrow id="algorithm1.6.6.1.m1.1.1" xref="algorithm1.6.6.1.m1.1.1.cmml"><mrow id="algorithm1.6.6.1.m1.1.1.2" xref="algorithm1.6.6.1.m1.1.1.2.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.6.6.1.m1.1.1.2.2" xref="algorithm1.6.6.1.m1.1.1.2.2b.cmml"><em id="algorithm1.6.6.1.m1.1.1.2.2.1nest" class="ltx_emph ltx_font_typewriter">len(</em></mtext><mo lspace="0em" rspace="0em" id="algorithm1.6.6.1.m1.1.1.2.1" xref="algorithm1.6.6.1.m1.1.1.2.1.cmml">​</mo><mi id="algorithm1.6.6.1.m1.1.1.2.3" xref="algorithm1.6.6.1.m1.1.1.2.3.cmml">𝐜</mi><mo lspace="0em" rspace="0em" id="algorithm1.6.6.1.m1.1.1.2.1a" xref="algorithm1.6.6.1.m1.1.1.2.1.cmml">​</mo><mtext class="ltx_mathvariant_monospace" id="algorithm1.6.6.1.m1.1.1.2.4" xref="algorithm1.6.6.1.m1.1.1.2.4b.cmml"><em id="algorithm1.6.6.1.m1.1.1.2.4.1nest" class="ltx_emph ltx_font_typewriter">)</em></mtext></mrow><mo id="algorithm1.6.6.1.m1.1.1.1" xref="algorithm1.6.6.1.m1.1.1.1.cmml">&gt;</mo><mi id="algorithm1.6.6.1.m1.1.1.3" xref="algorithm1.6.6.1.m1.1.1.3.cmml">r</mi></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.6.6.1.m1.1b"><apply id="algorithm1.6.6.1.m1.1.1.cmml" xref="algorithm1.6.6.1.m1.1.1"><gt id="algorithm1.6.6.1.m1.1.1.1.cmml" xref="algorithm1.6.6.1.m1.1.1.1"></gt><apply id="algorithm1.6.6.1.m1.1.1.2.cmml" xref="algorithm1.6.6.1.m1.1.1.2"><times id="algorithm1.6.6.1.m1.1.1.2.1.cmml" xref="algorithm1.6.6.1.m1.1.1.2.1"></times><ci id="algorithm1.6.6.1.m1.1.1.2.2b.cmml" xref="algorithm1.6.6.1.m1.1.1.2.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.6.6.1.m1.1.1.2.2.cmml" xref="algorithm1.6.6.1.m1.1.1.2.2"><em id="algorithm1.6.6.1.m1.1.1.2.2.1anest" class="ltx_emph ltx_font_typewriter">len(</em></mtext></ci><ci id="algorithm1.6.6.1.m1.1.1.2.3.cmml" xref="algorithm1.6.6.1.m1.1.1.2.3">𝐜</ci><ci id="algorithm1.6.6.1.m1.1.1.2.4b.cmml" xref="algorithm1.6.6.1.m1.1.1.2.4"><mtext class="ltx_mathvariant_monospace" id="algorithm1.6.6.1.m1.1.1.2.4.cmml" xref="algorithm1.6.6.1.m1.1.1.2.4"><em id="algorithm1.6.6.1.m1.1.1.2.4.1anest" class="ltx_emph ltx_font_typewriter">)</em></mtext></ci></apply><ci id="algorithm1.6.6.1.m1.1.1.3.cmml" xref="algorithm1.6.6.1.m1.1.1.3">𝑟</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.6.6.1.m1.1c">\texttt{len(}\mathbf{c}\texttt{)}&gt;r</annotation></semantics></math></em> <span id="algorithm1.6.6.3" class="ltx_text ltx_font_bold">then</span>
</div>
<div id="algorithm1.7.7" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">3</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   
<span id="algorithm1.7.7.1" class="ltx_text ltx_font_typewriter">prefix</span> <math id="algorithm1.7.7.m1.1" class="ltx_Math" alttext="\leftarrow\mathbf{q}+[{\color[rgb]{.5,.5,.5}\texttt{``[part\_schema]''}}]" display="inline"><semantics id="algorithm1.7.7.m1.1a"><mrow id="algorithm1.7.7.m1.1.2" xref="algorithm1.7.7.m1.1.2.cmml"><mi id="algorithm1.7.7.m1.1.2.2" xref="algorithm1.7.7.m1.1.2.2.cmml"></mi><mo stretchy="false" id="algorithm1.7.7.m1.1.2.1" xref="algorithm1.7.7.m1.1.2.1.cmml">←</mo><mrow id="algorithm1.7.7.m1.1.2.3" xref="algorithm1.7.7.m1.1.2.3.cmml"><mi id="algorithm1.7.7.m1.1.2.3.2" xref="algorithm1.7.7.m1.1.2.3.2.cmml">𝐪</mi><mo id="algorithm1.7.7.m1.1.2.3.1" xref="algorithm1.7.7.m1.1.2.3.1.cmml">+</mo><mrow id="algorithm1.7.7.m1.1.2.3.3.2" xref="algorithm1.7.7.m1.1.2.3.3.1.cmml"><mo stretchy="false" id="algorithm1.7.7.m1.1.2.3.3.2.1" xref="algorithm1.7.7.m1.1.2.3.3.1.1.cmml">[</mo><mtext class="ltx_mathvariant_monospace" mathcolor="#808080" id="algorithm1.7.7.m1.1.1" xref="algorithm1.7.7.m1.1.1a.cmml">‘‘[part_schema]’’</mtext><mo stretchy="false" id="algorithm1.7.7.m1.1.2.3.3.2.2" xref="algorithm1.7.7.m1.1.2.3.3.1.1.cmml">]</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.7.7.m1.1b"><apply id="algorithm1.7.7.m1.1.2.cmml" xref="algorithm1.7.7.m1.1.2"><ci id="algorithm1.7.7.m1.1.2.1.cmml" xref="algorithm1.7.7.m1.1.2.1">←</ci><csymbol cd="latexml" id="algorithm1.7.7.m1.1.2.2.cmml" xref="algorithm1.7.7.m1.1.2.2">absent</csymbol><apply id="algorithm1.7.7.m1.1.2.3.cmml" xref="algorithm1.7.7.m1.1.2.3"><plus id="algorithm1.7.7.m1.1.2.3.1.cmml" xref="algorithm1.7.7.m1.1.2.3.1"></plus><ci id="algorithm1.7.7.m1.1.2.3.2.cmml" xref="algorithm1.7.7.m1.1.2.3.2">𝐪</ci><apply id="algorithm1.7.7.m1.1.2.3.3.1.cmml" xref="algorithm1.7.7.m1.1.2.3.3.2"><csymbol cd="latexml" id="algorithm1.7.7.m1.1.2.3.3.1.1.cmml" xref="algorithm1.7.7.m1.1.2.3.3.2.1">delimited-[]</csymbol><ci id="algorithm1.7.7.m1.1.1a.cmml" xref="algorithm1.7.7.m1.1.1"><mtext class="ltx_mathvariant_monospace" mathcolor="#808080" id="algorithm1.7.7.m1.1.1.cmml" xref="algorithm1.7.7.m1.1.1">‘‘[part_schema]’’</mtext></ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.7.7.m1.1c">\leftarrow\mathbf{q}+[{\color[rgb]{.5,.5,.5}\texttt{``[part\_schema]''}}]</annotation></semantics></math>;
</div>
<div id="algorithm1.16.17" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">4</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   

</div>
<div id="algorithm1.16.18" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">5</span><span id="algorithm1.16.18.1" class="ltx_text ltx_font_bold">else</span> 
</div>
<div id="algorithm1.8.8" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">6</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   
<span id="algorithm1.8.8.1" class="ltx_text ltx_font_typewriter">prefix</span> <math id="algorithm1.8.8.m1.1" class="ltx_Math" alttext="\leftarrow\mathbf{q}+[{\color[rgb]{.5,.5,.5}\texttt{``[full\_schema]''}}]" display="inline"><semantics id="algorithm1.8.8.m1.1a"><mrow id="algorithm1.8.8.m1.1.2" xref="algorithm1.8.8.m1.1.2.cmml"><mi id="algorithm1.8.8.m1.1.2.2" xref="algorithm1.8.8.m1.1.2.2.cmml"></mi><mo stretchy="false" id="algorithm1.8.8.m1.1.2.1" xref="algorithm1.8.8.m1.1.2.1.cmml">←</mo><mrow id="algorithm1.8.8.m1.1.2.3" xref="algorithm1.8.8.m1.1.2.3.cmml"><mi id="algorithm1.8.8.m1.1.2.3.2" xref="algorithm1.8.8.m1.1.2.3.2.cmml">𝐪</mi><mo id="algorithm1.8.8.m1.1.2.3.1" xref="algorithm1.8.8.m1.1.2.3.1.cmml">+</mo><mrow id="algorithm1.8.8.m1.1.2.3.3.2" xref="algorithm1.8.8.m1.1.2.3.3.1.cmml"><mo stretchy="false" id="algorithm1.8.8.m1.1.2.3.3.2.1" xref="algorithm1.8.8.m1.1.2.3.3.1.1.cmml">[</mo><mtext class="ltx_mathvariant_monospace" mathcolor="#808080" id="algorithm1.8.8.m1.1.1" xref="algorithm1.8.8.m1.1.1a.cmml">‘‘[full_schema]’’</mtext><mo stretchy="false" id="algorithm1.8.8.m1.1.2.3.3.2.2" xref="algorithm1.8.8.m1.1.2.3.3.1.1.cmml">]</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.8.8.m1.1b"><apply id="algorithm1.8.8.m1.1.2.cmml" xref="algorithm1.8.8.m1.1.2"><ci id="algorithm1.8.8.m1.1.2.1.cmml" xref="algorithm1.8.8.m1.1.2.1">←</ci><csymbol cd="latexml" id="algorithm1.8.8.m1.1.2.2.cmml" xref="algorithm1.8.8.m1.1.2.2">absent</csymbol><apply id="algorithm1.8.8.m1.1.2.3.cmml" xref="algorithm1.8.8.m1.1.2.3"><plus id="algorithm1.8.8.m1.1.2.3.1.cmml" xref="algorithm1.8.8.m1.1.2.3.1"></plus><ci id="algorithm1.8.8.m1.1.2.3.2.cmml" xref="algorithm1.8.8.m1.1.2.3.2">𝐪</ci><apply id="algorithm1.8.8.m1.1.2.3.3.1.cmml" xref="algorithm1.8.8.m1.1.2.3.3.2"><csymbol cd="latexml" id="algorithm1.8.8.m1.1.2.3.3.1.1.cmml" xref="algorithm1.8.8.m1.1.2.3.3.2.1">delimited-[]</csymbol><ci id="algorithm1.8.8.m1.1.1a.cmml" xref="algorithm1.8.8.m1.1.1"><mtext class="ltx_mathvariant_monospace" mathcolor="#808080" id="algorithm1.8.8.m1.1.1.cmml" xref="algorithm1.8.8.m1.1.1">‘‘[full_schema]’’</mtext></ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.8.8.m1.1c">\leftarrow\mathbf{q}+[{\color[rgb]{.5,.5,.5}\texttt{``[full\_schema]''}}]</annotation></semantics></math>;
</div>
<div id="algorithm1.16.19" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">7</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   

</div>
<div id="algorithm1.16.20" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">8</span> end if
</div>
<div id="algorithm1.16.21" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">9</span>
</div>
<div id="algorithm1.9.9" class="ltx_listingline">
<math id="algorithm1.9.9.m1.1" class="ltx_Math" alttext="\texttt{sp}\leftarrow\texttt{prefix}" display="inline"><semantics id="algorithm1.9.9.m1.1a"><mrow id="algorithm1.9.9.m1.1.1" xref="algorithm1.9.9.m1.1.1.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.9.9.m1.1.1.2" xref="algorithm1.9.9.m1.1.1.2a.cmml">sp</mtext><mo stretchy="false" id="algorithm1.9.9.m1.1.1.1" xref="algorithm1.9.9.m1.1.1.1.cmml">←</mo><mtext class="ltx_mathvariant_monospace" id="algorithm1.9.9.m1.1.1.3" xref="algorithm1.9.9.m1.1.1.3a.cmml">prefix</mtext></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.9.9.m1.1b"><apply id="algorithm1.9.9.m1.1.1.cmml" xref="algorithm1.9.9.m1.1.1"><ci id="algorithm1.9.9.m1.1.1.1.cmml" xref="algorithm1.9.9.m1.1.1.1">←</ci><ci id="algorithm1.9.9.m1.1.1.2a.cmml" xref="algorithm1.9.9.m1.1.1.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.9.9.m1.1.1.2.cmml" xref="algorithm1.9.9.m1.1.1.2">sp</mtext></ci><ci id="algorithm1.9.9.m1.1.1.3a.cmml" xref="algorithm1.9.9.m1.1.1.3"><mtext class="ltx_mathvariant_monospace" id="algorithm1.9.9.m1.1.1.3.cmml" xref="algorithm1.9.9.m1.1.1.3">prefix</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.9.9.m1.1c">\texttt{sp}\leftarrow\texttt{prefix}</annotation></semantics></math>;
</div>
<div id="algorithm1.16.22" class="ltx_listingline"> <span id="algorithm1.16.22.1" class="ltx_text ltx_font_typewriter" style="color:#0000FF;"> # </span><span id="algorithm1.16.22.2" class="ltx_text ltx_font_typewriter" style="color:#0000FF;">one sp per split  </span>
</div>
<div id="algorithm1.11.11" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">10</span>
<span id="algorithm1.11.11.3" class="ltx_text ltx_font_bold">for</span> <em id="algorithm1.11.11.2" class="ltx_emph ltx_font_italic"><math id="algorithm1.10.10.1.m1.1" class="ltx_Math" alttext="j\leftarrow 1" display="inline"><semantics id="algorithm1.10.10.1.m1.1a"><mrow id="algorithm1.10.10.1.m1.1.1" xref="algorithm1.10.10.1.m1.1.1.cmml"><mi id="algorithm1.10.10.1.m1.1.1.2" xref="algorithm1.10.10.1.m1.1.1.2.cmml">j</mi><mo stretchy="false" id="algorithm1.10.10.1.m1.1.1.1" xref="algorithm1.10.10.1.m1.1.1.1.cmml">←</mo><mn id="algorithm1.10.10.1.m1.1.1.3" xref="algorithm1.10.10.1.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.10.10.1.m1.1b"><apply id="algorithm1.10.10.1.m1.1.1.cmml" xref="algorithm1.10.10.1.m1.1.1"><ci id="algorithm1.10.10.1.m1.1.1.1.cmml" xref="algorithm1.10.10.1.m1.1.1.1">←</ci><ci id="algorithm1.10.10.1.m1.1.1.2.cmml" xref="algorithm1.10.10.1.m1.1.1.2">𝑗</ci><cn type="integer" id="algorithm1.10.10.1.m1.1.1.3.cmml" xref="algorithm1.10.10.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.10.10.1.m1.1c">j\leftarrow 1</annotation></semantics></math> to <math id="algorithm1.11.11.2.m2.1" class="ltx_Math" alttext="\sum_{j=1}^{T}C_{j}" display="inline"><semantics id="algorithm1.11.11.2.m2.1a"><mrow id="algorithm1.11.11.2.m2.1.1" xref="algorithm1.11.11.2.m2.1.1.cmml"><msubsup id="algorithm1.11.11.2.m2.1.1.1" xref="algorithm1.11.11.2.m2.1.1.1.cmml"><mo id="algorithm1.11.11.2.m2.1.1.1.2.2" xref="algorithm1.11.11.2.m2.1.1.1.2.2.cmml">∑</mo><mrow id="algorithm1.11.11.2.m2.1.1.1.2.3" xref="algorithm1.11.11.2.m2.1.1.1.2.3.cmml"><mi id="algorithm1.11.11.2.m2.1.1.1.2.3.2" xref="algorithm1.11.11.2.m2.1.1.1.2.3.2.cmml">j</mi><mo id="algorithm1.11.11.2.m2.1.1.1.2.3.1" xref="algorithm1.11.11.2.m2.1.1.1.2.3.1.cmml">=</mo><mn id="algorithm1.11.11.2.m2.1.1.1.2.3.3" xref="algorithm1.11.11.2.m2.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="algorithm1.11.11.2.m2.1.1.1.3" xref="algorithm1.11.11.2.m2.1.1.1.3.cmml">T</mi></msubsup><msub id="algorithm1.11.11.2.m2.1.1.2" xref="algorithm1.11.11.2.m2.1.1.2.cmml"><mi id="algorithm1.11.11.2.m2.1.1.2.2" xref="algorithm1.11.11.2.m2.1.1.2.2.cmml">C</mi><mi id="algorithm1.11.11.2.m2.1.1.2.3" xref="algorithm1.11.11.2.m2.1.1.2.3.cmml">j</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.11.11.2.m2.1b"><apply id="algorithm1.11.11.2.m2.1.1.cmml" xref="algorithm1.11.11.2.m2.1.1"><apply id="algorithm1.11.11.2.m2.1.1.1.cmml" xref="algorithm1.11.11.2.m2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.11.11.2.m2.1.1.1.1.cmml" xref="algorithm1.11.11.2.m2.1.1.1">superscript</csymbol><apply id="algorithm1.11.11.2.m2.1.1.1.2.cmml" xref="algorithm1.11.11.2.m2.1.1.1"><csymbol cd="ambiguous" id="algorithm1.11.11.2.m2.1.1.1.2.1.cmml" xref="algorithm1.11.11.2.m2.1.1.1">subscript</csymbol><sum id="algorithm1.11.11.2.m2.1.1.1.2.2.cmml" xref="algorithm1.11.11.2.m2.1.1.1.2.2"></sum><apply id="algorithm1.11.11.2.m2.1.1.1.2.3.cmml" xref="algorithm1.11.11.2.m2.1.1.1.2.3"><eq id="algorithm1.11.11.2.m2.1.1.1.2.3.1.cmml" xref="algorithm1.11.11.2.m2.1.1.1.2.3.1"></eq><ci id="algorithm1.11.11.2.m2.1.1.1.2.3.2.cmml" xref="algorithm1.11.11.2.m2.1.1.1.2.3.2">𝑗</ci><cn type="integer" id="algorithm1.11.11.2.m2.1.1.1.2.3.3.cmml" xref="algorithm1.11.11.2.m2.1.1.1.2.3.3">1</cn></apply></apply><ci id="algorithm1.11.11.2.m2.1.1.1.3.cmml" xref="algorithm1.11.11.2.m2.1.1.1.3">𝑇</ci></apply><apply id="algorithm1.11.11.2.m2.1.1.2.cmml" xref="algorithm1.11.11.2.m2.1.1.2"><csymbol cd="ambiguous" id="algorithm1.11.11.2.m2.1.1.2.1.cmml" xref="algorithm1.11.11.2.m2.1.1.2">subscript</csymbol><ci id="algorithm1.11.11.2.m2.1.1.2.2.cmml" xref="algorithm1.11.11.2.m2.1.1.2.2">𝐶</ci><ci id="algorithm1.11.11.2.m2.1.1.2.3.cmml" xref="algorithm1.11.11.2.m2.1.1.2.3">𝑗</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.11.11.2.m2.1c">\sum_{j=1}^{T}C_{j}</annotation></semantics></math></em> <span id="algorithm1.11.11.4" class="ltx_text ltx_font_bold">do</span> 
</div>
<div id="algorithm1.12.12" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">11</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   
<math id="algorithm1.12.12.m1.1" class="ltx_Math" alttext="\texttt{sp}\leftarrow\texttt{sp}+\mathbf{c}^{\text{tok}}\left[j\right]" display="inline"><semantics id="algorithm1.12.12.m1.1a"><mrow id="algorithm1.12.12.m1.1.2" xref="algorithm1.12.12.m1.1.2.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.12.12.m1.1.2.2" xref="algorithm1.12.12.m1.1.2.2a.cmml">sp</mtext><mo stretchy="false" id="algorithm1.12.12.m1.1.2.1" xref="algorithm1.12.12.m1.1.2.1.cmml">←</mo><mrow id="algorithm1.12.12.m1.1.2.3" xref="algorithm1.12.12.m1.1.2.3.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.12.12.m1.1.2.3.2" xref="algorithm1.12.12.m1.1.2.3.2a.cmml">sp</mtext><mo id="algorithm1.12.12.m1.1.2.3.1" xref="algorithm1.12.12.m1.1.2.3.1.cmml">+</mo><mrow id="algorithm1.12.12.m1.1.2.3.3" xref="algorithm1.12.12.m1.1.2.3.3.cmml"><msup id="algorithm1.12.12.m1.1.2.3.3.2" xref="algorithm1.12.12.m1.1.2.3.3.2.cmml"><mi id="algorithm1.12.12.m1.1.2.3.3.2.2" xref="algorithm1.12.12.m1.1.2.3.3.2.2.cmml">𝐜</mi><mtext id="algorithm1.12.12.m1.1.2.3.3.2.3" xref="algorithm1.12.12.m1.1.2.3.3.2.3a.cmml">tok</mtext></msup><mo lspace="0em" rspace="0em" id="algorithm1.12.12.m1.1.2.3.3.1" xref="algorithm1.12.12.m1.1.2.3.3.1.cmml">​</mo><mrow id="algorithm1.12.12.m1.1.2.3.3.3.2" xref="algorithm1.12.12.m1.1.2.3.3.3.1.cmml"><mo id="algorithm1.12.12.m1.1.2.3.3.3.2.1" xref="algorithm1.12.12.m1.1.2.3.3.3.1.1.cmml">[</mo><mi id="algorithm1.12.12.m1.1.1" xref="algorithm1.12.12.m1.1.1.cmml">j</mi><mo id="algorithm1.12.12.m1.1.2.3.3.3.2.2" xref="algorithm1.12.12.m1.1.2.3.3.3.1.1.cmml">]</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.12.12.m1.1b"><apply id="algorithm1.12.12.m1.1.2.cmml" xref="algorithm1.12.12.m1.1.2"><ci id="algorithm1.12.12.m1.1.2.1.cmml" xref="algorithm1.12.12.m1.1.2.1">←</ci><ci id="algorithm1.12.12.m1.1.2.2a.cmml" xref="algorithm1.12.12.m1.1.2.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.12.12.m1.1.2.2.cmml" xref="algorithm1.12.12.m1.1.2.2">sp</mtext></ci><apply id="algorithm1.12.12.m1.1.2.3.cmml" xref="algorithm1.12.12.m1.1.2.3"><plus id="algorithm1.12.12.m1.1.2.3.1.cmml" xref="algorithm1.12.12.m1.1.2.3.1"></plus><ci id="algorithm1.12.12.m1.1.2.3.2a.cmml" xref="algorithm1.12.12.m1.1.2.3.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.12.12.m1.1.2.3.2.cmml" xref="algorithm1.12.12.m1.1.2.3.2">sp</mtext></ci><apply id="algorithm1.12.12.m1.1.2.3.3.cmml" xref="algorithm1.12.12.m1.1.2.3.3"><times id="algorithm1.12.12.m1.1.2.3.3.1.cmml" xref="algorithm1.12.12.m1.1.2.3.3.1"></times><apply id="algorithm1.12.12.m1.1.2.3.3.2.cmml" xref="algorithm1.12.12.m1.1.2.3.3.2"><csymbol cd="ambiguous" id="algorithm1.12.12.m1.1.2.3.3.2.1.cmml" xref="algorithm1.12.12.m1.1.2.3.3.2">superscript</csymbol><ci id="algorithm1.12.12.m1.1.2.3.3.2.2.cmml" xref="algorithm1.12.12.m1.1.2.3.3.2.2">𝐜</ci><ci id="algorithm1.12.12.m1.1.2.3.3.2.3a.cmml" xref="algorithm1.12.12.m1.1.2.3.3.2.3"><mtext mathsize="70%" id="algorithm1.12.12.m1.1.2.3.3.2.3.cmml" xref="algorithm1.12.12.m1.1.2.3.3.2.3">tok</mtext></ci></apply><apply id="algorithm1.12.12.m1.1.2.3.3.3.1.cmml" xref="algorithm1.12.12.m1.1.2.3.3.3.2"><csymbol cd="latexml" id="algorithm1.12.12.m1.1.2.3.3.3.1.1.cmml" xref="algorithm1.12.12.m1.1.2.3.3.3.2.1">delimited-[]</csymbol><ci id="algorithm1.12.12.m1.1.1.cmml" xref="algorithm1.12.12.m1.1.1">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.12.12.m1.1c">\texttt{sp}\leftarrow\texttt{sp}+\mathbf{c}^{\text{tok}}\left[j\right]</annotation></semantics></math>;
</div>
<div id="algorithm1.16.23" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">12</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   

</div>
<div id="algorithm1.13.13" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">13</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   <span id="algorithm1.13.13.2" class="ltx_text ltx_font_bold">if</span> <em id="algorithm1.13.13.1" class="ltx_emph ltx_font_italic"><math id="algorithm1.13.13.1.m1.1" class="ltx_Math" alttext="j\mod r=0\textnormal{{ or }}j=\sum_{j=1}^{T}C_{j}" display="inline"><semantics id="algorithm1.13.13.1.m1.1a"><mrow id="algorithm1.13.13.1.m1.1.1" xref="algorithm1.13.13.1.m1.1.1.cmml"><mrow id="algorithm1.13.13.1.m1.1.1.2" xref="algorithm1.13.13.1.m1.1.1.2.cmml"><mi id="algorithm1.13.13.1.m1.1.1.2.2" xref="algorithm1.13.13.1.m1.1.1.2.2.cmml">j</mi><mo id="algorithm1.13.13.1.m1.1.1.2.1" xref="algorithm1.13.13.1.m1.1.1.2.1.cmml">mod</mo><mi id="algorithm1.13.13.1.m1.1.1.2.3" xref="algorithm1.13.13.1.m1.1.1.2.3.cmml">r</mi></mrow><mo id="algorithm1.13.13.1.m1.1.1.3" xref="algorithm1.13.13.1.m1.1.1.3.cmml">=</mo><mrow id="algorithm1.13.13.1.m1.1.1.4" xref="algorithm1.13.13.1.m1.1.1.4.cmml"><mn id="algorithm1.13.13.1.m1.1.1.4.2" xref="algorithm1.13.13.1.m1.1.1.4.2.cmml">0</mn><mo lspace="0em" rspace="0em" id="algorithm1.13.13.1.m1.1.1.4.1" xref="algorithm1.13.13.1.m1.1.1.4.1.cmml">​</mo><mtext class="ltx_mathvariant_monospace" id="algorithm1.13.13.1.m1.1.1.4.3" xref="algorithm1.13.13.1.m1.1.1.4.3b.cmml"><em id="algorithm1.13.13.1.m1.1.1.4.3.1nest" class="ltx_emph ltx_font_typewriter ltx_font_bold ltx_font_upright"> or </em></mtext><mo lspace="0em" rspace="0em" id="algorithm1.13.13.1.m1.1.1.4.1a" xref="algorithm1.13.13.1.m1.1.1.4.1.cmml">​</mo><mi id="algorithm1.13.13.1.m1.1.1.4.4" xref="algorithm1.13.13.1.m1.1.1.4.4.cmml">j</mi></mrow><mo rspace="0.111em" id="algorithm1.13.13.1.m1.1.1.5" xref="algorithm1.13.13.1.m1.1.1.5.cmml">=</mo><mrow id="algorithm1.13.13.1.m1.1.1.6" xref="algorithm1.13.13.1.m1.1.1.6.cmml"><msubsup id="algorithm1.13.13.1.m1.1.1.6.1" xref="algorithm1.13.13.1.m1.1.1.6.1.cmml"><mo id="algorithm1.13.13.1.m1.1.1.6.1.2.2" xref="algorithm1.13.13.1.m1.1.1.6.1.2.2.cmml">∑</mo><mrow id="algorithm1.13.13.1.m1.1.1.6.1.2.3" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.cmml"><mi id="algorithm1.13.13.1.m1.1.1.6.1.2.3.2" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.2.cmml">j</mi><mo id="algorithm1.13.13.1.m1.1.1.6.1.2.3.1" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.1.cmml">=</mo><mn id="algorithm1.13.13.1.m1.1.1.6.1.2.3.3" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.3.cmml">1</mn></mrow><mi id="algorithm1.13.13.1.m1.1.1.6.1.3" xref="algorithm1.13.13.1.m1.1.1.6.1.3.cmml">T</mi></msubsup><msub id="algorithm1.13.13.1.m1.1.1.6.2" xref="algorithm1.13.13.1.m1.1.1.6.2.cmml"><mi id="algorithm1.13.13.1.m1.1.1.6.2.2" xref="algorithm1.13.13.1.m1.1.1.6.2.2.cmml">C</mi><mi id="algorithm1.13.13.1.m1.1.1.6.2.3" xref="algorithm1.13.13.1.m1.1.1.6.2.3.cmml">j</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.13.13.1.m1.1b"><apply id="algorithm1.13.13.1.m1.1.1.cmml" xref="algorithm1.13.13.1.m1.1.1"><and id="algorithm1.13.13.1.m1.1.1a.cmml" xref="algorithm1.13.13.1.m1.1.1"></and><apply id="algorithm1.13.13.1.m1.1.1b.cmml" xref="algorithm1.13.13.1.m1.1.1"><eq id="algorithm1.13.13.1.m1.1.1.3.cmml" xref="algorithm1.13.13.1.m1.1.1.3"></eq><apply id="algorithm1.13.13.1.m1.1.1.2.cmml" xref="algorithm1.13.13.1.m1.1.1.2"><csymbol cd="latexml" id="algorithm1.13.13.1.m1.1.1.2.1.cmml" xref="algorithm1.13.13.1.m1.1.1.2.1">modulo</csymbol><ci id="algorithm1.13.13.1.m1.1.1.2.2.cmml" xref="algorithm1.13.13.1.m1.1.1.2.2">𝑗</ci><ci id="algorithm1.13.13.1.m1.1.1.2.3.cmml" xref="algorithm1.13.13.1.m1.1.1.2.3">𝑟</ci></apply><apply id="algorithm1.13.13.1.m1.1.1.4.cmml" xref="algorithm1.13.13.1.m1.1.1.4"><times id="algorithm1.13.13.1.m1.1.1.4.1.cmml" xref="algorithm1.13.13.1.m1.1.1.4.1"></times><cn type="integer" id="algorithm1.13.13.1.m1.1.1.4.2.cmml" xref="algorithm1.13.13.1.m1.1.1.4.2">0</cn><ci id="algorithm1.13.13.1.m1.1.1.4.3b.cmml" xref="algorithm1.13.13.1.m1.1.1.4.3"><mtext class="ltx_mathvariant_monospace" id="algorithm1.13.13.1.m1.1.1.4.3.cmml" xref="algorithm1.13.13.1.m1.1.1.4.3"><em id="algorithm1.13.13.1.m1.1.1.4.3.1anest" class="ltx_emph ltx_font_typewriter ltx_font_bold ltx_font_upright"> or </em></mtext></ci><ci id="algorithm1.13.13.1.m1.1.1.4.4.cmml" xref="algorithm1.13.13.1.m1.1.1.4.4">𝑗</ci></apply></apply><apply id="algorithm1.13.13.1.m1.1.1c.cmml" xref="algorithm1.13.13.1.m1.1.1"><eq id="algorithm1.13.13.1.m1.1.1.5.cmml" xref="algorithm1.13.13.1.m1.1.1.5"></eq><share href="#algorithm1.13.13.1.m1.1.1.4.cmml" id="algorithm1.13.13.1.m1.1.1d.cmml" xref="algorithm1.13.13.1.m1.1.1"></share><apply id="algorithm1.13.13.1.m1.1.1.6.cmml" xref="algorithm1.13.13.1.m1.1.1.6"><apply id="algorithm1.13.13.1.m1.1.1.6.1.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1"><csymbol cd="ambiguous" id="algorithm1.13.13.1.m1.1.1.6.1.1.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1">superscript</csymbol><apply id="algorithm1.13.13.1.m1.1.1.6.1.2.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1"><csymbol cd="ambiguous" id="algorithm1.13.13.1.m1.1.1.6.1.2.1.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1">subscript</csymbol><sum id="algorithm1.13.13.1.m1.1.1.6.1.2.2.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1.2.2"></sum><apply id="algorithm1.13.13.1.m1.1.1.6.1.2.3.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3"><eq id="algorithm1.13.13.1.m1.1.1.6.1.2.3.1.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.1"></eq><ci id="algorithm1.13.13.1.m1.1.1.6.1.2.3.2.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.2">𝑗</ci><cn type="integer" id="algorithm1.13.13.1.m1.1.1.6.1.2.3.3.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1.2.3.3">1</cn></apply></apply><ci id="algorithm1.13.13.1.m1.1.1.6.1.3.cmml" xref="algorithm1.13.13.1.m1.1.1.6.1.3">𝑇</ci></apply><apply id="algorithm1.13.13.1.m1.1.1.6.2.cmml" xref="algorithm1.13.13.1.m1.1.1.6.2"><csymbol cd="ambiguous" id="algorithm1.13.13.1.m1.1.1.6.2.1.cmml" xref="algorithm1.13.13.1.m1.1.1.6.2">subscript</csymbol><ci id="algorithm1.13.13.1.m1.1.1.6.2.2.cmml" xref="algorithm1.13.13.1.m1.1.1.6.2.2">𝐶</ci><ci id="algorithm1.13.13.1.m1.1.1.6.2.3.cmml" xref="algorithm1.13.13.1.m1.1.1.6.2.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.13.13.1.m1.1c">j\mod r=0\textnormal{{ or }}j=\sum_{j=1}^{T}C_{j}</annotation></semantics></math></em> <span id="algorithm1.13.13.3" class="ltx_text ltx_font_bold">then</span> 
</div>
<div id="algorithm1.14.14" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">14</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>     <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   
<math id="algorithm1.14.14.m1.1" class="ltx_Math" alttext="\texttt{sp}\leftarrow\texttt{sp}+\mathbf{t}^{\text{tok}}" display="inline"><semantics id="algorithm1.14.14.m1.1a"><mrow id="algorithm1.14.14.m1.1.1" xref="algorithm1.14.14.m1.1.1.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.14.14.m1.1.1.2" xref="algorithm1.14.14.m1.1.1.2a.cmml">sp</mtext><mo stretchy="false" id="algorithm1.14.14.m1.1.1.1" xref="algorithm1.14.14.m1.1.1.1.cmml">←</mo><mrow id="algorithm1.14.14.m1.1.1.3" xref="algorithm1.14.14.m1.1.1.3.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.14.14.m1.1.1.3.2" xref="algorithm1.14.14.m1.1.1.3.2a.cmml">sp</mtext><mo id="algorithm1.14.14.m1.1.1.3.1" xref="algorithm1.14.14.m1.1.1.3.1.cmml">+</mo><msup id="algorithm1.14.14.m1.1.1.3.3" xref="algorithm1.14.14.m1.1.1.3.3.cmml"><mi id="algorithm1.14.14.m1.1.1.3.3.2" xref="algorithm1.14.14.m1.1.1.3.3.2.cmml">𝐭</mi><mtext id="algorithm1.14.14.m1.1.1.3.3.3" xref="algorithm1.14.14.m1.1.1.3.3.3a.cmml">tok</mtext></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.14.14.m1.1b"><apply id="algorithm1.14.14.m1.1.1.cmml" xref="algorithm1.14.14.m1.1.1"><ci id="algorithm1.14.14.m1.1.1.1.cmml" xref="algorithm1.14.14.m1.1.1.1">←</ci><ci id="algorithm1.14.14.m1.1.1.2a.cmml" xref="algorithm1.14.14.m1.1.1.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.14.14.m1.1.1.2.cmml" xref="algorithm1.14.14.m1.1.1.2">sp</mtext></ci><apply id="algorithm1.14.14.m1.1.1.3.cmml" xref="algorithm1.14.14.m1.1.1.3"><plus id="algorithm1.14.14.m1.1.1.3.1.cmml" xref="algorithm1.14.14.m1.1.1.3.1"></plus><ci id="algorithm1.14.14.m1.1.1.3.2a.cmml" xref="algorithm1.14.14.m1.1.1.3.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.14.14.m1.1.1.3.2.cmml" xref="algorithm1.14.14.m1.1.1.3.2">sp</mtext></ci><apply id="algorithm1.14.14.m1.1.1.3.3.cmml" xref="algorithm1.14.14.m1.1.1.3.3"><csymbol cd="ambiguous" id="algorithm1.14.14.m1.1.1.3.3.1.cmml" xref="algorithm1.14.14.m1.1.1.3.3">superscript</csymbol><ci id="algorithm1.14.14.m1.1.1.3.3.2.cmml" xref="algorithm1.14.14.m1.1.1.3.3.2">𝐭</ci><ci id="algorithm1.14.14.m1.1.1.3.3.3a.cmml" xref="algorithm1.14.14.m1.1.1.3.3.3"><mtext mathsize="70%" id="algorithm1.14.14.m1.1.1.3.3.3.cmml" xref="algorithm1.14.14.m1.1.1.3.3.3">tok</mtext></ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.14.14.m1.1c">\texttt{sp}\leftarrow\texttt{sp}+\mathbf{t}^{\text{tok}}</annotation></semantics></math>;
</div>
<div id="algorithm1.15.15" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">15</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>     <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   
<span id="algorithm1.15.15.1" class="ltx_text ltx_markedasmath ltx_font_typewriter">splits.append(sp)</span>;
</div>
<div id="algorithm1.16.24" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">16</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>     <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   

</div>
<div id="algorithm1.16.16" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">17</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>     <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   <math id="algorithm1.16.16.m1.1" class="ltx_Math" alttext="\texttt{sp}\leftarrow\texttt{prefix}" display="inline"><semantics id="algorithm1.16.16.m1.1a"><mrow id="algorithm1.16.16.m1.1.1" xref="algorithm1.16.16.m1.1.1.cmml"><mtext class="ltx_mathvariant_monospace" id="algorithm1.16.16.m1.1.1.2" xref="algorithm1.16.16.m1.1.1.2a.cmml">sp</mtext><mo stretchy="false" id="algorithm1.16.16.m1.1.1.1" xref="algorithm1.16.16.m1.1.1.1.cmml">←</mo><mtext class="ltx_mathvariant_monospace" id="algorithm1.16.16.m1.1.1.3" xref="algorithm1.16.16.m1.1.1.3a.cmml">prefix</mtext></mrow><annotation-xml encoding="MathML-Content" id="algorithm1.16.16.m1.1b"><apply id="algorithm1.16.16.m1.1.1.cmml" xref="algorithm1.16.16.m1.1.1"><ci id="algorithm1.16.16.m1.1.1.1.cmml" xref="algorithm1.16.16.m1.1.1.1">←</ci><ci id="algorithm1.16.16.m1.1.1.2a.cmml" xref="algorithm1.16.16.m1.1.1.2"><mtext class="ltx_mathvariant_monospace" id="algorithm1.16.16.m1.1.1.2.cmml" xref="algorithm1.16.16.m1.1.1.2">sp</mtext></ci><ci id="algorithm1.16.16.m1.1.1.3a.cmml" xref="algorithm1.16.16.m1.1.1.3"><mtext class="ltx_mathvariant_monospace" id="algorithm1.16.16.m1.1.1.3.cmml" xref="algorithm1.16.16.m1.1.1.3">prefix</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="algorithm1.16.16.m1.1c">\texttt{sp}\leftarrow\texttt{prefix}</annotation></semantics></math>;
</div>
<div id="algorithm1.16.25" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">18</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>     <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   

</div>
<div id="algorithm1.16.26" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">19</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>    end if
</div>
<div id="algorithm1.16.27" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">20</span>  <span class="ltx_rule" style="width:1px;height:100%;background:black;display:inline-block;"> </span>   
</div>
<div id="algorithm1.16.28" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">21</span> end for
</div>
<div id="algorithm1.16.29" class="ltx_listingline">
<span id="algorithm1.16.29.1" class="ltx_text ltx_font_bold">Return</span> <span id="algorithm1.16.29.2" class="ltx_text ltx_font_typewriter">splits</span>

</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_float"><span id="algorithm1.18.1.1" class="ltx_text ltx_font_bold">Algorithm 1</span> </span>Algorithm for splitting a schema into smaller splits.</figcaption>
</figure>
<div id="S4.SS1.SSS2.p4" class="ltx_para">
<p id="S4.SS1.SSS2.p4.2" class="ltx_p">The returned schema splits along with the SSP labels corresponding to the schema elements of each split are treated as independent instances during training. At test time, an input schema is split according to Algorithm <a href="#algorithm1" title="In 4.1.2 Approximating for Longer Schemata ‣ 4.1 Schema Selection ‣ 4 Database Context Selection ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, and the model is input with a batch of the resulting splits. After aggregating the results from all splits, we obtain the SSP label for each column <math id="S4.SS1.SSS2.p4.1.m1.1" class="ltx_Math" alttext="\in\mathbf{c}" display="inline"><semantics id="S4.SS1.SSS2.p4.1.m1.1a"><mrow id="S4.SS1.SSS2.p4.1.m1.1.1" xref="S4.SS1.SSS2.p4.1.m1.1.1.cmml"><mi id="S4.SS1.SSS2.p4.1.m1.1.1.2" xref="S4.SS1.SSS2.p4.1.m1.1.1.2.cmml"></mi><mo id="S4.SS1.SSS2.p4.1.m1.1.1.1" xref="S4.SS1.SSS2.p4.1.m1.1.1.1.cmml">∈</mo><mi id="S4.SS1.SSS2.p4.1.m1.1.1.3" xref="S4.SS1.SSS2.p4.1.m1.1.1.3.cmml">𝐜</mi></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p4.1.m1.1b"><apply id="S4.SS1.SSS2.p4.1.m1.1.1.cmml" xref="S4.SS1.SSS2.p4.1.m1.1.1"><in id="S4.SS1.SSS2.p4.1.m1.1.1.1.cmml" xref="S4.SS1.SSS2.p4.1.m1.1.1.1"></in><csymbol cd="latexml" id="S4.SS1.SSS2.p4.1.m1.1.1.2.cmml" xref="S4.SS1.SSS2.p4.1.m1.1.1.2">absent</csymbol><ci id="S4.SS1.SSS2.p4.1.m1.1.1.3.cmml" xref="S4.SS1.SSS2.p4.1.m1.1.1.3">𝐜</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p4.1.m1.1c">\in\mathbf{c}</annotation></semantics></math>. Inconsistencies across the SSP labels of tables are resolved using majority voting. We refer to this model as <math id="S4.SS1.SSS2.p4.2.m2.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S4.SS1.SSS2.p4.2.m2.1a"><msub id="S4.SS1.SSS2.p4.2.m2.1.1" xref="S4.SS1.SSS2.p4.2.m2.1.1.cmml"><mtext id="S4.SS1.SSS2.p4.2.m2.1.1.2" xref="S4.SS1.SSS2.p4.2.m2.1.1.2a.cmml">FastRAT</mtext><mtext id="S4.SS1.SSS2.p4.2.m2.1.1.3" xref="S4.SS1.SSS2.p4.2.m2.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.SSS2.p4.2.m2.1b"><apply id="S4.SS1.SSS2.p4.2.m2.1.1.cmml" xref="S4.SS1.SSS2.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.SSS2.p4.2.m2.1.1.1.cmml" xref="S4.SS1.SSS2.p4.2.m2.1.1">subscript</csymbol><ci id="S4.SS1.SSS2.p4.2.m2.1.1.2a.cmml" xref="S4.SS1.SSS2.p4.2.m2.1.1.2"><mtext id="S4.SS1.SSS2.p4.2.m2.1.1.2.cmml" xref="S4.SS1.SSS2.p4.2.m2.1.1.2">FastRAT</mtext></ci><ci id="S4.SS1.SSS2.p4.2.m2.1.1.3a.cmml" xref="S4.SS1.SSS2.p4.2.m2.1.1.3"><mtext mathsize="70%" id="S4.SS1.SSS2.p4.2.m2.1.1.3.cmml" xref="S4.SS1.SSS2.p4.2.m2.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.SSS2.p4.2.m2.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>.</p>
</div>
</section>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Value Selection</h3>

<div id="S4.SS2.p1" class="ltx_para">
<p id="S4.SS2.p1.1" class="ltx_p">The inference of LLMs for text-to-SQL parsing can be augmented with column values <cite class="ltx_cite ltx_citemacro_cite">Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>. We select values for columns in a schema (or a sub-schema) by simply matching keywords in questions and values. This is based on the assumption that LLMs can generalise to unseen values given a set of representative values; thus, the recall and precision of value selection are less critical. We consider value selection providing additional information for LLMs to discern covert differences among columns. An example of our resulting prompt is shown in Appendix <a href="#A2" title="Appendix B Prompt Formulation ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>.</p>
</div>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Experiments</h2>

<div id="S5.p1" class="ltx_para">
<p id="S5.p1.3" class="ltx_p">We run experiments using two approximators: <math id="S5.p1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.p1.1.m1.1a"><msub id="S5.p1.1.m1.1.1" xref="S5.p1.1.m1.1.1.cmml"><mtext id="S5.p1.1.m1.1.1.2" xref="S5.p1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.p1.1.m1.1.1.3" xref="S5.p1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.p1.1.m1.1b"><apply id="S5.p1.1.m1.1.1.cmml" xref="S5.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.p1.1.m1.1.1.1.cmml" xref="S5.p1.1.m1.1.1">subscript</csymbol><ci id="S5.p1.1.m1.1.1.2a.cmml" xref="S5.p1.1.m1.1.1.2"><mtext id="S5.p1.1.m1.1.1.2.cmml" xref="S5.p1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.p1.1.m1.1.1.3a.cmml" xref="S5.p1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.p1.1.m1.1.1.3.cmml" xref="S5.p1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> and Graphix-T5 <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib13" title="" class="ltx_ref">2023b</a>)</cite>.
Graphix-T5 is is the approximator used by DAIL-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>, and is included to facilitate a fair comparison against the closest work to ours. <math id="S5.p1.2.m2.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.p1.2.m2.1a"><msub id="S5.p1.2.m2.1.1" xref="S5.p1.2.m2.1.1.cmml"><mtext id="S5.p1.2.m2.1.1.2" xref="S5.p1.2.m2.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.p1.2.m2.1.1.3" xref="S5.p1.2.m2.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.p1.2.m2.1b"><apply id="S5.p1.2.m2.1.1.cmml" xref="S5.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S5.p1.2.m2.1.1.1.cmml" xref="S5.p1.2.m2.1.1">subscript</csymbol><ci id="S5.p1.2.m2.1.1.2a.cmml" xref="S5.p1.2.m2.1.1.2"><mtext id="S5.p1.2.m2.1.1.2.cmml" xref="S5.p1.2.m2.1.1.2">FastRAT</mtext></ci><ci id="S5.p1.2.m2.1.1.3a.cmml" xref="S5.p1.2.m2.1.1.3"><mtext mathsize="70%" id="S5.p1.2.m2.1.1.3.cmml" xref="S5.p1.2.m2.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.2.m2.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> is trained and tested using <math id="S5.p1.3.m3.1" class="ltx_Math" alttext="r=64" display="inline"><semantics id="S5.p1.3.m3.1a"><mrow id="S5.p1.3.m3.1.1" xref="S5.p1.3.m3.1.1.cmml"><mi id="S5.p1.3.m3.1.1.2" xref="S5.p1.3.m3.1.1.2.cmml">r</mi><mo id="S5.p1.3.m3.1.1.1" xref="S5.p1.3.m3.1.1.1.cmml">=</mo><mn id="S5.p1.3.m3.1.1.3" xref="S5.p1.3.m3.1.1.3.cmml">64</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.p1.3.m3.1b"><apply id="S5.p1.3.m3.1.1.cmml" xref="S5.p1.3.m3.1.1"><eq id="S5.p1.3.m3.1.1.1.cmml" xref="S5.p1.3.m3.1.1.1"></eq><ci id="S5.p1.3.m3.1.1.2.cmml" xref="S5.p1.3.m3.1.1.2">𝑟</ci><cn type="integer" id="S5.p1.3.m3.1.1.3.cmml" xref="S5.p1.3.m3.1.1.3">64</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.3.m3.1c">r=64</annotation></semantics></math>, unless otherwise stated (cf. Section <a href="#S5.SS4" title="5.4 Schema Splitting ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5.4</span></a>). For all experiments, we use 5 (question, SQL) examples.</p>
</div>
<div id="S5.p2" class="ltx_para">
<p id="S5.p2.1" class="ltx_p">We test our approach against both closed- and open-source LLMs: 
<span id="S5.I1" class="ltx_inline-enumerate">
<span id="S5.I1.i1" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(i)</span> <span id="S5.I1.i1.1" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span><span id="S5.I1.i1.2" class="ltx_text">(<span id="S5.I1.i1.2.1" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo-0613</span>), </span></span>
<span id="S5.I1.i2" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(ii)</span> <span id="S5.I1.i2.1" class="ltx_text ltx_font_typewriter">gpt-4</span><span id="S5.I1.i2.2" class="ltx_text">(<span id="S5.I1.i2.2.1" class="ltx_text ltx_font_typewriter">gpt-4-0613</span>) and </span></span>
<span id="S5.I1.i3" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(iii)</span> <span id="S5.I1.i3.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span><span id="S5.I1.i3.2" class="ltx_text">. </span></span>
</span> Results using additional models from the DeepSeek family are provided in Appendix <a href="#A3.SS4" title="C.4 Experiments with Open-Source LLMs ‣ Appendix C Implementation Details ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C.4</span></a>.</p>
</div>
<section id="S5.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.1 </span>Datasets</h3>

<div id="S5.SS1.p1" class="ltx_para">
<p id="S5.SS1.p1.2" class="ltx_p">We experiment with several SQL datasets, seeking to explore the effectiveness of our approach on both monolingual and cross-lingual setups. Specifically, we report experiments on CS<span id="S5.SS1.p1.2.1" class="ltx_text ltx_font_smallcaps">pider</span> <cite class="ltx_cite ltx_citemacro_cite">Min et al. (<a href="#bib.bib15" title="" class="ltx_ref">2019</a>)</cite> and S<span id="S5.SS1.p1.2.2" class="ltx_text ltx_font_smallcaps">pider</span> <cite class="ltx_cite ltx_citemacro_cite">Yu et al. (<a href="#bib.bib32" title="" class="ltx_ref">2018</a>)</cite>.
Since CS<span id="S5.SS1.p1.2.3" class="ltx_text ltx_font_smallcaps">pider</span> is a translated version of S<span id="S5.SS1.p1.2.4" class="ltx_text ltx_font_smallcaps">pider</span> in Chinese, when it comes to the natural language questions, the characteristics of the two with respect to structure and number of examples are identical.
We focus our evaluation on the development sets<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>Appendix <a href="#A4" title="Appendix D Spider and CSpider Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">D</span></a> includes results on the test sets.</span></span></span>, which are used as test sets in our experiments. These splits consists of <math id="S5.SS1.p1.1.m1.2" class="ltx_Math" alttext="1,034" display="inline"><semantics id="S5.SS1.p1.1.m1.2a"><mrow id="S5.SS1.p1.1.m1.2.3.2" xref="S5.SS1.p1.1.m1.2.3.1.cmml"><mn id="S5.SS1.p1.1.m1.1.1" xref="S5.SS1.p1.1.m1.1.1.cmml">1</mn><mo id="S5.SS1.p1.1.m1.2.3.2.1" xref="S5.SS1.p1.1.m1.2.3.1.cmml">,</mo><mn id="S5.SS1.p1.1.m1.2.2" xref="S5.SS1.p1.1.m1.2.2.cmml">034</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS1.p1.1.m1.2b"><list id="S5.SS1.p1.1.m1.2.3.1.cmml" xref="S5.SS1.p1.1.m1.2.3.2"><cn type="integer" id="S5.SS1.p1.1.m1.1.1.cmml" xref="S5.SS1.p1.1.m1.1.1">1</cn><cn type="integer" id="S5.SS1.p1.1.m1.2.2.cmml" xref="S5.SS1.p1.1.m1.2.2">034</cn></list></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p1.1.m1.2c">1,034</annotation></semantics></math> examples of questions on <math id="S5.SS1.p1.2.m2.1" class="ltx_Math" alttext="20" display="inline"><semantics id="S5.SS1.p1.2.m2.1a"><mn id="S5.SS1.p1.2.m2.1.1" xref="S5.SS1.p1.2.m2.1.1.cmml">20</mn><annotation-xml encoding="MathML-Content" id="S5.SS1.p1.2.m2.1b"><cn type="integer" id="S5.SS1.p1.2.m2.1.1.cmml" xref="S5.SS1.p1.2.m2.1.1">20</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p1.2.m2.1c">20</annotation></semantics></math> unique databases that are not met at training time.</p>
</div>
<div id="S5.SS1.p2" class="ltx_para">
<p id="S5.SS1.p2.1" class="ltx_p">We rely on the training splits to maintain an index of (question, SQL) examples, one for each dataset. Using these splits, we train a monolingual and a cross-lingual version of <math id="S5.SS1.p2.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.SS1.p2.1.m1.1a"><msub id="S5.SS1.p2.1.m1.1.1" xref="S5.SS1.p2.1.m1.1.1.cmml"><mtext id="S5.SS1.p2.1.m1.1.1.2" xref="S5.SS1.p2.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.SS1.p2.1.m1.1.1.3" xref="S5.SS1.p2.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.1.m1.1b"><apply id="S5.SS1.p2.1.m1.1.1.cmml" xref="S5.SS1.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS1.p2.1.m1.1.1.1.cmml" xref="S5.SS1.p2.1.m1.1.1">subscript</csymbol><ci id="S5.SS1.p2.1.m1.1.1.2a.cmml" xref="S5.SS1.p2.1.m1.1.1.2"><mtext id="S5.SS1.p2.1.m1.1.1.2.cmml" xref="S5.SS1.p2.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.SS1.p2.1.m1.1.1.3a.cmml" xref="S5.SS1.p2.1.m1.1.1.3"><mtext mathsize="70%" id="S5.SS1.p2.1.m1.1.1.3.cmml" xref="S5.SS1.p2.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>.</p>
</div>
<div id="S5.SS1.p3" class="ltx_para">
<p id="S5.SS1.p3.1" class="ltx_p">Furthermore, we use popular S<span id="S5.SS1.p3.1.1" class="ltx_text ltx_font_smallcaps">pider</span> variants: 
<span id="S5.I2" class="ltx_inline-enumerate">
<span id="S5.I2.i1" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(i)</span> <span id="S5.I2.i1.1" class="ltx_text">S<span id="S5.I2.i1.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-DK <cite class="ltx_cite ltx_citemacro_cite">Gan et al. (<a href="#bib.bib9" title="" class="ltx_ref">2021b</a>)</cite>, </span></span>
<span id="S5.I2.i2" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(ii)</span> <span id="S5.I2.i2.1" class="ltx_text">S<span id="S5.I2.i2.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="S5.I2.i2.1.2" class="ltx_text ltx_font_smallcaps">eal</span> <cite class="ltx_cite ltx_citemacro_cite">Deng et al. (<a href="#bib.bib5" title="" class="ltx_ref">2021</a>)</cite> and </span></span>
<span id="S5.I2.i3" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(iii)</span> <span id="S5.I2.i3.1" class="ltx_text">S<span id="S5.I2.i3.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="S5.I2.i3.1.2" class="ltx_text ltx_font_smallcaps">yn</span> <cite class="ltx_cite ltx_citemacro_cite">Gan et al. (<a href="#bib.bib8" title="" class="ltx_ref">2021a</a>)</cite> </span></span>
</span> to evaluate zero-shot domain generalisation in English (leveraging the S<span id="S5.SS1.p3.1.2" class="ltx_text ltx_font_smallcaps">pider</span> (question, SQL) examples index).</p>
</div>
<div id="S5.SS1.p4" class="ltx_para">
<p id="S5.SS1.p4.1" class="ltx_p">Consistently with the relevant leaderboards<span id="footnote3" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">3</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">3</sup><span class="ltx_tag ltx_tag_note">3</span><a target="_blank" href="https://taolusi.github.io/CSpider-explorer/" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://taolusi.github.io/CSpider-explorer/</a> and <a target="_blank" href="https://yale-lily.github.io/spider" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://yale-lily.github.io/spider</a></span></span></span>, we report results using execution (EX) and exact match (EM) accuracy.<span id="footnote4" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">4</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">4</sup><span class="ltx_tag ltx_tag_note">4</span>EX and EM scores are computed using: <a target="_blank" href="https://github.com/taoyds/test-suite-sql-eval" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/taoyds/test-suite-sql-eval</a>.</span></span></span> Since CS<span id="S5.SS1.p4.1.1" class="ltx_text ltx_font_smallcaps">pider</span> comes without relevant DB content, we follow previous works, and we focus our evaluation on EM scores <cite class="ltx_cite ltx_citemacro_cite">Vougiouklis et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>); Cao et al. (<a href="#bib.bib2" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
<figure id="S5.T2" class="ltx_table">
<table id="S5.T2.45" class="ltx_tabular ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T2.45.46.1" class="ltx_tr">
<td id="S5.T2.45.46.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_tt" style="width:142.3pt;">
<span id="S5.T2.45.46.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.45.46.1.1.1.1" class="ltx_p"><span id="S5.T2.45.46.1.1.1.1.1" class="ltx_text ltx_font_bold">Model</span></span>
</span>
</td>
<td id="S5.T2.45.46.1.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_tt" style="width:19.9pt;">
<span id="S5.T2.45.46.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.45.46.1.2.1.1" class="ltx_p"><span id="S5.T2.45.46.1.2.1.1.1" class="ltx_text ltx_font_bold">EX</span></span>
</span>
</td>
<td id="S5.T2.45.46.1.3" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_tt" style="width:19.9pt;">
<span id="S5.T2.45.46.1.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.45.46.1.3.1.1" class="ltx_p"><span id="S5.T2.45.46.1.3.1.1.1" class="ltx_text ltx_font_bold">EM</span></span>
</span>
</td>
</tr>
<tr id="S5.T2.2.2" class="ltx_tr">
<td id="S5.T2.2.2.3" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:142.3pt;">
<span id="S5.T2.2.2.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.2.2.3.1.1" class="ltx_p">GraPPa <cite class="ltx_cite ltx_citemacro_cite">Yu et al. (<a href="#bib.bib31" title="" class="ltx_ref">2021</a>)</cite></span>
</span>
</td>
<td id="S5.T2.1.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:19.9pt;">
<span id="S5.T2.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.1.1.1.1.1" class="ltx_p"><math id="S5.T2.1.1.1.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.1.1.1.1.1.m1.1a"><mo id="S5.T2.1.1.1.1.1.m1.1.1" xref="S5.T2.1.1.1.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.1.1.1.1.1.m1.1b"><minus id="S5.T2.1.1.1.1.1.m1.1.1.cmml" xref="S5.T2.1.1.1.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.1.1.1.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.2.2.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:19.9pt;">
<span id="S5.T2.2.2.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.2.2.2.1.1" class="ltx_p"><math id="S5.T2.2.2.2.1.1.m1.1" class="ltx_Math" alttext="73.6" display="inline"><semantics id="S5.T2.2.2.2.1.1.m1.1a"><mn id="S5.T2.2.2.2.1.1.m1.1.1" xref="S5.T2.2.2.2.1.1.m1.1.1.cmml">73.6</mn><annotation-xml encoding="MathML-Content" id="S5.T2.2.2.2.1.1.m1.1b"><cn type="float" id="S5.T2.2.2.2.1.1.m1.1.1.cmml" xref="S5.T2.2.2.2.1.1.m1.1.1">73.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.2.2.2.1.1.m1.1c">73.6</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.4.4" class="ltx_tr">
<td id="S5.T2.4.4.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.4.4.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.4.4.3.1.1" class="ltx_p">FastRAT <cite class="ltx_cite ltx_citemacro_cite">Vougiouklis et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.3.3.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.3.3.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.3.3.1.1.1" class="ltx_p"><math id="S5.T2.3.3.1.1.1.m1.1" class="ltx_Math" alttext="73.2" display="inline"><semantics id="S5.T2.3.3.1.1.1.m1.1a"><mn id="S5.T2.3.3.1.1.1.m1.1.1" xref="S5.T2.3.3.1.1.1.m1.1.1.cmml">73.2</mn><annotation-xml encoding="MathML-Content" id="S5.T2.3.3.1.1.1.m1.1b"><cn type="float" id="S5.T2.3.3.1.1.1.m1.1.1.cmml" xref="S5.T2.3.3.1.1.1.m1.1.1">73.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.3.3.1.1.1.m1.1c">73.2</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.4.4.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.4.4.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.4.4.2.1.1" class="ltx_p"><math id="S5.T2.4.4.2.1.1.m1.1" class="ltx_Math" alttext="69.1" display="inline"><semantics id="S5.T2.4.4.2.1.1.m1.1a"><mn id="S5.T2.4.4.2.1.1.m1.1.1" xref="S5.T2.4.4.2.1.1.m1.1.1.cmml">69.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.4.4.2.1.1.m1.1b"><cn type="float" id="S5.T2.4.4.2.1.1.m1.1.1.cmml" xref="S5.T2.4.4.2.1.1.m1.1.1">69.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.4.4.2.1.1.m1.1c">69.1</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.7.7" class="ltx_tr">
<td id="S5.T2.5.5.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.5.5.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.5.5.1.1.1" class="ltx_p"><math id="S5.T2.5.5.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T2.5.5.1.1.1.m1.1a"><msub id="S5.T2.5.5.1.1.1.m1.1.1" xref="S5.T2.5.5.1.1.1.m1.1.1.cmml"><mtext id="S5.T2.5.5.1.1.1.m1.1.1.2" xref="S5.T2.5.5.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T2.5.5.1.1.1.m1.1.1.3" xref="S5.T2.5.5.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T2.5.5.1.1.1.m1.1b"><apply id="S5.T2.5.5.1.1.1.m1.1.1.cmml" xref="S5.T2.5.5.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T2.5.5.1.1.1.m1.1.1.1.cmml" xref="S5.T2.5.5.1.1.1.m1.1.1">subscript</csymbol><ci id="S5.T2.5.5.1.1.1.m1.1.1.2a.cmml" xref="S5.T2.5.5.1.1.1.m1.1.1.2"><mtext id="S5.T2.5.5.1.1.1.m1.1.1.2.cmml" xref="S5.T2.5.5.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T2.5.5.1.1.1.m1.1.1.3a.cmml" xref="S5.T2.5.5.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T2.5.5.1.1.1.m1.1.1.3.cmml" xref="S5.T2.5.5.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.5.5.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.6.6.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.6.6.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.6.6.2.1.1" class="ltx_p"><math id="S5.T2.6.6.2.1.1.m1.1" class="ltx_Math" alttext="71.5" display="inline"><semantics id="S5.T2.6.6.2.1.1.m1.1a"><mn id="S5.T2.6.6.2.1.1.m1.1.1" xref="S5.T2.6.6.2.1.1.m1.1.1.cmml">71.5</mn><annotation-xml encoding="MathML-Content" id="S5.T2.6.6.2.1.1.m1.1b"><cn type="float" id="S5.T2.6.6.2.1.1.m1.1.1.cmml" xref="S5.T2.6.6.2.1.1.m1.1.1">71.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.6.6.2.1.1.m1.1c">71.5</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.7.7.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.7.7.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.7.7.3.1.1" class="ltx_p"><math id="S5.T2.7.7.3.1.1.m1.1" class="ltx_Math" alttext="64.2" display="inline"><semantics id="S5.T2.7.7.3.1.1.m1.1a"><mn id="S5.T2.7.7.3.1.1.m1.1.1" xref="S5.T2.7.7.3.1.1.m1.1.1.cmml">64.2</mn><annotation-xml encoding="MathML-Content" id="S5.T2.7.7.3.1.1.m1.1b"><cn type="float" id="S5.T2.7.7.3.1.1.m1.1.1.cmml" xref="S5.T2.7.7.3.1.1.m1.1.1">64.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.7.7.3.1.1.m1.1c">64.2</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.9.9" class="ltx_tr">
<td id="S5.T2.9.9.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.9.9.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.9.9.3.1.1" class="ltx_p">Graphix-T5 <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib13" title="" class="ltx_ref">2023b</a>)</cite></span>
</span>
</td>
<td id="S5.T2.8.8.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.8.8.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.8.8.1.1.1" class="ltx_p"><math id="S5.T2.8.8.1.1.1.m1.1" class="ltx_Math" alttext="81.0" display="inline"><semantics id="S5.T2.8.8.1.1.1.m1.1a"><mn id="S5.T2.8.8.1.1.1.m1.1.1" xref="S5.T2.8.8.1.1.1.m1.1.1.cmml">81.0</mn><annotation-xml encoding="MathML-Content" id="S5.T2.8.8.1.1.1.m1.1b"><cn type="float" id="S5.T2.8.8.1.1.1.m1.1.1.cmml" xref="S5.T2.8.8.1.1.1.m1.1.1">81.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.8.8.1.1.1.m1.1c">81.0</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.9.9.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.9.9.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.9.9.2.1.1" class="ltx_p"><math id="S5.T2.9.9.2.1.1.m1.1" class="ltx_Math" alttext="77.1" display="inline"><semantics id="S5.T2.9.9.2.1.1.m1.1a"><mn id="S5.T2.9.9.2.1.1.m1.1.1" xref="S5.T2.9.9.2.1.1.m1.1.1.cmml">77.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.9.9.2.1.1.m1.1b"><cn type="float" id="S5.T2.9.9.2.1.1.m1.1.1.cmml" xref="S5.T2.9.9.2.1.1.m1.1.1">77.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.9.9.2.1.1.m1.1c">77.1</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.11.11" class="ltx_tr">
<td id="S5.T2.11.11.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.11.11.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.11.11.3.1.1" class="ltx_p">RESDSQL <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib12" title="" class="ltx_ref">2023a</a>)</cite></span>
</span>
</td>
<td id="S5.T2.10.10.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.10.10.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.10.10.1.1.1" class="ltx_p"><math id="S5.T2.10.10.1.1.1.m1.1" class="ltx_Math" alttext="84.1" display="inline"><semantics id="S5.T2.10.10.1.1.1.m1.1a"><mn id="S5.T2.10.10.1.1.1.m1.1.1" xref="S5.T2.10.10.1.1.1.m1.1.1.cmml">84.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.10.10.1.1.1.m1.1b"><cn type="float" id="S5.T2.10.10.1.1.1.m1.1.1.cmml" xref="S5.T2.10.10.1.1.1.m1.1.1">84.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.10.10.1.1.1.m1.1c">84.1</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.11.11.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.11.11.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.11.11.2.1.1" class="ltx_p"><math id="S5.T2.11.11.2.1.1.m1.1" class="ltx_Math" alttext="\mathbf{80.5}" display="inline"><semantics id="S5.T2.11.11.2.1.1.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T2.11.11.2.1.1.m1.1.1" xref="S5.T2.11.11.2.1.1.m1.1.1.cmml">80.5</mn><annotation-xml encoding="MathML-Content" id="S5.T2.11.11.2.1.1.m1.1b"><cn type="float" id="S5.T2.11.11.2.1.1.m1.1.1.cmml" xref="S5.T2.11.11.2.1.1.m1.1.1">80.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.11.11.2.1.1.m1.1c">\mathbf{80.5}</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.45.47.2" class="ltx_tr">
<td id="S5.T2.45.47.2.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:142.3pt;" colspan="3"><span id="S5.T2.45.47.2.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span></td>
</tr>
<tr id="S5.T2.14.14" class="ltx_tr">
<td id="S5.T2.12.12.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.12.12.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.12.12.1.1.1" class="ltx_p">Ours (w/ <math id="S5.T2.12.12.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T2.12.12.1.1.1.m1.1a"><msub id="S5.T2.12.12.1.1.1.m1.1.1" xref="S5.T2.12.12.1.1.1.m1.1.1.cmml"><mtext id="S5.T2.12.12.1.1.1.m1.1.1.2" xref="S5.T2.12.12.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T2.12.12.1.1.1.m1.1.1.3" xref="S5.T2.12.12.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T2.12.12.1.1.1.m1.1b"><apply id="S5.T2.12.12.1.1.1.m1.1.1.cmml" xref="S5.T2.12.12.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T2.12.12.1.1.1.m1.1.1.1.cmml" xref="S5.T2.12.12.1.1.1.m1.1.1">subscript</csymbol><ci id="S5.T2.12.12.1.1.1.m1.1.1.2a.cmml" xref="S5.T2.12.12.1.1.1.m1.1.1.2"><mtext id="S5.T2.12.12.1.1.1.m1.1.1.2.cmml" xref="S5.T2.12.12.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T2.12.12.1.1.1.m1.1.1.3a.cmml" xref="S5.T2.12.12.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T2.12.12.1.1.1.m1.1.1.3.cmml" xref="S5.T2.12.12.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.12.12.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</span>
</span>
</td>
<td id="S5.T2.13.13.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.13.13.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.13.13.2.1.1" class="ltx_p"><math id="S5.T2.13.13.2.1.1.m1.1" class="ltx_Math" alttext="81.5" display="inline"><semantics id="S5.T2.13.13.2.1.1.m1.1a"><mn id="S5.T2.13.13.2.1.1.m1.1.1" xref="S5.T2.13.13.2.1.1.m1.1.1.cmml">81.5</mn><annotation-xml encoding="MathML-Content" id="S5.T2.13.13.2.1.1.m1.1b"><cn type="float" id="S5.T2.13.13.2.1.1.m1.1.1.cmml" xref="S5.T2.13.13.2.1.1.m1.1.1">81.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.13.13.2.1.1.m1.1c">81.5</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.14.14.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.14.14.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.14.14.3.1.1" class="ltx_p"><math id="S5.T2.14.14.3.1.1.m1.1" class="ltx_Math" alttext="62.1" display="inline"><semantics id="S5.T2.14.14.3.1.1.m1.1a"><mn id="S5.T2.14.14.3.1.1.m1.1.1" xref="S5.T2.14.14.3.1.1.m1.1.1.cmml">62.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.14.14.3.1.1.m1.1b"><cn type="float" id="S5.T2.14.14.3.1.1.m1.1.1.cmml" xref="S5.T2.14.14.3.1.1.m1.1.1">62.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.14.14.3.1.1.m1.1c">62.1</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.16.16" class="ltx_tr">
<td id="S5.T2.16.16.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.16.16.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.16.16.3.1.1" class="ltx_p">Ours (w/ Graphix-T5)</span>
</span>
</td>
<td id="S5.T2.15.15.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.15.15.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.15.15.1.1.1" class="ltx_p"><math id="S5.T2.15.15.1.1.1.m1.1" class="ltx_Math" alttext="83.4" display="inline"><semantics id="S5.T2.15.15.1.1.1.m1.1a"><mn id="S5.T2.15.15.1.1.1.m1.1.1" xref="S5.T2.15.15.1.1.1.m1.1.1.cmml">83.4</mn><annotation-xml encoding="MathML-Content" id="S5.T2.15.15.1.1.1.m1.1b"><cn type="float" id="S5.T2.15.15.1.1.1.m1.1.1.cmml" xref="S5.T2.15.15.1.1.1.m1.1.1">83.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.15.15.1.1.1.m1.1c">83.4</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.16.16.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.16.16.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.16.16.2.1.1" class="ltx_p"><math id="S5.T2.16.16.2.1.1.m1.1" class="ltx_Math" alttext="64.7" display="inline"><semantics id="S5.T2.16.16.2.1.1.m1.1a"><mn id="S5.T2.16.16.2.1.1.m1.1.1" xref="S5.T2.16.16.2.1.1.m1.1.1.cmml">64.7</mn><annotation-xml encoding="MathML-Content" id="S5.T2.16.16.2.1.1.m1.1b"><cn type="float" id="S5.T2.16.16.2.1.1.m1.1.1.cmml" xref="S5.T2.16.16.2.1.1.m1.1.1">64.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.16.16.2.1.1.m1.1c">64.7</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.45.48.3" class="ltx_tr">
<td id="S5.T2.45.48.3.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:142.3pt;" colspan="3"><span id="S5.T2.45.48.3.1.1" class="ltx_text ltx_font_typewriter">PaLM2</span></td>
</tr>
<tr id="S5.T2.18.18" class="ltx_tr">
<td id="S5.T2.18.18.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.18.18.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.18.18.3.1.1" class="ltx_p"><span id="S5.T2.18.18.3.1.1.1" class="ltx_text ltx_font_italic">Few-shot</span> SQL-PaLM <cite class="ltx_cite ltx_citemacro_cite">Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite></span>
</span>
</td>
<td id="S5.T2.17.17.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.17.17.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.17.17.1.1.1" class="ltx_p"><math id="S5.T2.17.17.1.1.1.m1.1" class="ltx_Math" alttext="82.7" display="inline"><semantics id="S5.T2.17.17.1.1.1.m1.1a"><mn id="S5.T2.17.17.1.1.1.m1.1.1" xref="S5.T2.17.17.1.1.1.m1.1.1.cmml">82.7</mn><annotation-xml encoding="MathML-Content" id="S5.T2.17.17.1.1.1.m1.1b"><cn type="float" id="S5.T2.17.17.1.1.1.m1.1.1.cmml" xref="S5.T2.17.17.1.1.1.m1.1.1">82.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.17.17.1.1.1.m1.1c">82.7</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.18.18.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.18.18.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.18.18.2.1.1" class="ltx_p"><math id="S5.T2.18.18.2.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.18.18.2.1.1.m1.1a"><mo id="S5.T2.18.18.2.1.1.m1.1.1" xref="S5.T2.18.18.2.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.18.18.2.1.1.m1.1b"><minus id="S5.T2.18.18.2.1.1.m1.1.1.cmml" xref="S5.T2.18.18.2.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.18.18.2.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.45.49.4" class="ltx_tr">
<td id="S5.T2.45.49.4.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:142.3pt;" colspan="3"><span id="S5.T2.45.49.4.1.1" class="ltx_text ltx_font_typewriter">text-davinci-003</span></td>
</tr>
<tr id="S5.T2.20.20" class="ltx_tr">
<td id="S5.T2.20.20.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.20.20.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.20.20.3.1.1" class="ltx_p">Zero-shot <cite class="ltx_cite ltx_citemacro_cite">Guo et al. (<a href="#bib.bib11" title="" class="ltx_ref">2024</a>)</cite></span>
</span>
</td>
<td id="S5.T2.19.19.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.19.19.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.19.19.1.1.1" class="ltx_p"><math id="S5.T2.19.19.1.1.1.m1.1" class="ltx_Math" alttext="73.1" display="inline"><semantics id="S5.T2.19.19.1.1.1.m1.1a"><mn id="S5.T2.19.19.1.1.1.m1.1.1" xref="S5.T2.19.19.1.1.1.m1.1.1.cmml">73.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.19.19.1.1.1.m1.1b"><cn type="float" id="S5.T2.19.19.1.1.1.m1.1.1.cmml" xref="S5.T2.19.19.1.1.1.m1.1.1">73.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.19.19.1.1.1.m1.1c">73.1</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.20.20.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.20.20.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.20.20.2.1.1" class="ltx_p"><math id="S5.T2.20.20.2.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.20.20.2.1.1.m1.1a"><mo id="S5.T2.20.20.2.1.1.m1.1.1" xref="S5.T2.20.20.2.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.20.20.2.1.1.m1.1b"><minus id="S5.T2.20.20.2.1.1.m1.1.1.cmml" xref="S5.T2.20.20.2.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.20.20.2.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.23.23" class="ltx_tr">
<td id="S5.T2.21.21.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.21.21.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.21.21.1.1.1" class="ltx_p">RAG w<math id="S5.T2.21.21.1.1.1.m1.1" class="ltx_Math" alttext="/" display="inline"><semantics id="S5.T2.21.21.1.1.1.m1.1a"><mo id="S5.T2.21.21.1.1.1.m1.1.1" xref="S5.T2.21.21.1.1.1.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="S5.T2.21.21.1.1.1.m1.1b"><divide id="S5.T2.21.21.1.1.1.m1.1.1.cmml" xref="S5.T2.21.21.1.1.1.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.21.21.1.1.1.m1.1c">/</annotation></semantics></math> Rev. Chain <cite class="ltx_cite ltx_citemacro_cite">Guo et al. (<a href="#bib.bib11" title="" class="ltx_ref">2024</a>)</cite></span>
</span>
</td>
<td id="S5.T2.22.22.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.22.22.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.22.22.2.1.1" class="ltx_p"><math id="S5.T2.22.22.2.1.1.m1.1" class="ltx_Math" alttext="\underline{85.0}" display="inline"><semantics id="S5.T2.22.22.2.1.1.m1.1a"><munder accentunder="true" id="S5.T2.22.22.2.1.1.m1.1.1" xref="S5.T2.22.22.2.1.1.m1.1.1.cmml"><mn id="S5.T2.22.22.2.1.1.m1.1.1.2" xref="S5.T2.22.22.2.1.1.m1.1.1.2.cmml">85.0</mn><mo id="S5.T2.22.22.2.1.1.m1.1.1.1" xref="S5.T2.22.22.2.1.1.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T2.22.22.2.1.1.m1.1b"><apply id="S5.T2.22.22.2.1.1.m1.1.1.cmml" xref="S5.T2.22.22.2.1.1.m1.1.1"><ci id="S5.T2.22.22.2.1.1.m1.1.1.1.cmml" xref="S5.T2.22.22.2.1.1.m1.1.1.1">¯</ci><cn type="float" id="S5.T2.22.22.2.1.1.m1.1.1.2.cmml" xref="S5.T2.22.22.2.1.1.m1.1.1.2">85.0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.22.22.2.1.1.m1.1c">\underline{85.0}</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.23.23.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.23.23.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.23.23.3.1.1" class="ltx_p"><math id="S5.T2.23.23.3.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.23.23.3.1.1.m1.1a"><mo id="S5.T2.23.23.3.1.1.m1.1.1" xref="S5.T2.23.23.3.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.23.23.3.1.1.m1.1b"><minus id="S5.T2.23.23.3.1.1.m1.1.1.cmml" xref="S5.T2.23.23.3.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.23.23.3.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.45.50.5" class="ltx_tr">
<td id="S5.T2.45.50.5.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:142.3pt;" colspan="3"><span id="S5.T2.45.50.5.1.1" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span></td>
</tr>
<tr id="S5.T2.25.25" class="ltx_tr">
<td id="S5.T2.25.25.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.25.25.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.25.25.3.1.1" class="ltx_p">Zero-shot <cite class="ltx_cite ltx_citemacro_cite">Liu et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.24.24.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.24.24.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.24.24.1.1.1" class="ltx_p"><math id="S5.T2.24.24.1.1.1.m1.1" class="ltx_Math" alttext="70.1" display="inline"><semantics id="S5.T2.24.24.1.1.1.m1.1a"><mn id="S5.T2.24.24.1.1.1.m1.1.1" xref="S5.T2.24.24.1.1.1.m1.1.1.cmml">70.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.24.24.1.1.1.m1.1b"><cn type="float" id="S5.T2.24.24.1.1.1.m1.1.1.cmml" xref="S5.T2.24.24.1.1.1.m1.1.1">70.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.24.24.1.1.1.m1.1c">70.1</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.25.25.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.25.25.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.25.25.2.1.1" class="ltx_p"><math id="S5.T2.25.25.2.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.25.25.2.1.1.m1.1a"><mo id="S5.T2.25.25.2.1.1.m1.1.1" xref="S5.T2.25.25.2.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.25.25.2.1.1.m1.1b"><minus id="S5.T2.25.25.2.1.1.m1.1.1.cmml" xref="S5.T2.25.25.2.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.25.25.2.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.27.27" class="ltx_tr">
<td id="S5.T2.27.27.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.27.27.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.27.27.3.1.1" class="ltx_p">C3 <cite class="ltx_cite ltx_citemacro_cite">Dong et al. (<a href="#bib.bib6" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.26.26.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.26.26.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.26.26.1.1.1" class="ltx_p"><math id="S5.T2.26.26.1.1.1.m1.1" class="ltx_Math" alttext="81.8" display="inline"><semantics id="S5.T2.26.26.1.1.1.m1.1a"><mn id="S5.T2.26.26.1.1.1.m1.1.1" xref="S5.T2.26.26.1.1.1.m1.1.1.cmml">81.8</mn><annotation-xml encoding="MathML-Content" id="S5.T2.26.26.1.1.1.m1.1b"><cn type="float" id="S5.T2.26.26.1.1.1.m1.1.1.cmml" xref="S5.T2.26.26.1.1.1.m1.1.1">81.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.26.26.1.1.1.m1.1c">81.8</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.27.27.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.27.27.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.27.27.2.1.1" class="ltx_p"><math id="S5.T2.27.27.2.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.27.27.2.1.1.m1.1a"><mo id="S5.T2.27.27.2.1.1.m1.1.1" xref="S5.T2.27.27.2.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.27.27.2.1.1.m1.1b"><minus id="S5.T2.27.27.2.1.1.m1.1.1.cmml" xref="S5.T2.27.27.2.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.27.27.2.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.29.29" class="ltx_tr">
<td id="S5.T2.29.29.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.29.29.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.29.29.3.1.1" class="ltx_p">DAIL-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.28.28.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.28.28.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.28.28.1.1.1" class="ltx_p"><math id="S5.T2.28.28.1.1.1.m1.1" class="ltx_Math" alttext="79.0" display="inline"><semantics id="S5.T2.28.28.1.1.1.m1.1a"><mn id="S5.T2.28.28.1.1.1.m1.1.1" xref="S5.T2.28.28.1.1.1.m1.1.1.cmml">79.0</mn><annotation-xml encoding="MathML-Content" id="S5.T2.28.28.1.1.1.m1.1b"><cn type="float" id="S5.T2.28.28.1.1.1.m1.1.1.cmml" xref="S5.T2.28.28.1.1.1.m1.1.1">79.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.28.28.1.1.1.m1.1c">79.0</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.29.29.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.29.29.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.29.29.2.1.1" class="ltx_p"><math id="S5.T2.29.29.2.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T2.29.29.2.1.1.m1.1a"><mo id="S5.T2.29.29.2.1.1.m1.1.1" xref="S5.T2.29.29.2.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T2.29.29.2.1.1.m1.1b"><minus id="S5.T2.29.29.2.1.1.m1.1.1.cmml" xref="S5.T2.29.29.2.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.29.29.2.1.1.m1.1c">-</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.32.32" class="ltx_tr">
<td id="S5.T2.30.30.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.30.30.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.30.30.1.1.1" class="ltx_p">Ours (w/ <math id="S5.T2.30.30.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T2.30.30.1.1.1.m1.1a"><msub id="S5.T2.30.30.1.1.1.m1.1.1" xref="S5.T2.30.30.1.1.1.m1.1.1.cmml"><mtext id="S5.T2.30.30.1.1.1.m1.1.1.2" xref="S5.T2.30.30.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T2.30.30.1.1.1.m1.1.1.3" xref="S5.T2.30.30.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T2.30.30.1.1.1.m1.1b"><apply id="S5.T2.30.30.1.1.1.m1.1.1.cmml" xref="S5.T2.30.30.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T2.30.30.1.1.1.m1.1.1.1.cmml" xref="S5.T2.30.30.1.1.1.m1.1.1">subscript</csymbol><ci id="S5.T2.30.30.1.1.1.m1.1.1.2a.cmml" xref="S5.T2.30.30.1.1.1.m1.1.1.2"><mtext id="S5.T2.30.30.1.1.1.m1.1.1.2.cmml" xref="S5.T2.30.30.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T2.30.30.1.1.1.m1.1.1.3a.cmml" xref="S5.T2.30.30.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T2.30.30.1.1.1.m1.1.1.3.cmml" xref="S5.T2.30.30.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.30.30.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</span>
</span>
</td>
<td id="S5.T2.31.31.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.31.31.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.31.31.2.1.1" class="ltx_p"><math id="S5.T2.31.31.2.1.1.m1.1" class="ltx_Math" alttext="82.0" display="inline"><semantics id="S5.T2.31.31.2.1.1.m1.1a"><mn id="S5.T2.31.31.2.1.1.m1.1.1" xref="S5.T2.31.31.2.1.1.m1.1.1.cmml">82.0</mn><annotation-xml encoding="MathML-Content" id="S5.T2.31.31.2.1.1.m1.1b"><cn type="float" id="S5.T2.31.31.2.1.1.m1.1.1.cmml" xref="S5.T2.31.31.2.1.1.m1.1.1">82.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.31.31.2.1.1.m1.1c">82.0</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.32.32.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.32.32.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.32.32.3.1.1" class="ltx_p"><math id="S5.T2.32.32.3.1.1.m1.1" class="ltx_Math" alttext="65.7" display="inline"><semantics id="S5.T2.32.32.3.1.1.m1.1a"><mn id="S5.T2.32.32.3.1.1.m1.1.1" xref="S5.T2.32.32.3.1.1.m1.1.1.cmml">65.7</mn><annotation-xml encoding="MathML-Content" id="S5.T2.32.32.3.1.1.m1.1b"><cn type="float" id="S5.T2.32.32.3.1.1.m1.1.1.cmml" xref="S5.T2.32.32.3.1.1.m1.1.1">65.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.32.32.3.1.1.m1.1c">65.7</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.34.34" class="ltx_tr">
<td id="S5.T2.34.34.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.34.34.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.34.34.3.1.1" class="ltx_p">Ours (w/ Graphix-T5)</span>
</span>
</td>
<td id="S5.T2.33.33.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.33.33.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.33.33.1.1.1" class="ltx_p"><math id="S5.T2.33.33.1.1.1.m1.1" class="ltx_Math" alttext="83.0" display="inline"><semantics id="S5.T2.33.33.1.1.1.m1.1a"><mn id="S5.T2.33.33.1.1.1.m1.1.1" xref="S5.T2.33.33.1.1.1.m1.1.1.cmml">83.0</mn><annotation-xml encoding="MathML-Content" id="S5.T2.33.33.1.1.1.m1.1b"><cn type="float" id="S5.T2.33.33.1.1.1.m1.1.1.cmml" xref="S5.T2.33.33.1.1.1.m1.1.1">83.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.33.33.1.1.1.m1.1c">83.0</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.34.34.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.34.34.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.34.34.2.1.1" class="ltx_p"><math id="S5.T2.34.34.2.1.1.m1.1" class="ltx_Math" alttext="68.8" display="inline"><semantics id="S5.T2.34.34.2.1.1.m1.1a"><mn id="S5.T2.34.34.2.1.1.m1.1.1" xref="S5.T2.34.34.2.1.1.m1.1.1.cmml">68.8</mn><annotation-xml encoding="MathML-Content" id="S5.T2.34.34.2.1.1.m1.1b"><cn type="float" id="S5.T2.34.34.2.1.1.m1.1.1.cmml" xref="S5.T2.34.34.2.1.1.m1.1.1">68.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.34.34.2.1.1.m1.1c">68.8</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.45.51.6" class="ltx_tr">
<td id="S5.T2.45.51.6.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:142.3pt;" colspan="3"><span id="S5.T2.45.51.6.1.1" class="ltx_text ltx_font_typewriter">gpt-4</span></td>
</tr>
<tr id="S5.T2.36.36" class="ltx_tr">
<td id="S5.T2.36.36.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.36.36.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.36.36.3.1.1" class="ltx_p">Zero-shot <cite class="ltx_cite ltx_citemacro_cite">Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.35.35.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.35.35.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.35.35.1.1.1" class="ltx_p"><math id="S5.T2.35.35.1.1.1.m1.1" class="ltx_Math" alttext="72.9" display="inline"><semantics id="S5.T2.35.35.1.1.1.m1.1a"><mn id="S5.T2.35.35.1.1.1.m1.1.1" xref="S5.T2.35.35.1.1.1.m1.1.1.cmml">72.9</mn><annotation-xml encoding="MathML-Content" id="S5.T2.35.35.1.1.1.m1.1b"><cn type="float" id="S5.T2.35.35.1.1.1.m1.1.1.cmml" xref="S5.T2.35.35.1.1.1.m1.1.1">72.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.35.35.1.1.1.m1.1c">72.9</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.36.36.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.36.36.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.36.36.2.1.1" class="ltx_p"><math id="S5.T2.36.36.2.1.1.m1.1" class="ltx_Math" alttext="40.4" display="inline"><semantics id="S5.T2.36.36.2.1.1.m1.1a"><mn id="S5.T2.36.36.2.1.1.m1.1.1" xref="S5.T2.36.36.2.1.1.m1.1.1.cmml">40.4</mn><annotation-xml encoding="MathML-Content" id="S5.T2.36.36.2.1.1.m1.1b"><cn type="float" id="S5.T2.36.36.2.1.1.m1.1.1.cmml" xref="S5.T2.36.36.2.1.1.m1.1.1">40.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.36.36.2.1.1.m1.1c">40.4</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.38.38" class="ltx_tr">
<td id="S5.T2.38.38.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.38.38.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.38.38.3.1.1" class="ltx_p">DIN-SQL <cite class="ltx_cite ltx_citemacro_cite">Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.37.37.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.37.37.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.37.37.1.1.1" class="ltx_p"><math id="S5.T2.37.37.1.1.1.m1.1" class="ltx_Math" alttext="82.8" display="inline"><semantics id="S5.T2.37.37.1.1.1.m1.1a"><mn id="S5.T2.37.37.1.1.1.m1.1.1" xref="S5.T2.37.37.1.1.1.m1.1.1.cmml">82.8</mn><annotation-xml encoding="MathML-Content" id="S5.T2.37.37.1.1.1.m1.1b"><cn type="float" id="S5.T2.37.37.1.1.1.m1.1.1.cmml" xref="S5.T2.37.37.1.1.1.m1.1.1">82.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.37.37.1.1.1.m1.1c">82.8</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.38.38.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.38.38.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.38.38.2.1.1" class="ltx_p"><math id="S5.T2.38.38.2.1.1.m1.1" class="ltx_Math" alttext="60.1" display="inline"><semantics id="S5.T2.38.38.2.1.1.m1.1a"><mn id="S5.T2.38.38.2.1.1.m1.1.1" xref="S5.T2.38.38.2.1.1.m1.1.1.cmml">60.1</mn><annotation-xml encoding="MathML-Content" id="S5.T2.38.38.2.1.1.m1.1b"><cn type="float" id="S5.T2.38.38.2.1.1.m1.1.1.cmml" xref="S5.T2.38.38.2.1.1.m1.1.1">60.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.38.38.2.1.1.m1.1c">60.1</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.40.40" class="ltx_tr">
<td id="S5.T2.40.40.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.40.40.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.40.40.3.1.1" class="ltx_p">DAIL-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite></span>
</span>
</td>
<td id="S5.T2.39.39.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.39.39.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.39.39.1.1.1" class="ltx_p"><math id="S5.T2.39.39.1.1.1.m1.1" class="ltx_Math" alttext="83.6" display="inline"><semantics id="S5.T2.39.39.1.1.1.m1.1a"><mn id="S5.T2.39.39.1.1.1.m1.1.1" xref="S5.T2.39.39.1.1.1.m1.1.1.cmml">83.6</mn><annotation-xml encoding="MathML-Content" id="S5.T2.39.39.1.1.1.m1.1b"><cn type="float" id="S5.T2.39.39.1.1.1.m1.1.1.cmml" xref="S5.T2.39.39.1.1.1.m1.1.1">83.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.39.39.1.1.1.m1.1c">83.6</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.40.40.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.40.40.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.40.40.2.1.1" class="ltx_p"><math id="S5.T2.40.40.2.1.1.m1.1" class="ltx_Math" alttext="68.7" display="inline"><semantics id="S5.T2.40.40.2.1.1.m1.1a"><mn id="S5.T2.40.40.2.1.1.m1.1.1" xref="S5.T2.40.40.2.1.1.m1.1.1.cmml">68.7</mn><annotation-xml encoding="MathML-Content" id="S5.T2.40.40.2.1.1.m1.1b"><cn type="float" id="S5.T2.40.40.2.1.1.m1.1.1.cmml" xref="S5.T2.40.40.2.1.1.m1.1.1">68.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.40.40.2.1.1.m1.1c">68.7</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.43.43" class="ltx_tr">
<td id="S5.T2.41.41.1" class="ltx_td ltx_align_justify ltx_align_middle" style="width:142.3pt;">
<span id="S5.T2.41.41.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.41.41.1.1.1" class="ltx_p">Ours (w/ <math id="S5.T2.41.41.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T2.41.41.1.1.1.m1.1a"><msub id="S5.T2.41.41.1.1.1.m1.1.1" xref="S5.T2.41.41.1.1.1.m1.1.1.cmml"><mtext id="S5.T2.41.41.1.1.1.m1.1.1.2" xref="S5.T2.41.41.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T2.41.41.1.1.1.m1.1.1.3" xref="S5.T2.41.41.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T2.41.41.1.1.1.m1.1b"><apply id="S5.T2.41.41.1.1.1.m1.1.1.cmml" xref="S5.T2.41.41.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T2.41.41.1.1.1.m1.1.1.1.cmml" xref="S5.T2.41.41.1.1.1.m1.1.1">subscript</csymbol><ci id="S5.T2.41.41.1.1.1.m1.1.1.2a.cmml" xref="S5.T2.41.41.1.1.1.m1.1.1.2"><mtext id="S5.T2.41.41.1.1.1.m1.1.1.2.cmml" xref="S5.T2.41.41.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T2.41.41.1.1.1.m1.1.1.3a.cmml" xref="S5.T2.41.41.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T2.41.41.1.1.1.m1.1.1.3.cmml" xref="S5.T2.41.41.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.41.41.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</span>
</span>
</td>
<td id="S5.T2.42.42.2" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.42.42.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.42.42.2.1.1" class="ltx_p"><math id="S5.T2.42.42.2.1.1.m1.1" class="ltx_Math" alttext="84.3" display="inline"><semantics id="S5.T2.42.42.2.1.1.m1.1a"><mn id="S5.T2.42.42.2.1.1.m1.1.1" xref="S5.T2.42.42.2.1.1.m1.1.1.cmml">84.3</mn><annotation-xml encoding="MathML-Content" id="S5.T2.42.42.2.1.1.m1.1b"><cn type="float" id="S5.T2.42.42.2.1.1.m1.1.1.cmml" xref="S5.T2.42.42.2.1.1.m1.1.1">84.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.42.42.2.1.1.m1.1c">84.3</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.43.43.3" class="ltx_td ltx_align_justify ltx_align_middle" style="width:19.9pt;">
<span id="S5.T2.43.43.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.43.43.3.1.1" class="ltx_p"><math id="S5.T2.43.43.3.1.1.m1.1" class="ltx_Math" alttext="73.8" display="inline"><semantics id="S5.T2.43.43.3.1.1.m1.1a"><mn id="S5.T2.43.43.3.1.1.m1.1.1" xref="S5.T2.43.43.3.1.1.m1.1.1.cmml">73.8</mn><annotation-xml encoding="MathML-Content" id="S5.T2.43.43.3.1.1.m1.1b"><cn type="float" id="S5.T2.43.43.3.1.1.m1.1.1.cmml" xref="S5.T2.43.43.3.1.1.m1.1.1">73.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.43.43.3.1.1.m1.1c">73.8</annotation></semantics></math></span>
</span>
</td>
</tr>
<tr id="S5.T2.45.45" class="ltx_tr">
<td id="S5.T2.45.45.3" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb" style="width:142.3pt;">
<span id="S5.T2.45.45.3.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.45.45.3.1.1" class="ltx_p">Ours (w/ Graphix-T5)</span>
</span>
</td>
<td id="S5.T2.44.44.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb" style="width:19.9pt;">
<span id="S5.T2.44.44.1.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.44.44.1.1.1" class="ltx_p"><math id="S5.T2.44.44.1.1.1.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\mathbf{86.6}}" display="inline"><semantics id="S5.T2.44.44.1.1.1.m1.1a"><mn class="ltx_mathvariant_bold" mathcolor="#0000FF" mathvariant="bold" id="S5.T2.44.44.1.1.1.m1.1.1" xref="S5.T2.44.44.1.1.1.m1.1.1.cmml">86.6</mn><annotation-xml encoding="MathML-Content" id="S5.T2.44.44.1.1.1.m1.1b"><cn type="float" id="S5.T2.44.44.1.1.1.m1.1.1.cmml" xref="S5.T2.44.44.1.1.1.m1.1.1">86.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.44.44.1.1.1.m1.1c">{\color[rgb]{0,0,1}\mathbf{86.6}}</annotation></semantics></math></span>
</span>
</td>
<td id="S5.T2.45.45.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb" style="width:19.9pt;">
<span id="S5.T2.45.45.2.1" class="ltx_inline-block ltx_align_top">
<span id="S5.T2.45.45.2.1.1" class="ltx_p"><math id="S5.T2.45.45.2.1.1.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\underline{77.3}}" display="inline"><semantics id="S5.T2.45.45.2.1.1.m1.1a"><munder accentunder="true" id="S5.T2.45.45.2.1.1.m1.1.1" xref="S5.T2.45.45.2.1.1.m1.1.1.cmml"><mn mathcolor="#0000FF" id="S5.T2.45.45.2.1.1.m1.1.1.2" xref="S5.T2.45.45.2.1.1.m1.1.1.2.cmml">77.3</mn><mo mathcolor="#0000FF" id="S5.T2.45.45.2.1.1.m1.1.1.1" xref="S5.T2.45.45.2.1.1.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T2.45.45.2.1.1.m1.1b"><apply id="S5.T2.45.45.2.1.1.m1.1.1.cmml" xref="S5.T2.45.45.2.1.1.m1.1.1"><ci id="S5.T2.45.45.2.1.1.m1.1.1.1.cmml" xref="S5.T2.45.45.2.1.1.m1.1.1.1">¯</ci><cn type="float" id="S5.T2.45.45.2.1.1.m1.1.1.2.cmml" xref="S5.T2.45.45.2.1.1.m1.1.1.2">77.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.45.45.2.1.1.m1.1c">{\color[rgb]{0,0,1}\underline{77.3}}</annotation></semantics></math></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 2: </span>EX and EM accuracies on the development split of S<span id="S5.T2.52.1" class="ltx_text ltx_font_smallcaps">pider</span>. Fine-tuning-based baselines are listed at the top part of the table. Results of our approach are shown with both <math id="S5.T2.47.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T2.47.m1.1b"><msub id="S5.T2.47.m1.1.1" xref="S5.T2.47.m1.1.1.cmml"><mtext id="S5.T2.47.m1.1.1.2" xref="S5.T2.47.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T2.47.m1.1.1.3" xref="S5.T2.47.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T2.47.m1.1c"><apply id="S5.T2.47.m1.1.1.cmml" xref="S5.T2.47.m1.1.1"><csymbol cd="ambiguous" id="S5.T2.47.m1.1.1.1.cmml" xref="S5.T2.47.m1.1.1">subscript</csymbol><ci id="S5.T2.47.m1.1.1.2a.cmml" xref="S5.T2.47.m1.1.1.2"><mtext id="S5.T2.47.m1.1.1.2.cmml" xref="S5.T2.47.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T2.47.m1.1.1.3a.cmml" xref="S5.T2.47.m1.1.1.3"><mtext mathsize="70%" id="S5.T2.47.m1.1.1.3.cmml" xref="S5.T2.47.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T2.47.m1.1d">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> and Graphix-T5 as approximators. The <span id="S5.T2.53.2" class="ltx_text ltx_font_bold">best</span> model is in bold, the <span id="S5.T2.54.3" class="ltx_text ltx_framed ltx_framed_underline">second best</span> is underlined, and the <span id="S5.T2.55.4" class="ltx_text" style="color:#0000FF;">best prompt-based setup</span> is in blue.</figcaption>
</figure>
</section>
<section id="S5.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.2 </span>Baselines</h3>

<div id="S5.SS2.p1" class="ltx_para">
<p id="S5.SS2.p1.1" class="ltx_p">We dichotomize the landscape of baselines in fine-tuning- and prompting-based baselines. Further details are provided in Appendix <a href="#A5" title="Appendix E Baselines ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">E</span></a>.</p>
</div>
<section id="S5.SS2.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Fine-tuning-based</h5>

<div id="S5.SS2.SSS0.Px1.p1" class="ltx_para">
<span id="S5.I3" class="ltx_inline-enumerate">
<span id="S5.I3.i1" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(i)</span> <span id="S5.I3.i1.1" class="ltx_text ltx_font_bold">GraPPa</span><span id="S5.I3.i1.2" class="ltx_text">, </span></span>
<span id="S5.I3.i2" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(ii)</span> <span id="S5.I3.i2.1" class="ltx_text ltx_font_bold">DG-MAML</span><span id="S5.I3.i2.2" class="ltx_text">, </span></span>
<span id="S5.I3.i3" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(iii)</span> <span id="S5.I3.i3.1" class="ltx_text ltx_font_bold">FastRAT</span><span id="S5.I3.i3.2" class="ltx_text">, </span></span>
<span id="S5.I3.i4" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(iv)</span> <span id="S5.I3.i4.1" class="ltx_text ltx_font_bold">Graphix-T5</span><span id="S5.I3.i4.2" class="ltx_text">, </span></span>
<span id="S5.I3.i5" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(v)</span> <span id="S5.I3.i5.1" class="ltx_text ltx_font_bold">RESDSQL</span><span id="S5.I3.i5.2" class="ltx_text">and </span></span>
<span id="S5.I3.i6" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(vi)</span> <span id="S5.I3.i6.1" class="ltx_text ltx_font_bold">HG2AST</span><span id="S5.I3.i6.2" class="ltx_text">.</span></span>
</span>
</div>
</section>
<section id="S5.SS2.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Prompting-based</h5>

<div id="S5.SS2.SSS0.Px2.p1" class="ltx_para">
<p id="S5.SS2.SSS0.Px2.p1.1" class="ltx_p">Zero-shot LLM prompting has been explored by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib11" title="" class="ltx_ref">Guo et al.</a>; <a href="#bib.bib14" title="" class="ltx_ref">Liu et al.</a>; <a href="#bib.bib18" title="" class="ltx_ref">Pourreza and Rafiei</a></cite>; 
<span id="S5.I4" class="ltx_inline-enumerate">
<span id="S5.I4.i1" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(i)</span> <span id="S5.I4.i1.1" class="ltx_text ltx_font_bold">C3</span><span id="S5.I4.i1.2" class="ltx_text">introduces calibration bias for LLM prompting; </span></span>
<span id="S5.I4.i2" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(ii)</span> <span id="S5.I4.i2.1" class="ltx_text ltx_font_bold">DIN-SQL</span><span id="S5.I4.i2.2" class="ltx_text">uses chain-of-thought prompting with pre-defined prompting templates tailored for the assessed question hardness; </span></span>
<span id="S5.I4.i3" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(iii)</span> <span id="S5.I4.i3.1" class="ltx_text ltx_font_bold">DAIL-SQL</span><span id="S5.I4.i3.2" class="ltx_text">uses query approximation and SQL skeleton-based similarities for example selection; </span></span>
<span id="S5.I4.i4" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(iv)</span> <span id="S5.I4.i4.1" class="ltx_text ltx_font_bold">SQL-PaLM</span><span id="S5.I4.i4.2" class="ltx_text">proposes a framework for <span id="S5.I4.i4.2.1" class="ltx_text ltx_font_italic">soft</span> column selection and execution-based refinement; </span></span>
<span id="S5.I4.i5" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(v)</span> <span id="S5.I4.i5.1" class="ltx_text ltx_font_bold">RAG w<math id="S5.I4.i5.1.m1.1" class="ltx_Math" alttext="\mathbf{/}" display="inline"><semantics id="S5.I4.i5.1.m1.1a"><mo id="S5.I4.i5.1.m1.1.1" xref="S5.I4.i5.1.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="S5.I4.i5.1.m1.1b"><divide id="S5.I4.i5.1.m1.1.1.cmml" xref="S5.I4.i5.1.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="S5.I4.i5.1.m1.1c">\mathbf{/}</annotation></semantics></math> Rev. Chain</span><span id="S5.I4.i5.2" class="ltx_text">augments the input prompt with question skeleton-based example retrieval and an execution-based revision chain.</span></span>
</span></p>
</div>
<figure id="S5.T3" class="ltx_table">
<figure id="S5.T3.94" class="ltx_figure ltx_minipage ltx_align_middle" style="width:429.3pt;">
<table id="S5.T3.94.94" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T3.94.94.95.1" class="ltx_tr">
<th id="S5.T3.94.94.95.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt" rowspan="2"><span id="S5.T3.94.94.95.1.1.1" class="ltx_text ltx_font_bold">Model</span></th>
<td id="S5.T3.94.94.95.1.2" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="S5.T3.94.94.95.1.2.1" class="ltx_text ltx_font_bold">S<span id="S5.T3.94.94.95.1.2.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-DK</span></td>
<td id="S5.T3.94.94.95.1.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="S5.T3.94.94.95.1.3.1" class="ltx_text ltx_font_bold">S<span id="S5.T3.94.94.95.1.3.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="S5.T3.94.94.95.1.3.1.2" class="ltx_text ltx_font_smallcaps">eal</span></span></td>
<td id="S5.T3.94.94.95.1.4" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="S5.T3.94.94.95.1.4.1" class="ltx_text ltx_font_bold">S<span id="S5.T3.94.94.95.1.4.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="S5.T3.94.94.95.1.4.1.2" class="ltx_text ltx_font_smallcaps">yn</span></span></td>
<td id="S5.T3.94.94.95.1.5" class="ltx_td ltx_align_center ltx_border_tt"><span id="S5.T3.94.94.95.1.5.1" class="ltx_text ltx_font_bold">CS<span id="S5.T3.94.94.95.1.5.1.1" class="ltx_text ltx_font_smallcaps">pider</span></span></td>
</tr>
<tr id="S5.T3.94.94.96.2" class="ltx_tr">
<td id="S5.T3.94.94.96.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.1.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="S5.T3.94.94.96.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.2.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="S5.T3.94.94.96.2.3" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.3.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="S5.T3.94.94.96.2.4" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.4.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="S5.T3.94.94.96.2.5" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.5.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="S5.T3.94.94.96.2.6" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.6.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="S5.T3.94.94.96.2.7" class="ltx_td ltx_align_center ltx_border_t"><span id="S5.T3.94.94.96.2.7.1" class="ltx_text ltx_font_bold">EM</span></td>
</tr>
<tr id="S5.T3.7.7.7" class="ltx_tr">
<th id="S5.T3.7.7.7.8" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">RAT-SQL + BERT <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib29" title="" class="ltx_ref">2020</a>)</cite>
</th>
<td id="S5.T3.1.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.1.1.1.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.1.1.1.1.m1.1a"><mo id="S5.T3.1.1.1.1.m1.1.1" xref="S5.T3.1.1.1.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.1.1.1.1.m1.1b"><minus id="S5.T3.1.1.1.1.m1.1.1.cmml" xref="S5.T3.1.1.1.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.1.1.1.1.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.2.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.2.2.2.2.m1.1" class="ltx_Math" alttext="40.9" display="inline"><semantics id="S5.T3.2.2.2.2.m1.1a"><mn id="S5.T3.2.2.2.2.m1.1.1" xref="S5.T3.2.2.2.2.m1.1.1.cmml">40.9</mn><annotation-xml encoding="MathML-Content" id="S5.T3.2.2.2.2.m1.1b"><cn type="float" id="S5.T3.2.2.2.2.m1.1.1.cmml" xref="S5.T3.2.2.2.2.m1.1.1">40.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.2.2.2.2.m1.1c">40.9</annotation></semantics></math></td>
<td id="S5.T3.3.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.3.3.3.3.m1.1" class="ltx_Math" alttext="62.1" display="inline"><semantics id="S5.T3.3.3.3.3.m1.1a"><mn id="S5.T3.3.3.3.3.m1.1.1" xref="S5.T3.3.3.3.3.m1.1.1.cmml">62.1</mn><annotation-xml encoding="MathML-Content" id="S5.T3.3.3.3.3.m1.1b"><cn type="float" id="S5.T3.3.3.3.3.m1.1.1.cmml" xref="S5.T3.3.3.3.3.m1.1.1">62.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.3.3.3.3.m1.1c">62.1</annotation></semantics></math></td>
<td id="S5.T3.4.4.4.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.4.4.4.4.m1.1" class="ltx_Math" alttext="58.1" display="inline"><semantics id="S5.T3.4.4.4.4.m1.1a"><mn id="S5.T3.4.4.4.4.m1.1.1" xref="S5.T3.4.4.4.4.m1.1.1.cmml">58.1</mn><annotation-xml encoding="MathML-Content" id="S5.T3.4.4.4.4.m1.1b"><cn type="float" id="S5.T3.4.4.4.4.m1.1.1.cmml" xref="S5.T3.4.4.4.4.m1.1.1">58.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.4.4.4.4.m1.1c">58.1</annotation></semantics></math></td>
<td id="S5.T3.5.5.5.5" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.5.5.5.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.5.5.5.5.m1.1a"><mo id="S5.T3.5.5.5.5.m1.1.1" xref="S5.T3.5.5.5.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.5.5.5.5.m1.1b"><minus id="S5.T3.5.5.5.5.m1.1.1.cmml" xref="S5.T3.5.5.5.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.5.5.5.5.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.6.6.6.6" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.6.6.6.6.m1.1" class="ltx_Math" alttext="48.2" display="inline"><semantics id="S5.T3.6.6.6.6.m1.1a"><mn id="S5.T3.6.6.6.6.m1.1.1" xref="S5.T3.6.6.6.6.m1.1.1.cmml">48.2</mn><annotation-xml encoding="MathML-Content" id="S5.T3.6.6.6.6.m1.1b"><cn type="float" id="S5.T3.6.6.6.6.m1.1.1.cmml" xref="S5.T3.6.6.6.6.m1.1.1">48.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.6.6.6.6.m1.1c">48.2</annotation></semantics></math></td>
<td id="S5.T3.7.7.7.7" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T3.7.7.7.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.7.7.7.7.m1.1a"><mo id="S5.T3.7.7.7.7.m1.1.1" xref="S5.T3.7.7.7.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.7.7.7.7.m1.1b"><minus id="S5.T3.7.7.7.7.m1.1.1.cmml" xref="S5.T3.7.7.7.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.7.7.7.7.m1.1c">-</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.14.14.14" class="ltx_tr">
<th id="S5.T3.14.14.14.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">DG-MAML <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib28" title="" class="ltx_ref">2021</a>)</cite>
</th>
<td id="S5.T3.8.8.8.1" class="ltx_td ltx_align_center"><math id="S5.T3.8.8.8.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.8.8.8.1.m1.1a"><mo id="S5.T3.8.8.8.1.m1.1.1" xref="S5.T3.8.8.8.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.8.8.8.1.m1.1b"><minus id="S5.T3.8.8.8.1.m1.1.1.cmml" xref="S5.T3.8.8.8.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.8.8.8.1.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.9.9.9.2" class="ltx_td ltx_align_center"><math id="S5.T3.9.9.9.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.9.9.9.2.m1.1a"><mo id="S5.T3.9.9.9.2.m1.1.1" xref="S5.T3.9.9.9.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.9.9.9.2.m1.1b"><minus id="S5.T3.9.9.9.2.m1.1.1.cmml" xref="S5.T3.9.9.9.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.9.9.9.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.10.10.10.3" class="ltx_td ltx_align_center"><math id="S5.T3.10.10.10.3.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.10.10.10.3.m1.1a"><mo id="S5.T3.10.10.10.3.m1.1.1" xref="S5.T3.10.10.10.3.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.10.10.10.3.m1.1b"><minus id="S5.T3.10.10.10.3.m1.1.1.cmml" xref="S5.T3.10.10.10.3.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.10.10.10.3.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.11.11.11.4" class="ltx_td ltx_align_center"><math id="S5.T3.11.11.11.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.11.11.11.4.m1.1a"><mo id="S5.T3.11.11.11.4.m1.1.1" xref="S5.T3.11.11.11.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.11.11.11.4.m1.1b"><minus id="S5.T3.11.11.11.4.m1.1.1.cmml" xref="S5.T3.11.11.11.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.11.11.11.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.12.12.12.5" class="ltx_td ltx_align_center"><math id="S5.T3.12.12.12.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.12.12.12.5.m1.1a"><mo id="S5.T3.12.12.12.5.m1.1.1" xref="S5.T3.12.12.12.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.12.12.12.5.m1.1b"><minus id="S5.T3.12.12.12.5.m1.1.1.cmml" xref="S5.T3.12.12.12.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.12.12.12.5.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.13.13.13.6" class="ltx_td ltx_align_center"><math id="S5.T3.13.13.13.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.13.13.13.6.m1.1a"><mo id="S5.T3.13.13.13.6.m1.1.1" xref="S5.T3.13.13.13.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.13.13.13.6.m1.1b"><minus id="S5.T3.13.13.13.6.m1.1.1.cmml" xref="S5.T3.13.13.13.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.13.13.13.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.14.14.14.7" class="ltx_td ltx_align_center"><math id="S5.T3.14.14.14.7.m1.1" class="ltx_Math" alttext="51.0" display="inline"><semantics id="S5.T3.14.14.14.7.m1.1a"><mn id="S5.T3.14.14.14.7.m1.1.1" xref="S5.T3.14.14.14.7.m1.1.1.cmml">51.0</mn><annotation-xml encoding="MathML-Content" id="S5.T3.14.14.14.7.m1.1b"><cn type="float" id="S5.T3.14.14.14.7.m1.1.1.cmml" xref="S5.T3.14.14.14.7.m1.1.1">51.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.14.14.14.7.m1.1c">51.0</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.21.21.21" class="ltx_tr">
<th id="S5.T3.21.21.21.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">FastRAT <cite class="ltx_cite ltx_citemacro_cite">Vougiouklis et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite>
</th>
<td id="S5.T3.15.15.15.1" class="ltx_td ltx_align_center"><math id="S5.T3.15.15.15.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.15.15.15.1.m1.1a"><mo id="S5.T3.15.15.15.1.m1.1.1" xref="S5.T3.15.15.15.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.15.15.15.1.m1.1b"><minus id="S5.T3.15.15.15.1.m1.1.1.cmml" xref="S5.T3.15.15.15.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.15.15.15.1.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.16.16.16.2" class="ltx_td ltx_align_center"><math id="S5.T3.16.16.16.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.16.16.16.2.m1.1a"><mo id="S5.T3.16.16.16.2.m1.1.1" xref="S5.T3.16.16.16.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.16.16.16.2.m1.1b"><minus id="S5.T3.16.16.16.2.m1.1.1.cmml" xref="S5.T3.16.16.16.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.16.16.16.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.17.17.17.3" class="ltx_td ltx_align_center"><math id="S5.T3.17.17.17.3.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.17.17.17.3.m1.1a"><mo id="S5.T3.17.17.17.3.m1.1.1" xref="S5.T3.17.17.17.3.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.17.17.17.3.m1.1b"><minus id="S5.T3.17.17.17.3.m1.1.1.cmml" xref="S5.T3.17.17.17.3.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.17.17.17.3.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.18.18.18.4" class="ltx_td ltx_align_center"><math id="S5.T3.18.18.18.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.18.18.18.4.m1.1a"><mo id="S5.T3.18.18.18.4.m1.1.1" xref="S5.T3.18.18.18.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.18.18.18.4.m1.1b"><minus id="S5.T3.18.18.18.4.m1.1.1.cmml" xref="S5.T3.18.18.18.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.18.18.18.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.19.19.19.5" class="ltx_td ltx_align_center"><math id="S5.T3.19.19.19.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.19.19.19.5.m1.1a"><mo id="S5.T3.19.19.19.5.m1.1.1" xref="S5.T3.19.19.19.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.19.19.19.5.m1.1b"><minus id="S5.T3.19.19.19.5.m1.1.1.cmml" xref="S5.T3.19.19.19.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.19.19.19.5.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.20.20.20.6" class="ltx_td ltx_align_center"><math id="S5.T3.20.20.20.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.20.20.20.6.m1.1a"><mo id="S5.T3.20.20.20.6.m1.1.1" xref="S5.T3.20.20.20.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.20.20.20.6.m1.1b"><minus id="S5.T3.20.20.20.6.m1.1.1.cmml" xref="S5.T3.20.20.20.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.20.20.20.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.21.21.21.7" class="ltx_td ltx_align_center"><math id="S5.T3.21.21.21.7.m1.1" class="ltx_Math" alttext="\underline{61.3}" display="inline"><semantics id="S5.T3.21.21.21.7.m1.1a"><munder accentunder="true" id="S5.T3.21.21.21.7.m1.1.1" xref="S5.T3.21.21.21.7.m1.1.1.cmml"><mn id="S5.T3.21.21.21.7.m1.1.1.2" xref="S5.T3.21.21.21.7.m1.1.1.2.cmml">61.3</mn><mo id="S5.T3.21.21.21.7.m1.1.1.1" xref="S5.T3.21.21.21.7.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.21.21.21.7.m1.1b"><apply id="S5.T3.21.21.21.7.m1.1.1.cmml" xref="S5.T3.21.21.21.7.m1.1.1"><ci id="S5.T3.21.21.21.7.m1.1.1.1.cmml" xref="S5.T3.21.21.21.7.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.21.21.21.7.m1.1.1.2.cmml" xref="S5.T3.21.21.21.7.m1.1.1.2">61.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.21.21.21.7.m1.1c">\underline{61.3}</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.28.28.28" class="ltx_tr">
<th id="S5.T3.22.22.22.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T3.22.22.22.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T3.22.22.22.1.m1.1a"><msub id="S5.T3.22.22.22.1.m1.1.1" xref="S5.T3.22.22.22.1.m1.1.1.cmml"><mtext id="S5.T3.22.22.22.1.m1.1.1.2" xref="S5.T3.22.22.22.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T3.22.22.22.1.m1.1.1.3" xref="S5.T3.22.22.22.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T3.22.22.22.1.m1.1b"><apply id="S5.T3.22.22.22.1.m1.1.1.cmml" xref="S5.T3.22.22.22.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.22.22.22.1.m1.1.1.1.cmml" xref="S5.T3.22.22.22.1.m1.1.1">subscript</csymbol><ci id="S5.T3.22.22.22.1.m1.1.1.2a.cmml" xref="S5.T3.22.22.22.1.m1.1.1.2"><mtext id="S5.T3.22.22.22.1.m1.1.1.2.cmml" xref="S5.T3.22.22.22.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T3.22.22.22.1.m1.1.1.3a.cmml" xref="S5.T3.22.22.22.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T3.22.22.22.1.m1.1.1.3.cmml" xref="S5.T3.22.22.22.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.22.22.22.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<td id="S5.T3.23.23.23.2" class="ltx_td ltx_align_center"><math id="S5.T3.23.23.23.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.23.23.23.2.m1.1a"><mo id="S5.T3.23.23.23.2.m1.1.1" xref="S5.T3.23.23.23.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.23.23.23.2.m1.1b"><minus id="S5.T3.23.23.23.2.m1.1.1.cmml" xref="S5.T3.23.23.23.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.23.23.23.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.24.24.24.3" class="ltx_td ltx_align_center"><math id="S5.T3.24.24.24.3.m1.1" class="ltx_Math" alttext="44.1" display="inline"><semantics id="S5.T3.24.24.24.3.m1.1a"><mn id="S5.T3.24.24.24.3.m1.1.1" xref="S5.T3.24.24.24.3.m1.1.1.cmml">44.1</mn><annotation-xml encoding="MathML-Content" id="S5.T3.24.24.24.3.m1.1b"><cn type="float" id="S5.T3.24.24.24.3.m1.1.1.cmml" xref="S5.T3.24.24.24.3.m1.1.1">44.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.24.24.24.3.m1.1c">44.1</annotation></semantics></math></td>
<td id="S5.T3.25.25.25.4" class="ltx_td ltx_align_center"><math id="S5.T3.25.25.25.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.25.25.25.4.m1.1a"><mo id="S5.T3.25.25.25.4.m1.1.1" xref="S5.T3.25.25.25.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.25.25.25.4.m1.1b"><minus id="S5.T3.25.25.25.4.m1.1.1.cmml" xref="S5.T3.25.25.25.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.25.25.25.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.26.26.26.5" class="ltx_td ltx_align_center"><math id="S5.T3.26.26.26.5.m1.1" class="ltx_Math" alttext="47.8" display="inline"><semantics id="S5.T3.26.26.26.5.m1.1a"><mn id="S5.T3.26.26.26.5.m1.1.1" xref="S5.T3.26.26.26.5.m1.1.1.cmml">47.8</mn><annotation-xml encoding="MathML-Content" id="S5.T3.26.26.26.5.m1.1b"><cn type="float" id="S5.T3.26.26.26.5.m1.1.1.cmml" xref="S5.T3.26.26.26.5.m1.1.1">47.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.26.26.26.5.m1.1c">47.8</annotation></semantics></math></td>
<td id="S5.T3.27.27.27.6" class="ltx_td ltx_align_center"><math id="S5.T3.27.27.27.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.27.27.27.6.m1.1a"><mo id="S5.T3.27.27.27.6.m1.1.1" xref="S5.T3.27.27.27.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.27.27.27.6.m1.1b"><minus id="S5.T3.27.27.27.6.m1.1.1.cmml" xref="S5.T3.27.27.27.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.27.27.27.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.28.28.28.8" class="ltx_td ltx_align_center">48.5</td>
<td id="S5.T3.28.28.28.7" class="ltx_td ltx_align_center"><math id="S5.T3.28.28.28.7.m1.1" class="ltx_Math" alttext="53.2" display="inline"><semantics id="S5.T3.28.28.28.7.m1.1a"><mn id="S5.T3.28.28.28.7.m1.1.1" xref="S5.T3.28.28.28.7.m1.1.1.cmml">53.2</mn><annotation-xml encoding="MathML-Content" id="S5.T3.28.28.28.7.m1.1b"><cn type="float" id="S5.T3.28.28.28.7.m1.1.1.cmml" xref="S5.T3.28.28.28.7.m1.1.1">53.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.28.28.28.7.m1.1c">53.2</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.35.35.35" class="ltx_tr">
<th id="S5.T3.35.35.35.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">HG2AST <cite class="ltx_cite ltx_citemacro_cite">Cao et al. (<a href="#bib.bib2" title="" class="ltx_ref">2023</a>)</cite>
</th>
<td id="S5.T3.29.29.29.1" class="ltx_td ltx_align_center"><math id="S5.T3.29.29.29.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.29.29.29.1.m1.1a"><mo id="S5.T3.29.29.29.1.m1.1.1" xref="S5.T3.29.29.29.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.29.29.29.1.m1.1b"><minus id="S5.T3.29.29.29.1.m1.1.1.cmml" xref="S5.T3.29.29.29.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.29.29.29.1.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.30.30.30.2" class="ltx_td ltx_align_center"><math id="S5.T3.30.30.30.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.30.30.30.2.m1.1a"><mo id="S5.T3.30.30.30.2.m1.1.1" xref="S5.T3.30.30.30.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.30.30.30.2.m1.1b"><minus id="S5.T3.30.30.30.2.m1.1.1.cmml" xref="S5.T3.30.30.30.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.30.30.30.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.31.31.31.3" class="ltx_td ltx_align_center"><math id="S5.T3.31.31.31.3.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.31.31.31.3.m1.1a"><mo id="S5.T3.31.31.31.3.m1.1.1" xref="S5.T3.31.31.31.3.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.31.31.31.3.m1.1b"><minus id="S5.T3.31.31.31.3.m1.1.1.cmml" xref="S5.T3.31.31.31.3.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.31.31.31.3.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.32.32.32.4" class="ltx_td ltx_align_center"><math id="S5.T3.32.32.32.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.32.32.32.4.m1.1a"><mo id="S5.T3.32.32.32.4.m1.1.1" xref="S5.T3.32.32.32.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.32.32.32.4.m1.1b"><minus id="S5.T3.32.32.32.4.m1.1.1.cmml" xref="S5.T3.32.32.32.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.32.32.32.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.33.33.33.5" class="ltx_td ltx_align_center"><math id="S5.T3.33.33.33.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.33.33.33.5.m1.1a"><mo id="S5.T3.33.33.33.5.m1.1.1" xref="S5.T3.33.33.33.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.33.33.33.5.m1.1b"><minus id="S5.T3.33.33.33.5.m1.1.1.cmml" xref="S5.T3.33.33.33.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.33.33.33.5.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.34.34.34.6" class="ltx_td ltx_align_center"><math id="S5.T3.34.34.34.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.34.34.34.6.m1.1a"><mo id="S5.T3.34.34.34.6.m1.1.1" xref="S5.T3.34.34.34.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.34.34.34.6.m1.1b"><minus id="S5.T3.34.34.34.6.m1.1.1.cmml" xref="S5.T3.34.34.34.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.34.34.34.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.35.35.35.7" class="ltx_td ltx_align_center">
<math id="S5.T3.35.35.35.7.m1.1" class="ltx_Math" alttext="61.0" display="inline"><semantics id="S5.T3.35.35.35.7.m1.1a"><mn id="S5.T3.35.35.35.7.m1.1.1" xref="S5.T3.35.35.35.7.m1.1.1.cmml">61.0</mn><annotation-xml encoding="MathML-Content" id="S5.T3.35.35.35.7.m1.1b"><cn type="float" id="S5.T3.35.35.35.7.m1.1.1.cmml" xref="S5.T3.35.35.35.7.m1.1.1">61.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.35.35.35.7.m1.1c">61.0</annotation></semantics></math><span id="footnote5" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">5</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">5</sup><span class="ltx_tag ltx_tag_note">5</span>Without using question translation; <math id="footnote5.m1.1" class="ltx_Math" alttext="64.0" display="inline"><semantics id="footnote5.m1.1b"><mn id="footnote5.m1.1.1" xref="footnote5.m1.1.1.cmml">64.0</mn><annotation-xml encoding="MathML-Content" id="footnote5.m1.1c"><cn type="float" id="footnote5.m1.1.1.cmml" xref="footnote5.m1.1.1">64.0</cn></annotation-xml><annotation encoding="application/x-tex" id="footnote5.m1.1d">64.0</annotation></semantics></math> EM when question translation is used.</span></span></span>
</td>
</tr>
<tr id="S5.T3.42.42.42" class="ltx_tr">
<th id="S5.T3.42.42.42.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">RESDSQL <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib12" title="" class="ltx_ref">2023a</a>)</cite>
</th>
<td id="S5.T3.36.36.36.1" class="ltx_td ltx_align_center"><math id="S5.T3.36.36.36.1.m1.1" class="ltx_Math" alttext="66.0" display="inline"><semantics id="S5.T3.36.36.36.1.m1.1a"><mn id="S5.T3.36.36.36.1.m1.1.1" xref="S5.T3.36.36.36.1.m1.1.1.cmml">66.0</mn><annotation-xml encoding="MathML-Content" id="S5.T3.36.36.36.1.m1.1b"><cn type="float" id="S5.T3.36.36.36.1.m1.1.1.cmml" xref="S5.T3.36.36.36.1.m1.1.1">66.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.36.36.36.1.m1.1c">66.0</annotation></semantics></math></td>
<td id="S5.T3.37.37.37.2" class="ltx_td ltx_align_center"><math id="S5.T3.37.37.37.2.m1.1" class="ltx_Math" alttext="\underline{55.3}" display="inline"><semantics id="S5.T3.37.37.37.2.m1.1a"><munder accentunder="true" id="S5.T3.37.37.37.2.m1.1.1" xref="S5.T3.37.37.37.2.m1.1.1.cmml"><mn id="S5.T3.37.37.37.2.m1.1.1.2" xref="S5.T3.37.37.37.2.m1.1.1.2.cmml">55.3</mn><mo id="S5.T3.37.37.37.2.m1.1.1.1" xref="S5.T3.37.37.37.2.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.37.37.37.2.m1.1b"><apply id="S5.T3.37.37.37.2.m1.1.1.cmml" xref="S5.T3.37.37.37.2.m1.1.1"><ci id="S5.T3.37.37.37.2.m1.1.1.1.cmml" xref="S5.T3.37.37.37.2.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.37.37.37.2.m1.1.1.2.cmml" xref="S5.T3.37.37.37.2.m1.1.1.2">55.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.37.37.37.2.m1.1c">\underline{55.3}</annotation></semantics></math></td>
<td id="S5.T3.38.38.38.3" class="ltx_td ltx_align_center"><math id="S5.T3.38.38.38.3.m1.1" class="ltx_Math" alttext="\mathbf{81.9}" display="inline"><semantics id="S5.T3.38.38.38.3.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.38.38.38.3.m1.1.1" xref="S5.T3.38.38.38.3.m1.1.1.cmml">81.9</mn><annotation-xml encoding="MathML-Content" id="S5.T3.38.38.38.3.m1.1b"><cn type="float" id="S5.T3.38.38.38.3.m1.1.1.cmml" xref="S5.T3.38.38.38.3.m1.1.1">81.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.38.38.38.3.m1.1c">\mathbf{81.9}</annotation></semantics></math></td>
<td id="S5.T3.39.39.39.4" class="ltx_td ltx_align_center"><math id="S5.T3.39.39.39.4.m1.1" class="ltx_Math" alttext="\mathbf{77.4}" display="inline"><semantics id="S5.T3.39.39.39.4.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.39.39.39.4.m1.1.1" xref="S5.T3.39.39.39.4.m1.1.1.cmml">77.4</mn><annotation-xml encoding="MathML-Content" id="S5.T3.39.39.39.4.m1.1b"><cn type="float" id="S5.T3.39.39.39.4.m1.1.1.cmml" xref="S5.T3.39.39.39.4.m1.1.1">77.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.39.39.39.4.m1.1c">\mathbf{77.4}</annotation></semantics></math></td>
<td id="S5.T3.40.40.40.5" class="ltx_td ltx_align_center"><math id="S5.T3.40.40.40.5.m1.1" class="ltx_Math" alttext="\mathbf{76.9}" display="inline"><semantics id="S5.T3.40.40.40.5.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.40.40.40.5.m1.1.1" xref="S5.T3.40.40.40.5.m1.1.1.cmml">76.9</mn><annotation-xml encoding="MathML-Content" id="S5.T3.40.40.40.5.m1.1b"><cn type="float" id="S5.T3.40.40.40.5.m1.1.1.cmml" xref="S5.T3.40.40.40.5.m1.1.1">76.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.40.40.40.5.m1.1c">\mathbf{76.9}</annotation></semantics></math></td>
<td id="S5.T3.41.41.41.6" class="ltx_td ltx_align_center"><math id="S5.T3.41.41.41.6.m1.1" class="ltx_Math" alttext="\mathbf{69.1}" display="inline"><semantics id="S5.T3.41.41.41.6.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T3.41.41.41.6.m1.1.1" xref="S5.T3.41.41.41.6.m1.1.1.cmml">69.1</mn><annotation-xml encoding="MathML-Content" id="S5.T3.41.41.41.6.m1.1b"><cn type="float" id="S5.T3.41.41.41.6.m1.1.1.cmml" xref="S5.T3.41.41.41.6.m1.1.1">69.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.41.41.41.6.m1.1c">\mathbf{69.1}</annotation></semantics></math></td>
<td id="S5.T3.42.42.42.7" class="ltx_td ltx_align_center"><math id="S5.T3.42.42.42.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.42.42.42.7.m1.1a"><mo id="S5.T3.42.42.42.7.m1.1.1" xref="S5.T3.42.42.42.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.42.42.42.7.m1.1b"><minus id="S5.T3.42.42.42.7.m1.1.1.cmml" xref="S5.T3.42.42.42.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.42.42.42.7.m1.1c">-</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.94.94.97.3" class="ltx_tr">
<th id="S5.T3.94.94.97.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="7"><span id="S5.T3.94.94.97.3.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span></th>
<td id="S5.T3.94.94.97.3.2" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S5.T3.50.50.50" class="ltx_tr">
<th id="S5.T3.43.43.43.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ <math id="S5.T3.43.43.43.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T3.43.43.43.1.m1.1a"><msub id="S5.T3.43.43.43.1.m1.1.1" xref="S5.T3.43.43.43.1.m1.1.1.cmml"><mtext id="S5.T3.43.43.43.1.m1.1.1.2" xref="S5.T3.43.43.43.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T3.43.43.43.1.m1.1.1.3" xref="S5.T3.43.43.43.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T3.43.43.43.1.m1.1b"><apply id="S5.T3.43.43.43.1.m1.1.1.cmml" xref="S5.T3.43.43.43.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.43.43.43.1.m1.1.1.1.cmml" xref="S5.T3.43.43.43.1.m1.1.1">subscript</csymbol><ci id="S5.T3.43.43.43.1.m1.1.1.2a.cmml" xref="S5.T3.43.43.43.1.m1.1.1.2"><mtext id="S5.T3.43.43.43.1.m1.1.1.2.cmml" xref="S5.T3.43.43.43.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T3.43.43.43.1.m1.1.1.3a.cmml" xref="S5.T3.43.43.43.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T3.43.43.43.1.m1.1.1.3.cmml" xref="S5.T3.43.43.43.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.43.43.43.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="S5.T3.44.44.44.2" class="ltx_td ltx_align_center"><math id="S5.T3.44.44.44.2.m1.1" class="ltx_Math" alttext="\underline{70.5}" display="inline"><semantics id="S5.T3.44.44.44.2.m1.1a"><munder accentunder="true" id="S5.T3.44.44.44.2.m1.1.1" xref="S5.T3.44.44.44.2.m1.1.1.cmml"><mn id="S5.T3.44.44.44.2.m1.1.1.2" xref="S5.T3.44.44.44.2.m1.1.1.2.cmml">70.5</mn><mo id="S5.T3.44.44.44.2.m1.1.1.1" xref="S5.T3.44.44.44.2.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.44.44.44.2.m1.1b"><apply id="S5.T3.44.44.44.2.m1.1.1.cmml" xref="S5.T3.44.44.44.2.m1.1.1"><ci id="S5.T3.44.44.44.2.m1.1.1.1.cmml" xref="S5.T3.44.44.44.2.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.44.44.44.2.m1.1.1.2.cmml" xref="S5.T3.44.44.44.2.m1.1.1.2">70.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.44.44.44.2.m1.1c">\underline{70.5}</annotation></semantics></math></td>
<td id="S5.T3.45.45.45.3" class="ltx_td ltx_align_center"><math id="S5.T3.45.45.45.3.m1.1" class="ltx_Math" alttext="46.4" display="inline"><semantics id="S5.T3.45.45.45.3.m1.1a"><mn id="S5.T3.45.45.45.3.m1.1.1" xref="S5.T3.45.45.45.3.m1.1.1.cmml">46.4</mn><annotation-xml encoding="MathML-Content" id="S5.T3.45.45.45.3.m1.1b"><cn type="float" id="S5.T3.45.45.45.3.m1.1.1.cmml" xref="S5.T3.45.45.45.3.m1.1.1">46.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.45.45.45.3.m1.1c">46.4</annotation></semantics></math></td>
<td id="S5.T3.46.46.46.4" class="ltx_td ltx_align_center"><math id="S5.T3.46.46.46.4.m1.1" class="ltx_Math" alttext="77.4" display="inline"><semantics id="S5.T3.46.46.46.4.m1.1a"><mn id="S5.T3.46.46.46.4.m1.1.1" xref="S5.T3.46.46.46.4.m1.1.1.cmml">77.4</mn><annotation-xml encoding="MathML-Content" id="S5.T3.46.46.46.4.m1.1b"><cn type="float" id="S5.T3.46.46.46.4.m1.1.1.cmml" xref="S5.T3.46.46.46.4.m1.1.1">77.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.46.46.46.4.m1.1c">77.4</annotation></semantics></math></td>
<td id="S5.T3.47.47.47.5" class="ltx_td ltx_align_center"><math id="S5.T3.47.47.47.5.m1.1" class="ltx_Math" alttext="59.3" display="inline"><semantics id="S5.T3.47.47.47.5.m1.1a"><mn id="S5.T3.47.47.47.5.m1.1.1" xref="S5.T3.47.47.47.5.m1.1.1.cmml">59.3</mn><annotation-xml encoding="MathML-Content" id="S5.T3.47.47.47.5.m1.1b"><cn type="float" id="S5.T3.47.47.47.5.m1.1.1.cmml" xref="S5.T3.47.47.47.5.m1.1.1">59.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.47.47.47.5.m1.1c">59.3</annotation></semantics></math></td>
<td id="S5.T3.48.48.48.6" class="ltx_td ltx_align_center"><math id="S5.T3.48.48.48.6.m1.1" class="ltx_Math" alttext="68.7" display="inline"><semantics id="S5.T3.48.48.48.6.m1.1a"><mn id="S5.T3.48.48.48.6.m1.1.1" xref="S5.T3.48.48.48.6.m1.1.1.cmml">68.7</mn><annotation-xml encoding="MathML-Content" id="S5.T3.48.48.48.6.m1.1b"><cn type="float" id="S5.T3.48.48.48.6.m1.1.1.cmml" xref="S5.T3.48.48.48.6.m1.1.1">68.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.48.48.48.6.m1.1c">68.7</annotation></semantics></math></td>
<td id="S5.T3.49.49.49.7" class="ltx_td ltx_align_center"><math id="S5.T3.49.49.49.7.m1.1" class="ltx_Math" alttext="49.5" display="inline"><semantics id="S5.T3.49.49.49.7.m1.1a"><mn id="S5.T3.49.49.49.7.m1.1.1" xref="S5.T3.49.49.49.7.m1.1.1.cmml">49.5</mn><annotation-xml encoding="MathML-Content" id="S5.T3.49.49.49.7.m1.1b"><cn type="float" id="S5.T3.49.49.49.7.m1.1.1.cmml" xref="S5.T3.49.49.49.7.m1.1.1">49.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.49.49.49.7.m1.1c">49.5</annotation></semantics></math></td>
<td id="S5.T3.50.50.50.8" class="ltx_td ltx_align_center"><math id="S5.T3.50.50.50.8.m1.1" class="ltx_Math" alttext="55.9" display="inline"><semantics id="S5.T3.50.50.50.8.m1.1a"><mn id="S5.T3.50.50.50.8.m1.1.1" xref="S5.T3.50.50.50.8.m1.1.1.cmml">55.9</mn><annotation-xml encoding="MathML-Content" id="S5.T3.50.50.50.8.m1.1b"><cn type="float" id="S5.T3.50.50.50.8.m1.1.1.cmml" xref="S5.T3.50.50.50.8.m1.1.1">55.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.50.50.50.8.m1.1c">55.9</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.94.94.98.4" class="ltx_tr">
<th id="S5.T3.94.94.98.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="7"><span id="S5.T3.94.94.98.4.1.1" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span></th>
<td id="S5.T3.94.94.98.4.2" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S5.T3.57.57.57" class="ltx_tr">
<th id="S5.T3.57.57.57.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">Zero-shot <cite class="ltx_cite ltx_citemacro_cite">Liu et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite>
</th>
<td id="S5.T3.51.51.51.1" class="ltx_td ltx_align_center"><math id="S5.T3.51.51.51.1.m1.1" class="ltx_Math" alttext="62.6" display="inline"><semantics id="S5.T3.51.51.51.1.m1.1a"><mn id="S5.T3.51.51.51.1.m1.1.1" xref="S5.T3.51.51.51.1.m1.1.1.cmml">62.6</mn><annotation-xml encoding="MathML-Content" id="S5.T3.51.51.51.1.m1.1b"><cn type="float" id="S5.T3.51.51.51.1.m1.1.1.cmml" xref="S5.T3.51.51.51.1.m1.1.1">62.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.51.51.51.1.m1.1c">62.6</annotation></semantics></math></td>
<td id="S5.T3.52.52.52.2" class="ltx_td ltx_align_center"><math id="S5.T3.52.52.52.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.52.52.52.2.m1.1a"><mo id="S5.T3.52.52.52.2.m1.1.1" xref="S5.T3.52.52.52.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.52.52.52.2.m1.1b"><minus id="S5.T3.52.52.52.2.m1.1.1.cmml" xref="S5.T3.52.52.52.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.52.52.52.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.53.53.53.3" class="ltx_td ltx_align_center"><math id="S5.T3.53.53.53.3.m1.1" class="ltx_Math" alttext="63.4" display="inline"><semantics id="S5.T3.53.53.53.3.m1.1a"><mn id="S5.T3.53.53.53.3.m1.1.1" xref="S5.T3.53.53.53.3.m1.1.1.cmml">63.4</mn><annotation-xml encoding="MathML-Content" id="S5.T3.53.53.53.3.m1.1b"><cn type="float" id="S5.T3.53.53.53.3.m1.1.1.cmml" xref="S5.T3.53.53.53.3.m1.1.1">63.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.53.53.53.3.m1.1c">63.4</annotation></semantics></math></td>
<td id="S5.T3.54.54.54.4" class="ltx_td ltx_align_center"><math id="S5.T3.54.54.54.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.54.54.54.4.m1.1a"><mo id="S5.T3.54.54.54.4.m1.1.1" xref="S5.T3.54.54.54.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.54.54.54.4.m1.1b"><minus id="S5.T3.54.54.54.4.m1.1.1.cmml" xref="S5.T3.54.54.54.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.54.54.54.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.55.55.55.5" class="ltx_td ltx_align_center"><math id="S5.T3.55.55.55.5.m1.1" class="ltx_Math" alttext="58.6" display="inline"><semantics id="S5.T3.55.55.55.5.m1.1a"><mn id="S5.T3.55.55.55.5.m1.1.1" xref="S5.T3.55.55.55.5.m1.1.1.cmml">58.6</mn><annotation-xml encoding="MathML-Content" id="S5.T3.55.55.55.5.m1.1b"><cn type="float" id="S5.T3.55.55.55.5.m1.1.1.cmml" xref="S5.T3.55.55.55.5.m1.1.1">58.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.55.55.55.5.m1.1c">58.6</annotation></semantics></math></td>
<td id="S5.T3.56.56.56.6" class="ltx_td ltx_align_center"><math id="S5.T3.56.56.56.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.56.56.56.6.m1.1a"><mo id="S5.T3.56.56.56.6.m1.1.1" xref="S5.T3.56.56.56.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.56.56.56.6.m1.1b"><minus id="S5.T3.56.56.56.6.m1.1.1.cmml" xref="S5.T3.56.56.56.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.56.56.56.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.57.57.57.7" class="ltx_td ltx_align_center"><math id="S5.T3.57.57.57.7.m1.1" class="ltx_Math" alttext="32.6" display="inline"><semantics id="S5.T3.57.57.57.7.m1.1a"><mn id="S5.T3.57.57.57.7.m1.1.1" xref="S5.T3.57.57.57.7.m1.1.1.cmml">32.6</mn><annotation-xml encoding="MathML-Content" id="S5.T3.57.57.57.7.m1.1b"><cn type="float" id="S5.T3.57.57.57.7.m1.1.1.cmml" xref="S5.T3.57.57.57.7.m1.1.1">32.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.57.57.57.7.m1.1c">32.6</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.64.64.64" class="ltx_tr">
<th id="S5.T3.64.64.64.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">DAIL-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>
</th>
<td id="S5.T3.58.58.58.1" class="ltx_td ltx_align_center"><math id="S5.T3.58.58.58.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.58.58.58.1.m1.1a"><mo id="S5.T3.58.58.58.1.m1.1.1" xref="S5.T3.58.58.58.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.58.58.58.1.m1.1b"><minus id="S5.T3.58.58.58.1.m1.1.1.cmml" xref="S5.T3.58.58.58.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.58.58.58.1.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.59.59.59.2" class="ltx_td ltx_align_center"><math id="S5.T3.59.59.59.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.59.59.59.2.m1.1a"><mo id="S5.T3.59.59.59.2.m1.1.1" xref="S5.T3.59.59.59.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.59.59.59.2.m1.1b"><minus id="S5.T3.59.59.59.2.m1.1.1.cmml" xref="S5.T3.59.59.59.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.59.59.59.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.60.60.60.3" class="ltx_td ltx_align_center"><math id="S5.T3.60.60.60.3.m1.1" class="ltx_Math" alttext="67.9" display="inline"><semantics id="S5.T3.60.60.60.3.m1.1a"><mn id="S5.T3.60.60.60.3.m1.1.1" xref="S5.T3.60.60.60.3.m1.1.1.cmml">67.9</mn><annotation-xml encoding="MathML-Content" id="S5.T3.60.60.60.3.m1.1b"><cn type="float" id="S5.T3.60.60.60.3.m1.1.1.cmml" xref="S5.T3.60.60.60.3.m1.1.1">67.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.60.60.60.3.m1.1c">67.9</annotation></semantics></math></td>
<td id="S5.T3.61.61.61.4" class="ltx_td ltx_align_center"><math id="S5.T3.61.61.61.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.61.61.61.4.m1.1a"><mo id="S5.T3.61.61.61.4.m1.1.1" xref="S5.T3.61.61.61.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.61.61.61.4.m1.1b"><minus id="S5.T3.61.61.61.4.m1.1.1.cmml" xref="S5.T3.61.61.61.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.61.61.61.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.62.62.62.5" class="ltx_td ltx_align_center"><math id="S5.T3.62.62.62.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.62.62.62.5.m1.1a"><mo id="S5.T3.62.62.62.5.m1.1.1" xref="S5.T3.62.62.62.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.62.62.62.5.m1.1b"><minus id="S5.T3.62.62.62.5.m1.1.1.cmml" xref="S5.T3.62.62.62.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.62.62.62.5.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.63.63.63.6" class="ltx_td ltx_align_center"><math id="S5.T3.63.63.63.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.63.63.63.6.m1.1a"><mo id="S5.T3.63.63.63.6.m1.1.1" xref="S5.T3.63.63.63.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.63.63.63.6.m1.1b"><minus id="S5.T3.63.63.63.6.m1.1.1.cmml" xref="S5.T3.63.63.63.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.63.63.63.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.64.64.64.7" class="ltx_td ltx_align_center"><math id="S5.T3.64.64.64.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.64.64.64.7.m1.1a"><mo id="S5.T3.64.64.64.7.m1.1.1" xref="S5.T3.64.64.64.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.64.64.64.7.m1.1b"><minus id="S5.T3.64.64.64.7.m1.1.1.cmml" xref="S5.T3.64.64.64.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.64.64.64.7.m1.1c">-</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.72.72.72" class="ltx_tr">
<th id="S5.T3.65.65.65.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ <math id="S5.T3.65.65.65.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T3.65.65.65.1.m1.1a"><msub id="S5.T3.65.65.65.1.m1.1.1" xref="S5.T3.65.65.65.1.m1.1.1.cmml"><mtext id="S5.T3.65.65.65.1.m1.1.1.2" xref="S5.T3.65.65.65.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T3.65.65.65.1.m1.1.1.3" xref="S5.T3.65.65.65.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T3.65.65.65.1.m1.1b"><apply id="S5.T3.65.65.65.1.m1.1.1.cmml" xref="S5.T3.65.65.65.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.65.65.65.1.m1.1.1.1.cmml" xref="S5.T3.65.65.65.1.m1.1.1">subscript</csymbol><ci id="S5.T3.65.65.65.1.m1.1.1.2a.cmml" xref="S5.T3.65.65.65.1.m1.1.1.2"><mtext id="S5.T3.65.65.65.1.m1.1.1.2.cmml" xref="S5.T3.65.65.65.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T3.65.65.65.1.m1.1.1.3a.cmml" xref="S5.T3.65.65.65.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T3.65.65.65.1.m1.1.1.3.cmml" xref="S5.T3.65.65.65.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.65.65.65.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="S5.T3.66.66.66.2" class="ltx_td ltx_align_center"><math id="S5.T3.66.66.66.2.m1.1" class="ltx_Math" alttext="68.8" display="inline"><semantics id="S5.T3.66.66.66.2.m1.1a"><mn id="S5.T3.66.66.66.2.m1.1.1" xref="S5.T3.66.66.66.2.m1.1.1.cmml">68.8</mn><annotation-xml encoding="MathML-Content" id="S5.T3.66.66.66.2.m1.1b"><cn type="float" id="S5.T3.66.66.66.2.m1.1.1.cmml" xref="S5.T3.66.66.66.2.m1.1.1">68.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.66.66.66.2.m1.1c">68.8</annotation></semantics></math></td>
<td id="S5.T3.67.67.67.3" class="ltx_td ltx_align_center"><math id="S5.T3.67.67.67.3.m1.1" class="ltx_Math" alttext="49.3" display="inline"><semantics id="S5.T3.67.67.67.3.m1.1a"><mn id="S5.T3.67.67.67.3.m1.1.1" xref="S5.T3.67.67.67.3.m1.1.1.cmml">49.3</mn><annotation-xml encoding="MathML-Content" id="S5.T3.67.67.67.3.m1.1b"><cn type="float" id="S5.T3.67.67.67.3.m1.1.1.cmml" xref="S5.T3.67.67.67.3.m1.1.1">49.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.67.67.67.3.m1.1c">49.3</annotation></semantics></math></td>
<td id="S5.T3.68.68.68.4" class="ltx_td ltx_align_center"><math id="S5.T3.68.68.68.4.m1.1" class="ltx_Math" alttext="78.0" display="inline"><semantics id="S5.T3.68.68.68.4.m1.1a"><mn id="S5.T3.68.68.68.4.m1.1.1" xref="S5.T3.68.68.68.4.m1.1.1.cmml">78.0</mn><annotation-xml encoding="MathML-Content" id="S5.T3.68.68.68.4.m1.1b"><cn type="float" id="S5.T3.68.68.68.4.m1.1.1.cmml" xref="S5.T3.68.68.68.4.m1.1.1">78.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.68.68.68.4.m1.1c">78.0</annotation></semantics></math></td>
<td id="S5.T3.69.69.69.5" class="ltx_td ltx_align_center"><math id="S5.T3.69.69.69.5.m1.1" class="ltx_Math" alttext="60.8" display="inline"><semantics id="S5.T3.69.69.69.5.m1.1a"><mn id="S5.T3.69.69.69.5.m1.1.1" xref="S5.T3.69.69.69.5.m1.1.1.cmml">60.8</mn><annotation-xml encoding="MathML-Content" id="S5.T3.69.69.69.5.m1.1b"><cn type="float" id="S5.T3.69.69.69.5.m1.1.1.cmml" xref="S5.T3.69.69.69.5.m1.1.1">60.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.69.69.69.5.m1.1c">60.8</annotation></semantics></math></td>
<td id="S5.T3.70.70.70.6" class="ltx_td ltx_align_center"><math id="S5.T3.70.70.70.6.m1.1" class="ltx_Math" alttext="66.9" display="inline"><semantics id="S5.T3.70.70.70.6.m1.1a"><mn id="S5.T3.70.70.70.6.m1.1.1" xref="S5.T3.70.70.70.6.m1.1.1.cmml">66.9</mn><annotation-xml encoding="MathML-Content" id="S5.T3.70.70.70.6.m1.1b"><cn type="float" id="S5.T3.70.70.70.6.m1.1.1.cmml" xref="S5.T3.70.70.70.6.m1.1.1">66.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.70.70.70.6.m1.1c">66.9</annotation></semantics></math></td>
<td id="S5.T3.71.71.71.7" class="ltx_td ltx_align_center"><math id="S5.T3.71.71.71.7.m1.1" class="ltx_Math" alttext="51.2" display="inline"><semantics id="S5.T3.71.71.71.7.m1.1a"><mn id="S5.T3.71.71.71.7.m1.1.1" xref="S5.T3.71.71.71.7.m1.1.1.cmml">51.2</mn><annotation-xml encoding="MathML-Content" id="S5.T3.71.71.71.7.m1.1b"><cn type="float" id="S5.T3.71.71.71.7.m1.1.1.cmml" xref="S5.T3.71.71.71.7.m1.1.1">51.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.71.71.71.7.m1.1c">51.2</annotation></semantics></math></td>
<td id="S5.T3.72.72.72.8" class="ltx_td ltx_align_center"><math id="S5.T3.72.72.72.8.m1.1" class="ltx_Math" alttext="54.0" display="inline"><semantics id="S5.T3.72.72.72.8.m1.1a"><mn id="S5.T3.72.72.72.8.m1.1.1" xref="S5.T3.72.72.72.8.m1.1.1.cmml">54.0</mn><annotation-xml encoding="MathML-Content" id="S5.T3.72.72.72.8.m1.1b"><cn type="float" id="S5.T3.72.72.72.8.m1.1.1.cmml" xref="S5.T3.72.72.72.8.m1.1.1">54.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.72.72.72.8.m1.1c">54.0</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.94.94.99.5" class="ltx_tr">
<th id="S5.T3.94.94.99.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="S5.T3.94.94.99.5.1.1" class="ltx_text ltx_font_typewriter">PaLM2</span></th>
<td id="S5.T3.94.94.99.5.2" class="ltx_td ltx_border_t"></td>
<td id="S5.T3.94.94.99.5.3" class="ltx_td ltx_border_t"></td>
<td id="S5.T3.94.94.99.5.4" class="ltx_td ltx_border_t"></td>
<td id="S5.T3.94.94.99.5.5" class="ltx_td ltx_border_t"></td>
<td id="S5.T3.94.94.99.5.6" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S5.T3.79.79.79" class="ltx_tr">
<th id="S5.T3.79.79.79.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">
<span id="S5.T3.79.79.79.8.1" class="ltx_text ltx_font_italic">Few-shot</span> SQL-PaLM <cite class="ltx_cite ltx_citemacro_cite">Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S5.T3.73.73.73.1" class="ltx_td ltx_align_center"><math id="S5.T3.73.73.73.1.m1.1" class="ltx_Math" alttext="66.5" display="inline"><semantics id="S5.T3.73.73.73.1.m1.1a"><mn id="S5.T3.73.73.73.1.m1.1.1" xref="S5.T3.73.73.73.1.m1.1.1.cmml">66.5</mn><annotation-xml encoding="MathML-Content" id="S5.T3.73.73.73.1.m1.1b"><cn type="float" id="S5.T3.73.73.73.1.m1.1.1.cmml" xref="S5.T3.73.73.73.1.m1.1.1">66.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.73.73.73.1.m1.1c">66.5</annotation></semantics></math></td>
<td id="S5.T3.74.74.74.2" class="ltx_td ltx_align_center"><math id="S5.T3.74.74.74.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.74.74.74.2.m1.1a"><mo id="S5.T3.74.74.74.2.m1.1.1" xref="S5.T3.74.74.74.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.74.74.74.2.m1.1b"><minus id="S5.T3.74.74.74.2.m1.1.1.cmml" xref="S5.T3.74.74.74.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.74.74.74.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.75.75.75.3" class="ltx_td ltx_align_center"><math id="S5.T3.75.75.75.3.m1.1" class="ltx_Math" alttext="77.6" display="inline"><semantics id="S5.T3.75.75.75.3.m1.1a"><mn id="S5.T3.75.75.75.3.m1.1.1" xref="S5.T3.75.75.75.3.m1.1.1.cmml">77.6</mn><annotation-xml encoding="MathML-Content" id="S5.T3.75.75.75.3.m1.1b"><cn type="float" id="S5.T3.75.75.75.3.m1.1.1.cmml" xref="S5.T3.75.75.75.3.m1.1.1">77.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.75.75.75.3.m1.1c">77.6</annotation></semantics></math></td>
<td id="S5.T3.76.76.76.4" class="ltx_td ltx_align_center"><math id="S5.T3.76.76.76.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.76.76.76.4.m1.1a"><mo id="S5.T3.76.76.76.4.m1.1.1" xref="S5.T3.76.76.76.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.76.76.76.4.m1.1b"><minus id="S5.T3.76.76.76.4.m1.1.1.cmml" xref="S5.T3.76.76.76.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.76.76.76.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.77.77.77.5" class="ltx_td ltx_align_center"><math id="S5.T3.77.77.77.5.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\underline{74.6}}" display="inline"><semantics id="S5.T3.77.77.77.5.m1.1a"><munder accentunder="true" id="S5.T3.77.77.77.5.m1.1.1" xref="S5.T3.77.77.77.5.m1.1.1.cmml"><mn mathcolor="#0000FF" id="S5.T3.77.77.77.5.m1.1.1.2" xref="S5.T3.77.77.77.5.m1.1.1.2.cmml">74.6</mn><mo mathcolor="#0000FF" id="S5.T3.77.77.77.5.m1.1.1.1" xref="S5.T3.77.77.77.5.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.77.77.77.5.m1.1b"><apply id="S5.T3.77.77.77.5.m1.1.1.cmml" xref="S5.T3.77.77.77.5.m1.1.1"><ci id="S5.T3.77.77.77.5.m1.1.1.1.cmml" xref="S5.T3.77.77.77.5.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.77.77.77.5.m1.1.1.2.cmml" xref="S5.T3.77.77.77.5.m1.1.1.2">74.6</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.77.77.77.5.m1.1c">{\color[rgb]{0,0,1}\underline{74.6}}</annotation></semantics></math></td>
<td id="S5.T3.78.78.78.6" class="ltx_td ltx_align_center"><math id="S5.T3.78.78.78.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.78.78.78.6.m1.1a"><mo id="S5.T3.78.78.78.6.m1.1.1" xref="S5.T3.78.78.78.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.78.78.78.6.m1.1b"><minus id="S5.T3.78.78.78.6.m1.1.1.cmml" xref="S5.T3.78.78.78.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.78.78.78.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.79.79.79.7" class="ltx_td ltx_align_center"><math id="S5.T3.79.79.79.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.79.79.79.7.m1.1a"><mo id="S5.T3.79.79.79.7.m1.1.1" xref="S5.T3.79.79.79.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.79.79.79.7.m1.1b"><minus id="S5.T3.79.79.79.7.m1.1.1.cmml" xref="S5.T3.79.79.79.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.79.79.79.7.m1.1c">-</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.94.94.100.6" class="ltx_tr">
<th id="S5.T3.94.94.100.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="7"><span id="S5.T3.94.94.100.6.1.1" class="ltx_text ltx_font_typewriter">gpt-4</span></th>
<td id="S5.T3.94.94.100.6.2" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="S5.T3.86.86.86" class="ltx_tr">
<th id="S5.T3.86.86.86.8" class="ltx_td ltx_align_left ltx_th ltx_th_row">DAIL-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>
</th>
<td id="S5.T3.80.80.80.1" class="ltx_td ltx_align_center"><math id="S5.T3.80.80.80.1.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.80.80.80.1.m1.1a"><mo id="S5.T3.80.80.80.1.m1.1.1" xref="S5.T3.80.80.80.1.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.80.80.80.1.m1.1b"><minus id="S5.T3.80.80.80.1.m1.1.1.cmml" xref="S5.T3.80.80.80.1.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.80.80.80.1.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.81.81.81.2" class="ltx_td ltx_align_center"><math id="S5.T3.81.81.81.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.81.81.81.2.m1.1a"><mo id="S5.T3.81.81.81.2.m1.1.1" xref="S5.T3.81.81.81.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.81.81.81.2.m1.1b"><minus id="S5.T3.81.81.81.2.m1.1.1.cmml" xref="S5.T3.81.81.81.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.81.81.81.2.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.82.82.82.3" class="ltx_td ltx_align_center"><math id="S5.T3.82.82.82.3.m1.1" class="ltx_Math" alttext="76.0" display="inline"><semantics id="S5.T3.82.82.82.3.m1.1a"><mn id="S5.T3.82.82.82.3.m1.1.1" xref="S5.T3.82.82.82.3.m1.1.1.cmml">76.0</mn><annotation-xml encoding="MathML-Content" id="S5.T3.82.82.82.3.m1.1b"><cn type="float" id="S5.T3.82.82.82.3.m1.1.1.cmml" xref="S5.T3.82.82.82.3.m1.1.1">76.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.82.82.82.3.m1.1c">76.0</annotation></semantics></math></td>
<td id="S5.T3.83.83.83.4" class="ltx_td ltx_align_center"><math id="S5.T3.83.83.83.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.83.83.83.4.m1.1a"><mo id="S5.T3.83.83.83.4.m1.1.1" xref="S5.T3.83.83.83.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.83.83.83.4.m1.1b"><minus id="S5.T3.83.83.83.4.m1.1.1.cmml" xref="S5.T3.83.83.83.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.83.83.83.4.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.84.84.84.5" class="ltx_td ltx_align_center"><math id="S5.T3.84.84.84.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.84.84.84.5.m1.1a"><mo id="S5.T3.84.84.84.5.m1.1.1" xref="S5.T3.84.84.84.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.84.84.84.5.m1.1b"><minus id="S5.T3.84.84.84.5.m1.1.1.cmml" xref="S5.T3.84.84.84.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.84.84.84.5.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.85.85.85.6" class="ltx_td ltx_align_center"><math id="S5.T3.85.85.85.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.85.85.85.6.m1.1a"><mo id="S5.T3.85.85.85.6.m1.1.1" xref="S5.T3.85.85.85.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.85.85.85.6.m1.1b"><minus id="S5.T3.85.85.85.6.m1.1.1.cmml" xref="S5.T3.85.85.85.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.85.85.85.6.m1.1c">-</annotation></semantics></math></td>
<td id="S5.T3.86.86.86.7" class="ltx_td ltx_align_center"><math id="S5.T3.86.86.86.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="S5.T3.86.86.86.7.m1.1a"><mo id="S5.T3.86.86.86.7.m1.1.1" xref="S5.T3.86.86.86.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="S5.T3.86.86.86.7.m1.1b"><minus id="S5.T3.86.86.86.7.m1.1.1.cmml" xref="S5.T3.86.86.86.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.86.86.86.7.m1.1c">-</annotation></semantics></math></td>
</tr>
<tr id="S5.T3.94.94.94" class="ltx_tr">
<th id="S5.T3.87.87.87.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">Ours (w/ <math id="S5.T3.87.87.87.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T3.87.87.87.1.m1.1a"><msub id="S5.T3.87.87.87.1.m1.1.1" xref="S5.T3.87.87.87.1.m1.1.1.cmml"><mtext id="S5.T3.87.87.87.1.m1.1.1.2" xref="S5.T3.87.87.87.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T3.87.87.87.1.m1.1.1.3" xref="S5.T3.87.87.87.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T3.87.87.87.1.m1.1b"><apply id="S5.T3.87.87.87.1.m1.1.1.cmml" xref="S5.T3.87.87.87.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T3.87.87.87.1.m1.1.1.1.cmml" xref="S5.T3.87.87.87.1.m1.1.1">subscript</csymbol><ci id="S5.T3.87.87.87.1.m1.1.1.2a.cmml" xref="S5.T3.87.87.87.1.m1.1.1.2"><mtext id="S5.T3.87.87.87.1.m1.1.1.2.cmml" xref="S5.T3.87.87.87.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T3.87.87.87.1.m1.1.1.3a.cmml" xref="S5.T3.87.87.87.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T3.87.87.87.1.m1.1.1.3.cmml" xref="S5.T3.87.87.87.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.87.87.87.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="S5.T3.88.88.88.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.88.88.88.2.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\mathbf{72.3}}" display="inline"><semantics id="S5.T3.88.88.88.2.m1.1a"><mn class="ltx_mathvariant_bold" mathcolor="#0000FF" mathvariant="bold" id="S5.T3.88.88.88.2.m1.1.1" xref="S5.T3.88.88.88.2.m1.1.1.cmml">72.3</mn><annotation-xml encoding="MathML-Content" id="S5.T3.88.88.88.2.m1.1b"><cn type="float" id="S5.T3.88.88.88.2.m1.1.1.cmml" xref="S5.T3.88.88.88.2.m1.1.1">72.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.88.88.88.2.m1.1c">{\color[rgb]{0,0,1}\mathbf{72.3}}</annotation></semantics></math></td>
<td id="S5.T3.89.89.89.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.89.89.89.3.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\mathbf{59.1}}" display="inline"><semantics id="S5.T3.89.89.89.3.m1.1a"><mn class="ltx_mathvariant_bold" mathcolor="#0000FF" mathvariant="bold" id="S5.T3.89.89.89.3.m1.1.1" xref="S5.T3.89.89.89.3.m1.1.1.cmml">59.1</mn><annotation-xml encoding="MathML-Content" id="S5.T3.89.89.89.3.m1.1b"><cn type="float" id="S5.T3.89.89.89.3.m1.1.1.cmml" xref="S5.T3.89.89.89.3.m1.1.1">59.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.89.89.89.3.m1.1c">{\color[rgb]{0,0,1}\mathbf{59.1}}</annotation></semantics></math></td>
<td id="S5.T3.90.90.90.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.90.90.90.4.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\underline{80.9}}" display="inline"><semantics id="S5.T3.90.90.90.4.m1.1a"><munder accentunder="true" id="S5.T3.90.90.90.4.m1.1.1" xref="S5.T3.90.90.90.4.m1.1.1.cmml"><mn mathcolor="#0000FF" id="S5.T3.90.90.90.4.m1.1.1.2" xref="S5.T3.90.90.90.4.m1.1.1.2.cmml">80.9</mn><mo mathcolor="#0000FF" id="S5.T3.90.90.90.4.m1.1.1.1" xref="S5.T3.90.90.90.4.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.90.90.90.4.m1.1b"><apply id="S5.T3.90.90.90.4.m1.1.1.cmml" xref="S5.T3.90.90.90.4.m1.1.1"><ci id="S5.T3.90.90.90.4.m1.1.1.1.cmml" xref="S5.T3.90.90.90.4.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.90.90.90.4.m1.1.1.2.cmml" xref="S5.T3.90.90.90.4.m1.1.1.2">80.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.90.90.90.4.m1.1c">{\color[rgb]{0,0,1}\underline{80.9}}</annotation></semantics></math></td>
<td id="S5.T3.91.91.91.5" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.91.91.91.5.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\underline{66.1}}" display="inline"><semantics id="S5.T3.91.91.91.5.m1.1a"><munder accentunder="true" id="S5.T3.91.91.91.5.m1.1.1" xref="S5.T3.91.91.91.5.m1.1.1.cmml"><mn mathcolor="#0000FF" id="S5.T3.91.91.91.5.m1.1.1.2" xref="S5.T3.91.91.91.5.m1.1.1.2.cmml">66.1</mn><mo mathcolor="#0000FF" id="S5.T3.91.91.91.5.m1.1.1.1" xref="S5.T3.91.91.91.5.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.91.91.91.5.m1.1b"><apply id="S5.T3.91.91.91.5.m1.1.1.cmml" xref="S5.T3.91.91.91.5.m1.1.1"><ci id="S5.T3.91.91.91.5.m1.1.1.1.cmml" xref="S5.T3.91.91.91.5.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.91.91.91.5.m1.1.1.2.cmml" xref="S5.T3.91.91.91.5.m1.1.1.2">66.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.91.91.91.5.m1.1c">{\color[rgb]{0,0,1}\underline{66.1}}</annotation></semantics></math></td>
<td id="S5.T3.92.92.92.6" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.92.92.92.6.m1.1" class="ltx_Math" alttext="74.4" display="inline"><semantics id="S5.T3.92.92.92.6.m1.1a"><mn id="S5.T3.92.92.92.6.m1.1.1" xref="S5.T3.92.92.92.6.m1.1.1.cmml">74.4</mn><annotation-xml encoding="MathML-Content" id="S5.T3.92.92.92.6.m1.1b"><cn type="float" id="S5.T3.92.92.92.6.m1.1.1.cmml" xref="S5.T3.92.92.92.6.m1.1.1">74.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.92.92.92.6.m1.1c">74.4</annotation></semantics></math></td>
<td id="S5.T3.93.93.93.7" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.93.93.93.7.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\underline{61.3}}" display="inline"><semantics id="S5.T3.93.93.93.7.m1.1a"><munder accentunder="true" id="S5.T3.93.93.93.7.m1.1.1" xref="S5.T3.93.93.93.7.m1.1.1.cmml"><mn mathcolor="#0000FF" id="S5.T3.93.93.93.7.m1.1.1.2" xref="S5.T3.93.93.93.7.m1.1.1.2.cmml">61.3</mn><mo mathcolor="#0000FF" id="S5.T3.93.93.93.7.m1.1.1.1" xref="S5.T3.93.93.93.7.m1.1.1.1.cmml">¯</mo></munder><annotation-xml encoding="MathML-Content" id="S5.T3.93.93.93.7.m1.1b"><apply id="S5.T3.93.93.93.7.m1.1.1.cmml" xref="S5.T3.93.93.93.7.m1.1.1"><ci id="S5.T3.93.93.93.7.m1.1.1.1.cmml" xref="S5.T3.93.93.93.7.m1.1.1.1">¯</ci><cn type="float" id="S5.T3.93.93.93.7.m1.1.1.2.cmml" xref="S5.T3.93.93.93.7.m1.1.1.2">61.3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.93.93.93.7.m1.1c">{\color[rgb]{0,0,1}\underline{61.3}}</annotation></semantics></math></td>
<td id="S5.T3.94.94.94.8" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T3.94.94.94.8.m1.1" class="ltx_Math" alttext="{\color[rgb]{0,0,1}\mathbf{64.4}}" display="inline"><semantics id="S5.T3.94.94.94.8.m1.1a"><mn class="ltx_mathvariant_bold" mathcolor="#0000FF" mathvariant="bold" id="S5.T3.94.94.94.8.m1.1.1" xref="S5.T3.94.94.94.8.m1.1.1.cmml">64.4</mn><annotation-xml encoding="MathML-Content" id="S5.T3.94.94.94.8.m1.1b"><cn type="float" id="S5.T3.94.94.94.8.m1.1.1.cmml" xref="S5.T3.94.94.94.8.m1.1.1">64.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T3.94.94.94.8.m1.1c">{\color[rgb]{0,0,1}\mathbf{64.4}}</annotation></semantics></math></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Table 3: </span>Results on S<span id="S5.T3.94.104.1" class="ltx_text ltx_font_smallcaps">pider</span>-DK, S<span id="S5.T3.94.105.2" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="S5.T3.94.106.3" class="ltx_text ltx_font_smallcaps">eal</span>, S<span id="S5.T3.94.107.4" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="S5.T3.94.108.5" class="ltx_text ltx_font_smallcaps">yn</span> and CS<span id="S5.T3.94.109.6" class="ltx_text ltx_font_smallcaps">pider</span>. Fine-tuning-based baselines are listed at the top. The <span id="S5.T3.94.110.7" class="ltx_text ltx_font_bold">best</span> model is in bold, <span id="S5.T3.94.111.8" class="ltx_text ltx_framed ltx_framed_underline">second best</span> is underlined, and the <span id="S5.T3.94.112.9" class="ltx_text" style="color:#0000FF;">best prompt-based setup</span> is in blue.
</figcaption>
</figure>
</figure>
</section>
</section>
<section id="S5.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.3 </span>Text-to-SQL Evaluation</h3>

<div id="S5.SS3.p1" class="ltx_para">
<p id="S5.SS3.p1.3" class="ltx_p">Table <a href="#S5.T2" title="Table 2 ‣ 5.1 Datasets ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> and <a href="#S5.T3" title="Table 3 ‣ Prompting-based ‣ 5.2 Baselines ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> summarise the results of our approach with <span id="S5.SS3.p1.3.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span>, <span id="S5.SS3.p1.3.2" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span> and <span id="S5.SS3.p1.3.3" class="ltx_text ltx_font_typewriter">gpt-4</span> against the baselines. Our approach, comprising a single-prompting round, surpasses other LLM-based solutions, that incorporate several prompting iterations, for LLMs of the same capacity. We note consistent improvements over DAIL-SQL, the closest work to ours, even when <math id="S5.SS3.p1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.SS3.p1.1.m1.1a"><msub id="S5.SS3.p1.1.m1.1.1" xref="S5.SS3.p1.1.m1.1.1.cmml"><mtext id="S5.SS3.p1.1.m1.1.1.2" xref="S5.SS3.p1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.SS3.p1.1.m1.1.1.3" xref="S5.SS3.p1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.SS3.p1.1.m1.1b"><apply id="S5.SS3.p1.1.m1.1.1.cmml" xref="S5.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS3.p1.1.m1.1.1.1.cmml" xref="S5.SS3.p1.1.m1.1.1">subscript</csymbol><ci id="S5.SS3.p1.1.m1.1.1.2a.cmml" xref="S5.SS3.p1.1.m1.1.1.2"><mtext id="S5.SS3.p1.1.m1.1.1.2.cmml" xref="S5.SS3.p1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.SS3.p1.1.m1.1.1.3a.cmml" xref="S5.SS3.p1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.SS3.p1.1.m1.1.1.3.cmml" xref="S5.SS3.p1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> is used as approximator (i.e. a model consisting of <math id="S5.SS3.p1.2.m2.1" class="ltx_Math" alttext="&lt;500" display="inline"><semantics id="S5.SS3.p1.2.m2.1a"><mrow id="S5.SS3.p1.2.m2.1.1" xref="S5.SS3.p1.2.m2.1.1.cmml"><mi id="S5.SS3.p1.2.m2.1.1.2" xref="S5.SS3.p1.2.m2.1.1.2.cmml"></mi><mo id="S5.SS3.p1.2.m2.1.1.1" xref="S5.SS3.p1.2.m2.1.1.1.cmml">&lt;</mo><mn id="S5.SS3.p1.2.m2.1.1.3" xref="S5.SS3.p1.2.m2.1.1.3.cmml">500</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.p1.2.m2.1b"><apply id="S5.SS3.p1.2.m2.1.1.cmml" xref="S5.SS3.p1.2.m2.1.1"><lt id="S5.SS3.p1.2.m2.1.1.1.cmml" xref="S5.SS3.p1.2.m2.1.1.1"></lt><csymbol cd="latexml" id="S5.SS3.p1.2.m2.1.1.2.cmml" xref="S5.SS3.p1.2.m2.1.1.2">absent</csymbol><cn type="integer" id="S5.SS3.p1.2.m2.1.1.3.cmml" xref="S5.SS3.p1.2.m2.1.1.3">500</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p1.2.m2.1c">&lt;500</annotation></semantics></math>M vs the <math id="S5.SS3.p1.3.m3.1" class="ltx_Math" alttext="\geq 3" display="inline"><semantics id="S5.SS3.p1.3.m3.1a"><mrow id="S5.SS3.p1.3.m3.1.1" xref="S5.SS3.p1.3.m3.1.1.cmml"><mi id="S5.SS3.p1.3.m3.1.1.2" xref="S5.SS3.p1.3.m3.1.1.2.cmml"></mi><mo id="S5.SS3.p1.3.m3.1.1.1" xref="S5.SS3.p1.3.m3.1.1.1.cmml">≥</mo><mn id="S5.SS3.p1.3.m3.1.1.3" xref="S5.SS3.p1.3.m3.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.p1.3.m3.1b"><apply id="S5.SS3.p1.3.m3.1.1.cmml" xref="S5.SS3.p1.3.m3.1.1"><geq id="S5.SS3.p1.3.m3.1.1.1.cmml" xref="S5.SS3.p1.3.m3.1.1.1"></geq><csymbol cd="latexml" id="S5.SS3.p1.3.m3.1.1.2.cmml" xref="S5.SS3.p1.3.m3.1.1.2">absent</csymbol><cn type="integer" id="S5.SS3.p1.3.m3.1.1.3.cmml" xref="S5.SS3.p1.3.m3.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p1.3.m3.1c">\geq 3</annotation></semantics></math>B parameters that DAIL-SQL’s approximator is using). For the same approximator, our framework is able to meet, performance standards of DAIL-SQL (equipped with <span id="S5.SS3.p1.3.4" class="ltx_text ltx_font_typewriter">gpt-4</span> and an additional self-consistency prompting step) using an open-source model as backbone LLM, by achieving shorter prompts in a single prompting step.</p>
</div>
<div id="S5.SS3.p2" class="ltx_para">
<p id="S5.SS3.p2.1" class="ltx_p">S<span id="S5.SS3.p2.1.1" class="ltx_text ltx_font_smallcaps">pider</span> results are consistent with the results across the various Spider variants and CS<span id="S5.SS3.p2.1.2" class="ltx_text ltx_font_smallcaps">pider</span> <span id="footnote6" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">6</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">6</sup><span class="ltx_tag ltx_tag_note">6</span>In CS<span id="footnote6.1" class="ltx_text ltx_font_smallcaps">pider</span>, questions are fully translated in Chinese while the DB content remains in English. Due to this limitation, DB schema and content selection are disabled.</span></span></span> (Table <a href="#S5.T3" title="Table 3 ‣ Prompting-based ‣ 5.2 Baselines ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>).
Our approach levering <math id="S5.SS3.p2.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.SS3.p2.1.m1.1a"><msub id="S5.SS3.p2.1.m1.1.1" xref="S5.SS3.p2.1.m1.1.1.cmml"><mtext id="S5.SS3.p2.1.m1.1.1.2" xref="S5.SS3.p2.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.SS3.p2.1.m1.1.1.3" xref="S5.SS3.p2.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.SS3.p2.1.m1.1b"><apply id="S5.SS3.p2.1.m1.1.1.cmml" xref="S5.SS3.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS3.p2.1.m1.1.1.1.cmml" xref="S5.SS3.p2.1.m1.1.1">subscript</csymbol><ci id="S5.SS3.p2.1.m1.1.1.2a.cmml" xref="S5.SS3.p2.1.m1.1.1.2"><mtext id="S5.SS3.p2.1.m1.1.1.2.cmml" xref="S5.SS3.p2.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.SS3.p2.1.m1.1.1.3a.cmml" xref="S5.SS3.p2.1.m1.1.1.3"><mtext mathsize="70%" id="S5.SS3.p2.1.m1.1.1.3.cmml" xref="S5.SS3.p2.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p2.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> and AST-based re-ranking for example selection out-performs other prompting-based solution, and is in-line with the scores of state-of-the-art fine-tuning-based baselines. While <span id="S5.SS3.p2.1.3" class="ltx_text ltx_font_typewriter">gpt-4</span> is the most capable model within our framework (with this being more noticeable in the case of S<span id="S5.SS3.p2.1.4" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="S5.SS3.p2.1.5" class="ltx_text ltx_font_smallcaps">yn</span>), we observe surprising findings with DeepSeek with which in many cases our approach can surpass much more computationally expensive alternatives based on larger closed-source LLMs. Our findings remain consistent with <cite class="ltx_cite ltx_citemacro_cite">Liu et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite> since the EM scores of prompting-based methods fall behind those of their fine-tuning based counterparts.</p>
</div>
<section id="S5.SS3.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.3.1 </span>Schema Selection Evaluation</h4>

<figure id="S5.T4" class="ltx_table">
<table id="S5.T4.59" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S5.T4.59.60.1" class="ltx_tr">
<th id="S5.T4.59.60.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt"><span id="S5.T4.59.60.1.1.1" class="ltx_text ltx_font_bold">Approximator</span></th>
<th id="S5.T4.59.60.1.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt"><span id="S5.T4.59.60.1.2.1" class="ltx_text ltx_font_bold">Schema Selection Setup</span></th>
<td id="S5.T4.59.60.1.3" class="ltx_td ltx_align_center ltx_border_tt"><span id="S5.T4.59.60.1.3.1" class="ltx_text ltx_font_bold">Recall</span></td>
<td id="S5.T4.59.60.1.4" class="ltx_td ltx_align_center ltx_border_tt"><span id="S5.T4.59.60.1.4.1" class="ltx_text ltx_font_bold">Schema Shorten.</span></td>
<td id="S5.T4.59.60.1.5" class="ltx_td ltx_align_center ltx_border_tt"><span id="S5.T4.59.60.1.5.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="S5.T4.59.60.1.6" class="ltx_td ltx_align_center ltx_border_tt"><span id="S5.T4.59.60.1.6.1" class="ltx_text ltx_font_bold">EM</span></td>
</tr>
<tr id="S5.T4.4.4" class="ltx_tr">
<th id="S5.T4.4.4.5" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><span id="S5.T4.4.4.5.1" class="ltx_text ltx_font_italic">Oracle</span></th>
<th id="S5.T4.4.4.6" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Gold Query</th>
<td id="S5.T4.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.1.1.1.m1.1" class="ltx_Math" alttext="100.0" display="inline"><semantics id="S5.T4.1.1.1.m1.1a"><mn id="S5.T4.1.1.1.m1.1.1" xref="S5.T4.1.1.1.m1.1.1.cmml">100.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.1.1.1.m1.1b"><cn type="float" id="S5.T4.1.1.1.m1.1.1.cmml" xref="S5.T4.1.1.1.m1.1.1">100.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.1.1.1.m1.1c">100.0</annotation></semantics></math></td>
<td id="S5.T4.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.2.2.2.m1.1" class="ltx_Math" alttext="71.3" display="inline"><semantics id="S5.T4.2.2.2.m1.1a"><mn id="S5.T4.2.2.2.m1.1.1" xref="S5.T4.2.2.2.m1.1.1.cmml">71.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.2.2.2.m1.1b"><cn type="float" id="S5.T4.2.2.2.m1.1.1.cmml" xref="S5.T4.2.2.2.m1.1.1">71.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.2.2.2.m1.1c">71.3</annotation></semantics></math></td>
<td id="S5.T4.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.3.3.3.m1.1" class="ltx_Math" alttext="86.3" display="inline"><semantics id="S5.T4.3.3.3.m1.1a"><mn id="S5.T4.3.3.3.m1.1.1" xref="S5.T4.3.3.3.m1.1.1.cmml">86.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.3.3.3.m1.1b"><cn type="float" id="S5.T4.3.3.3.m1.1.1.cmml" xref="S5.T4.3.3.3.m1.1.1">86.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.3.3.3.m1.1c">86.3</annotation></semantics></math></td>
<td id="S5.T4.4.4.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.4.4.4.m1.1" class="ltx_Math" alttext="73.9" display="inline"><semantics id="S5.T4.4.4.4.m1.1a"><mn id="S5.T4.4.4.4.m1.1.1" xref="S5.T4.4.4.4.m1.1.1.cmml">73.9</mn><annotation-xml encoding="MathML-Content" id="S5.T4.4.4.4.m1.1b"><cn type="float" id="S5.T4.4.4.4.m1.1.1.cmml" xref="S5.T4.4.4.4.m1.1.1">73.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.4.4.4.m1.1c">73.9</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.10.10" class="ltx_tr">
<th id="S5.T4.5.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><math id="S5.T4.5.5.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.5.5.1.m1.1a"><msub id="S5.T4.5.5.1.m1.1.1" xref="S5.T4.5.5.1.m1.1.1.cmml"><mtext id="S5.T4.5.5.1.m1.1.1.2" xref="S5.T4.5.5.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.5.5.1.m1.1.1.3" xref="S5.T4.5.5.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.5.5.1.m1.1b"><apply id="S5.T4.5.5.1.m1.1.1.cmml" xref="S5.T4.5.5.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.5.5.1.m1.1.1.1.cmml" xref="S5.T4.5.5.1.m1.1.1">subscript</csymbol><ci id="S5.T4.5.5.1.m1.1.1.2a.cmml" xref="S5.T4.5.5.1.m1.1.1.2"><mtext id="S5.T4.5.5.1.m1.1.1.2.cmml" xref="S5.T4.5.5.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.5.5.1.m1.1.1.3a.cmml" xref="S5.T4.5.5.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.5.5.1.m1.1.1.3.cmml" xref="S5.T4.5.5.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.5.5.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.6.6.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">N<math id="S5.T4.6.6.2.m1.1" class="ltx_Math" alttext="/" display="inline"><semantics id="S5.T4.6.6.2.m1.1a"><mo id="S5.T4.6.6.2.m1.1.1" xref="S5.T4.6.6.2.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="S5.T4.6.6.2.m1.1b"><divide id="S5.T4.6.6.2.m1.1.1.cmml" xref="S5.T4.6.6.2.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.6.6.2.m1.1c">/</annotation></semantics></math>A</th>
<td id="S5.T4.7.7.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.7.7.3.m1.1" class="ltx_Math" alttext="100.0" display="inline"><semantics id="S5.T4.7.7.3.m1.1a"><mn id="S5.T4.7.7.3.m1.1.1" xref="S5.T4.7.7.3.m1.1.1.cmml">100.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.7.7.3.m1.1b"><cn type="float" id="S5.T4.7.7.3.m1.1.1.cmml" xref="S5.T4.7.7.3.m1.1.1">100.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.7.7.3.m1.1c">100.0</annotation></semantics></math></td>
<td id="S5.T4.8.8.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.8.8.4.m1.1" class="ltx_Math" alttext="0.0" display="inline"><semantics id="S5.T4.8.8.4.m1.1a"><mn id="S5.T4.8.8.4.m1.1.1" xref="S5.T4.8.8.4.m1.1.1.cmml">0.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.8.8.4.m1.1b"><cn type="float" id="S5.T4.8.8.4.m1.1.1.cmml" xref="S5.T4.8.8.4.m1.1.1">0.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.8.8.4.m1.1c">0.0</annotation></semantics></math></td>
<td id="S5.T4.9.9.5" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.9.9.5.m1.1" class="ltx_Math" alttext="79.3" display="inline"><semantics id="S5.T4.9.9.5.m1.1a"><mn id="S5.T4.9.9.5.m1.1.1" xref="S5.T4.9.9.5.m1.1.1.cmml">79.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.9.9.5.m1.1b"><cn type="float" id="S5.T4.9.9.5.m1.1.1.cmml" xref="S5.T4.9.9.5.m1.1.1">79.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.9.9.5.m1.1c">79.3</annotation></semantics></math></td>
<td id="S5.T4.10.10.6" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.10.10.6.m1.1" class="ltx_Math" alttext="63.6" display="inline"><semantics id="S5.T4.10.10.6.m1.1a"><mn id="S5.T4.10.10.6.m1.1.1" xref="S5.T4.10.10.6.m1.1.1.cmml">63.6</mn><annotation-xml encoding="MathML-Content" id="S5.T4.10.10.6.m1.1b"><cn type="float" id="S5.T4.10.10.6.m1.1.1.cmml" xref="S5.T4.10.10.6.m1.1.1">63.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.10.10.6.m1.1c">63.6</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.16.16" class="ltx_tr">
<th id="S5.T4.11.11.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T4.11.11.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.11.11.1.m1.1a"><msub id="S5.T4.11.11.1.m1.1.1" xref="S5.T4.11.11.1.m1.1.1.cmml"><mtext id="S5.T4.11.11.1.m1.1.1.2" xref="S5.T4.11.11.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.11.11.1.m1.1.1.3" xref="S5.T4.11.11.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.11.11.1.m1.1b"><apply id="S5.T4.11.11.1.m1.1.1.cmml" xref="S5.T4.11.11.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.11.11.1.m1.1.1.1.cmml" xref="S5.T4.11.11.1.m1.1.1">subscript</csymbol><ci id="S5.T4.11.11.1.m1.1.1.2a.cmml" xref="S5.T4.11.11.1.m1.1.1.2"><mtext id="S5.T4.11.11.1.m1.1.1.2.cmml" xref="S5.T4.11.11.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.11.11.1.m1.1.1.3a.cmml" xref="S5.T4.11.11.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.11.11.1.m1.1.1.3.cmml" xref="S5.T4.11.11.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.11.11.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.12.12.2" class="ltx_td ltx_align_left ltx_th ltx_th_row">BM25 (top-<math id="S5.T4.12.12.2.m1.1" class="ltx_Math" alttext="10" display="inline"><semantics id="S5.T4.12.12.2.m1.1a"><mn id="S5.T4.12.12.2.m1.1.1" xref="S5.T4.12.12.2.m1.1.1.cmml">10</mn><annotation-xml encoding="MathML-Content" id="S5.T4.12.12.2.m1.1b"><cn type="integer" id="S5.T4.12.12.2.m1.1.1.cmml" xref="S5.T4.12.12.2.m1.1.1">10</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.12.12.2.m1.1c">10</annotation></semantics></math>)</th>
<td id="S5.T4.13.13.3" class="ltx_td ltx_align_center"><math id="S5.T4.13.13.3.m1.1" class="ltx_Math" alttext="92.0" display="inline"><semantics id="S5.T4.13.13.3.m1.1a"><mn id="S5.T4.13.13.3.m1.1.1" xref="S5.T4.13.13.3.m1.1.1.cmml">92.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.13.13.3.m1.1b"><cn type="float" id="S5.T4.13.13.3.m1.1.1.cmml" xref="S5.T4.13.13.3.m1.1.1">92.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.13.13.3.m1.1c">92.0</annotation></semantics></math></td>
<td id="S5.T4.14.14.4" class="ltx_td ltx_align_center"><math id="S5.T4.14.14.4.m1.1" class="ltx_Math" alttext="36.5" display="inline"><semantics id="S5.T4.14.14.4.m1.1a"><mn id="S5.T4.14.14.4.m1.1.1" xref="S5.T4.14.14.4.m1.1.1.cmml">36.5</mn><annotation-xml encoding="MathML-Content" id="S5.T4.14.14.4.m1.1b"><cn type="float" id="S5.T4.14.14.4.m1.1.1.cmml" xref="S5.T4.14.14.4.m1.1.1">36.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.14.14.4.m1.1c">36.5</annotation></semantics></math></td>
<td id="S5.T4.15.15.5" class="ltx_td ltx_align_center"><math id="S5.T4.15.15.5.m1.1" class="ltx_Math" alttext="78.9" display="inline"><semantics id="S5.T4.15.15.5.m1.1a"><mn id="S5.T4.15.15.5.m1.1.1" xref="S5.T4.15.15.5.m1.1.1.cmml">78.9</mn><annotation-xml encoding="MathML-Content" id="S5.T4.15.15.5.m1.1b"><cn type="float" id="S5.T4.15.15.5.m1.1.1.cmml" xref="S5.T4.15.15.5.m1.1.1">78.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.15.15.5.m1.1c">78.9</annotation></semantics></math></td>
<td id="S5.T4.16.16.6" class="ltx_td ltx_align_center"><math id="S5.T4.16.16.6.m1.1" class="ltx_Math" alttext="64.1" display="inline"><semantics id="S5.T4.16.16.6.m1.1a"><mn id="S5.T4.16.16.6.m1.1.1" xref="S5.T4.16.16.6.m1.1.1.cmml">64.1</mn><annotation-xml encoding="MathML-Content" id="S5.T4.16.16.6.m1.1b"><cn type="float" id="S5.T4.16.16.6.m1.1.1.cmml" xref="S5.T4.16.16.6.m1.1.1">64.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.16.16.6.m1.1c">64.1</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.22.22" class="ltx_tr">
<th id="S5.T4.17.17.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T4.17.17.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.17.17.1.m1.1a"><msub id="S5.T4.17.17.1.m1.1.1" xref="S5.T4.17.17.1.m1.1.1.cmml"><mtext id="S5.T4.17.17.1.m1.1.1.2" xref="S5.T4.17.17.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.17.17.1.m1.1.1.3" xref="S5.T4.17.17.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.17.17.1.m1.1b"><apply id="S5.T4.17.17.1.m1.1.1.cmml" xref="S5.T4.17.17.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.17.17.1.m1.1.1.1.cmml" xref="S5.T4.17.17.1.m1.1.1">subscript</csymbol><ci id="S5.T4.17.17.1.m1.1.1.2a.cmml" xref="S5.T4.17.17.1.m1.1.1.2"><mtext id="S5.T4.17.17.1.m1.1.1.2.cmml" xref="S5.T4.17.17.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.17.17.1.m1.1.1.3a.cmml" xref="S5.T4.17.17.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.17.17.1.m1.1.1.3.cmml" xref="S5.T4.17.17.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.17.17.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.18.18.2" class="ltx_td ltx_align_left ltx_th ltx_th_row">BM25 (top-<math id="S5.T4.18.18.2.m1.1" class="ltx_Math" alttext="20" display="inline"><semantics id="S5.T4.18.18.2.m1.1a"><mn id="S5.T4.18.18.2.m1.1.1" xref="S5.T4.18.18.2.m1.1.1.cmml">20</mn><annotation-xml encoding="MathML-Content" id="S5.T4.18.18.2.m1.1b"><cn type="integer" id="S5.T4.18.18.2.m1.1.1.cmml" xref="S5.T4.18.18.2.m1.1.1">20</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.18.18.2.m1.1c">20</annotation></semantics></math>)</th>
<td id="S5.T4.19.19.3" class="ltx_td ltx_align_center"><math id="S5.T4.19.19.3.m1.1" class="ltx_Math" alttext="98.3" display="inline"><semantics id="S5.T4.19.19.3.m1.1a"><mn id="S5.T4.19.19.3.m1.1.1" xref="S5.T4.19.19.3.m1.1.1.cmml">98.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.19.19.3.m1.1b"><cn type="float" id="S5.T4.19.19.3.m1.1.1.cmml" xref="S5.T4.19.19.3.m1.1.1">98.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.19.19.3.m1.1c">98.3</annotation></semantics></math></td>
<td id="S5.T4.20.20.4" class="ltx_td ltx_align_center"><math id="S5.T4.20.20.4.m1.1" class="ltx_Math" alttext="14.1" display="inline"><semantics id="S5.T4.20.20.4.m1.1a"><mn id="S5.T4.20.20.4.m1.1.1" xref="S5.T4.20.20.4.m1.1.1.cmml">14.1</mn><annotation-xml encoding="MathML-Content" id="S5.T4.20.20.4.m1.1b"><cn type="float" id="S5.T4.20.20.4.m1.1.1.cmml" xref="S5.T4.20.20.4.m1.1.1">14.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.20.20.4.m1.1c">14.1</annotation></semantics></math></td>
<td id="S5.T4.21.21.5" class="ltx_td ltx_align_center"><math id="S5.T4.21.21.5.m1.1" class="ltx_Math" alttext="80.7" display="inline"><semantics id="S5.T4.21.21.5.m1.1a"><mn id="S5.T4.21.21.5.m1.1.1" xref="S5.T4.21.21.5.m1.1.1.cmml">80.7</mn><annotation-xml encoding="MathML-Content" id="S5.T4.21.21.5.m1.1b"><cn type="float" id="S5.T4.21.21.5.m1.1.1.cmml" xref="S5.T4.21.21.5.m1.1.1">80.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.21.21.5.m1.1c">80.7</annotation></semantics></math></td>
<td id="S5.T4.22.22.6" class="ltx_td ltx_align_center"><math id="S5.T4.22.22.6.m1.1" class="ltx_Math" alttext="64.9" display="inline"><semantics id="S5.T4.22.22.6.m1.1a"><mn id="S5.T4.22.22.6.m1.1.1" xref="S5.T4.22.22.6.m1.1.1.cmml">64.9</mn><annotation-xml encoding="MathML-Content" id="S5.T4.22.22.6.m1.1b"><cn type="float" id="S5.T4.22.22.6.m1.1.1.cmml" xref="S5.T4.22.22.6.m1.1.1">64.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.22.22.6.m1.1c">64.9</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.27.27" class="ltx_tr">
<th id="S5.T4.23.23.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T4.23.23.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.23.23.1.m1.1a"><msub id="S5.T4.23.23.1.m1.1.1" xref="S5.T4.23.23.1.m1.1.1.cmml"><mtext id="S5.T4.23.23.1.m1.1.1.2" xref="S5.T4.23.23.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.23.23.1.m1.1.1.3" xref="S5.T4.23.23.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.23.23.1.m1.1b"><apply id="S5.T4.23.23.1.m1.1.1.cmml" xref="S5.T4.23.23.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.23.23.1.m1.1.1.1.cmml" xref="S5.T4.23.23.1.m1.1.1">subscript</csymbol><ci id="S5.T4.23.23.1.m1.1.1.2a.cmml" xref="S5.T4.23.23.1.m1.1.1.2"><mtext id="S5.T4.23.23.1.m1.1.1.2.cmml" xref="S5.T4.23.23.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.23.23.1.m1.1.1.3a.cmml" xref="S5.T4.23.23.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.23.23.1.m1.1.1.3.cmml" xref="S5.T4.23.23.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.23.23.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.27.27.6" class="ltx_td ltx_align_left ltx_th ltx_th_row">Approx. Query</th>
<td id="S5.T4.24.24.2" class="ltx_td ltx_align_center"><math id="S5.T4.24.24.2.m1.1" class="ltx_Math" alttext="86.8" display="inline"><semantics id="S5.T4.24.24.2.m1.1a"><mn id="S5.T4.24.24.2.m1.1.1" xref="S5.T4.24.24.2.m1.1.1.cmml">86.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.24.24.2.m1.1b"><cn type="float" id="S5.T4.24.24.2.m1.1.1.cmml" xref="S5.T4.24.24.2.m1.1.1">86.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.24.24.2.m1.1c">86.8</annotation></semantics></math></td>
<td id="S5.T4.25.25.3" class="ltx_td ltx_align_center"><math id="S5.T4.25.25.3.m1.1" class="ltx_Math" alttext="71.3" display="inline"><semantics id="S5.T4.25.25.3.m1.1a"><mn id="S5.T4.25.25.3.m1.1.1" xref="S5.T4.25.25.3.m1.1.1.cmml">71.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.25.25.3.m1.1b"><cn type="float" id="S5.T4.25.25.3.m1.1.1.cmml" xref="S5.T4.25.25.3.m1.1.1">71.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.25.25.3.m1.1c">71.3</annotation></semantics></math></td>
<td id="S5.T4.26.26.4" class="ltx_td ltx_align_center"><math id="S5.T4.26.26.4.m1.1" class="ltx_Math" alttext="78.4" display="inline"><semantics id="S5.T4.26.26.4.m1.1a"><mn id="S5.T4.26.26.4.m1.1.1" xref="S5.T4.26.26.4.m1.1.1.cmml">78.4</mn><annotation-xml encoding="MathML-Content" id="S5.T4.26.26.4.m1.1b"><cn type="float" id="S5.T4.26.26.4.m1.1.1.cmml" xref="S5.T4.26.26.4.m1.1.1">78.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.26.26.4.m1.1c">78.4</annotation></semantics></math></td>
<td id="S5.T4.27.27.5" class="ltx_td ltx_align_center"><math id="S5.T4.27.27.5.m1.1" class="ltx_Math" alttext="63.8" display="inline"><semantics id="S5.T4.27.27.5.m1.1a"><mn id="S5.T4.27.27.5.m1.1.1" xref="S5.T4.27.27.5.m1.1.1.cmml">63.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.27.27.5.m1.1b"><cn type="float" id="S5.T4.27.27.5.m1.1.1.cmml" xref="S5.T4.27.27.5.m1.1.1">63.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.27.27.5.m1.1c">63.8</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.33.33" class="ltx_tr">
<th id="S5.T4.28.28.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T4.28.28.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.28.28.1.m1.1a"><msub id="S5.T4.28.28.1.m1.1.1" xref="S5.T4.28.28.1.m1.1.1.cmml"><mtext id="S5.T4.28.28.1.m1.1.1.2" xref="S5.T4.28.28.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.28.28.1.m1.1.1.3" xref="S5.T4.28.28.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.28.28.1.m1.1b"><apply id="S5.T4.28.28.1.m1.1.1.cmml" xref="S5.T4.28.28.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.28.28.1.m1.1.1.1.cmml" xref="S5.T4.28.28.1.m1.1.1">subscript</csymbol><ci id="S5.T4.28.28.1.m1.1.1.2a.cmml" xref="S5.T4.28.28.1.m1.1.1.2"><mtext id="S5.T4.28.28.1.m1.1.1.2.cmml" xref="S5.T4.28.28.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.28.28.1.m1.1.1.3a.cmml" xref="S5.T4.28.28.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.28.28.1.m1.1.1.3.cmml" xref="S5.T4.28.28.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.28.28.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.29.29.2" class="ltx_td ltx_align_left ltx_th ltx_th_row">Approx. Query + BM25 (top-<math id="S5.T4.29.29.2.m1.1" class="ltx_Math" alttext="7" display="inline"><semantics id="S5.T4.29.29.2.m1.1a"><mn id="S5.T4.29.29.2.m1.1.1" xref="S5.T4.29.29.2.m1.1.1.cmml">7</mn><annotation-xml encoding="MathML-Content" id="S5.T4.29.29.2.m1.1b"><cn type="integer" id="S5.T4.29.29.2.m1.1.1.cmml" xref="S5.T4.29.29.2.m1.1.1">7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.29.29.2.m1.1c">7</annotation></semantics></math>)</th>
<td id="S5.T4.30.30.3" class="ltx_td ltx_align_center"><math id="S5.T4.30.30.3.m1.1" class="ltx_Math" alttext="93.3" display="inline"><semantics id="S5.T4.30.30.3.m1.1a"><mn id="S5.T4.30.30.3.m1.1.1" xref="S5.T4.30.30.3.m1.1.1.cmml">93.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.30.30.3.m1.1b"><cn type="float" id="S5.T4.30.30.3.m1.1.1.cmml" xref="S5.T4.30.30.3.m1.1.1">93.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.30.30.3.m1.1c">93.3</annotation></semantics></math></td>
<td id="S5.T4.31.31.4" class="ltx_td ltx_align_center"><math id="S5.T4.31.31.4.m1.1" class="ltx_Math" alttext="50.4" display="inline"><semantics id="S5.T4.31.31.4.m1.1a"><mn id="S5.T4.31.31.4.m1.1.1" xref="S5.T4.31.31.4.m1.1.1.cmml">50.4</mn><annotation-xml encoding="MathML-Content" id="S5.T4.31.31.4.m1.1b"><cn type="float" id="S5.T4.31.31.4.m1.1.1.cmml" xref="S5.T4.31.31.4.m1.1.1">50.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.31.31.4.m1.1c">50.4</annotation></semantics></math></td>
<td id="S5.T4.32.32.5" class="ltx_td ltx_align_center"><math id="S5.T4.32.32.5.m1.1" class="ltx_Math" alttext="81.1" display="inline"><semantics id="S5.T4.32.32.5.m1.1a"><mn id="S5.T4.32.32.5.m1.1.1" xref="S5.T4.32.32.5.m1.1.1.cmml">81.1</mn><annotation-xml encoding="MathML-Content" id="S5.T4.32.32.5.m1.1b"><cn type="float" id="S5.T4.32.32.5.m1.1.1.cmml" xref="S5.T4.32.32.5.m1.1.1">81.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.32.32.5.m1.1c">81.1</annotation></semantics></math></td>
<td id="S5.T4.33.33.6" class="ltx_td ltx_align_center"><math id="S5.T4.33.33.6.m1.1" class="ltx_Math" alttext="65.4" display="inline"><semantics id="S5.T4.33.33.6.m1.1a"><mn id="S5.T4.33.33.6.m1.1.1" xref="S5.T4.33.33.6.m1.1.1.cmml">65.4</mn><annotation-xml encoding="MathML-Content" id="S5.T4.33.33.6.m1.1b"><cn type="float" id="S5.T4.33.33.6.m1.1.1.cmml" xref="S5.T4.33.33.6.m1.1.1">65.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.33.33.6.m1.1c">65.4</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.39.39" class="ltx_tr">
<th id="S5.T4.34.34.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T4.34.34.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.34.34.1.m1.1a"><msub id="S5.T4.34.34.1.m1.1.1" xref="S5.T4.34.34.1.m1.1.1.cmml"><mtext id="S5.T4.34.34.1.m1.1.1.2" xref="S5.T4.34.34.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.34.34.1.m1.1.1.3" xref="S5.T4.34.34.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.34.34.1.m1.1b"><apply id="S5.T4.34.34.1.m1.1.1.cmml" xref="S5.T4.34.34.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.34.34.1.m1.1.1.1.cmml" xref="S5.T4.34.34.1.m1.1.1">subscript</csymbol><ci id="S5.T4.34.34.1.m1.1.1.2a.cmml" xref="S5.T4.34.34.1.m1.1.1.2"><mtext id="S5.T4.34.34.1.m1.1.1.2.cmml" xref="S5.T4.34.34.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.34.34.1.m1.1.1.3a.cmml" xref="S5.T4.34.34.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.34.34.1.m1.1.1.3.cmml" xref="S5.T4.34.34.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.34.34.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.35.35.2" class="ltx_td ltx_align_left ltx_th ltx_th_row">Approx. Query + BM25 (top-<math id="S5.T4.35.35.2.m1.1" class="ltx_Math" alttext="10" display="inline"><semantics id="S5.T4.35.35.2.m1.1a"><mn id="S5.T4.35.35.2.m1.1.1" xref="S5.T4.35.35.2.m1.1.1.cmml">10</mn><annotation-xml encoding="MathML-Content" id="S5.T4.35.35.2.m1.1b"><cn type="integer" id="S5.T4.35.35.2.m1.1.1.cmml" xref="S5.T4.35.35.2.m1.1.1">10</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.35.35.2.m1.1c">10</annotation></semantics></math>)</th>
<td id="S5.T4.36.36.3" class="ltx_td ltx_align_center"><math id="S5.T4.36.36.3.m1.1" class="ltx_Math" alttext="97.0" display="inline"><semantics id="S5.T4.36.36.3.m1.1a"><mn id="S5.T4.36.36.3.m1.1.1" xref="S5.T4.36.36.3.m1.1.1.cmml">97.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.36.36.3.m1.1b"><cn type="float" id="S5.T4.36.36.3.m1.1.1.cmml" xref="S5.T4.36.36.3.m1.1.1">97.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.36.36.3.m1.1c">97.0</annotation></semantics></math></td>
<td id="S5.T4.37.37.4" class="ltx_td ltx_align_center"><math id="S5.T4.37.37.4.m1.1" class="ltx_Math" alttext="37.3" display="inline"><semantics id="S5.T4.37.37.4.m1.1a"><mn id="S5.T4.37.37.4.m1.1.1" xref="S5.T4.37.37.4.m1.1.1.cmml">37.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.37.37.4.m1.1b"><cn type="float" id="S5.T4.37.37.4.m1.1.1.cmml" xref="S5.T4.37.37.4.m1.1.1">37.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.37.37.4.m1.1c">37.3</annotation></semantics></math></td>
<td id="S5.T4.38.38.5" class="ltx_td ltx_align_center"><math id="S5.T4.38.38.5.m1.1" class="ltx_Math" alttext="81.2" display="inline"><semantics id="S5.T4.38.38.5.m1.1a"><mn id="S5.T4.38.38.5.m1.1.1" xref="S5.T4.38.38.5.m1.1.1.cmml">81.2</mn><annotation-xml encoding="MathML-Content" id="S5.T4.38.38.5.m1.1b"><cn type="float" id="S5.T4.38.38.5.m1.1.1.cmml" xref="S5.T4.38.38.5.m1.1.1">81.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.38.38.5.m1.1c">81.2</annotation></semantics></math></td>
<td id="S5.T4.39.39.6" class="ltx_td ltx_align_center"><math id="S5.T4.39.39.6.m1.1" class="ltx_Math" alttext="65.6" display="inline"><semantics id="S5.T4.39.39.6.m1.1a"><mn id="S5.T4.39.39.6.m1.1.1" xref="S5.T4.39.39.6.m1.1.1.cmml">65.6</mn><annotation-xml encoding="MathML-Content" id="S5.T4.39.39.6.m1.1b"><cn type="float" id="S5.T4.39.39.6.m1.1.1.cmml" xref="S5.T4.39.39.6.m1.1.1">65.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.39.39.6.m1.1c">65.6</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.45.45" class="ltx_tr">
<th id="S5.T4.40.40.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S5.T4.40.40.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T4.40.40.1.m1.1a"><msub id="S5.T4.40.40.1.m1.1.1" xref="S5.T4.40.40.1.m1.1.1.cmml"><mtext id="S5.T4.40.40.1.m1.1.1.2" xref="S5.T4.40.40.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T4.40.40.1.m1.1.1.3" xref="S5.T4.40.40.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T4.40.40.1.m1.1b"><apply id="S5.T4.40.40.1.m1.1.1.cmml" xref="S5.T4.40.40.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T4.40.40.1.m1.1.1.1.cmml" xref="S5.T4.40.40.1.m1.1.1">subscript</csymbol><ci id="S5.T4.40.40.1.m1.1.1.2a.cmml" xref="S5.T4.40.40.1.m1.1.1.2"><mtext id="S5.T4.40.40.1.m1.1.1.2.cmml" xref="S5.T4.40.40.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T4.40.40.1.m1.1.1.3a.cmml" xref="S5.T4.40.40.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T4.40.40.1.m1.1.1.3.cmml" xref="S5.T4.40.40.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.40.40.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<th id="S5.T4.41.41.2" class="ltx_td ltx_align_left ltx_th ltx_th_row">Approx. Query + BM25 (dynamic top-<math id="S5.T4.41.41.2.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.T4.41.41.2.m1.1a"><mi id="S5.T4.41.41.2.m1.1.1" xref="S5.T4.41.41.2.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.T4.41.41.2.m1.1b"><ci id="S5.T4.41.41.2.m1.1.1.cmml" xref="S5.T4.41.41.2.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.41.41.2.m1.1c">k</annotation></semantics></math>)</th>
<td id="S5.T4.42.42.3" class="ltx_td ltx_align_center"><math id="S5.T4.42.42.3.m1.1" class="ltx_Math" alttext="97.2" display="inline"><semantics id="S5.T4.42.42.3.m1.1a"><mn id="S5.T4.42.42.3.m1.1.1" xref="S5.T4.42.42.3.m1.1.1.cmml">97.2</mn><annotation-xml encoding="MathML-Content" id="S5.T4.42.42.3.m1.1b"><cn type="float" id="S5.T4.42.42.3.m1.1.1.cmml" xref="S5.T4.42.42.3.m1.1.1">97.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.42.42.3.m1.1c">97.2</annotation></semantics></math></td>
<td id="S5.T4.43.43.4" class="ltx_td ltx_align_center"><math id="S5.T4.43.43.4.m1.1" class="ltx_Math" alttext="49.0" display="inline"><semantics id="S5.T4.43.43.4.m1.1a"><mn id="S5.T4.43.43.4.m1.1.1" xref="S5.T4.43.43.4.m1.1.1.cmml">49.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.43.43.4.m1.1b"><cn type="float" id="S5.T4.43.43.4.m1.1.1.cmml" xref="S5.T4.43.43.4.m1.1.1">49.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.43.43.4.m1.1c">49.0</annotation></semantics></math></td>
<td id="S5.T4.44.44.5" class="ltx_td ltx_align_center"><math id="S5.T4.44.44.5.m1.1" class="ltx_Math" alttext="\mathbf{82.0}" display="inline"><semantics id="S5.T4.44.44.5.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.44.44.5.m1.1.1" xref="S5.T4.44.44.5.m1.1.1.cmml">82.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.44.44.5.m1.1b"><cn type="float" id="S5.T4.44.44.5.m1.1.1.cmml" xref="S5.T4.44.44.5.m1.1.1">82.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.44.44.5.m1.1c">\mathbf{82.0}</annotation></semantics></math></td>
<td id="S5.T4.45.45.6" class="ltx_td ltx_align_center"><math id="S5.T4.45.45.6.m1.1" class="ltx_Math" alttext="\mathbf{65.7}" display="inline"><semantics id="S5.T4.45.45.6.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.45.45.6.m1.1.1" xref="S5.T4.45.45.6.m1.1.1.cmml">65.7</mn><annotation-xml encoding="MathML-Content" id="S5.T4.45.45.6.m1.1b"><cn type="float" id="S5.T4.45.45.6.m1.1.1.cmml" xref="S5.T4.45.45.6.m1.1.1">65.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.45.45.6.m1.1c">\mathbf{65.7}</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.50.50" class="ltx_tr">
<th id="S5.T4.50.50.6" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">Graphix-T5</th>
<th id="S5.T4.46.46.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">N<math id="S5.T4.46.46.1.m1.1" class="ltx_Math" alttext="/" display="inline"><semantics id="S5.T4.46.46.1.m1.1a"><mo id="S5.T4.46.46.1.m1.1.1" xref="S5.T4.46.46.1.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="S5.T4.46.46.1.m1.1b"><divide id="S5.T4.46.46.1.m1.1.1.cmml" xref="S5.T4.46.46.1.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.46.46.1.m1.1c">/</annotation></semantics></math>A</th>
<td id="S5.T4.47.47.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.47.47.2.m1.1" class="ltx_Math" alttext="100.0" display="inline"><semantics id="S5.T4.47.47.2.m1.1a"><mn id="S5.T4.47.47.2.m1.1.1" xref="S5.T4.47.47.2.m1.1.1.cmml">100.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.47.47.2.m1.1b"><cn type="float" id="S5.T4.47.47.2.m1.1.1.cmml" xref="S5.T4.47.47.2.m1.1.1">100.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.47.47.2.m1.1c">100.0</annotation></semantics></math></td>
<td id="S5.T4.48.48.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.48.48.3.m1.1" class="ltx_Math" alttext="0.0" display="inline"><semantics id="S5.T4.48.48.3.m1.1a"><mn id="S5.T4.48.48.3.m1.1.1" xref="S5.T4.48.48.3.m1.1.1.cmml">0.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.48.48.3.m1.1b"><cn type="float" id="S5.T4.48.48.3.m1.1.1.cmml" xref="S5.T4.48.48.3.m1.1.1">0.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.48.48.3.m1.1c">0.0</annotation></semantics></math></td>
<td id="S5.T4.49.49.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.49.49.4.m1.1" class="ltx_Math" alttext="79.8" display="inline"><semantics id="S5.T4.49.49.4.m1.1a"><mn id="S5.T4.49.49.4.m1.1.1" xref="S5.T4.49.49.4.m1.1.1.cmml">79.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.49.49.4.m1.1b"><cn type="float" id="S5.T4.49.49.4.m1.1.1.cmml" xref="S5.T4.49.49.4.m1.1.1">79.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.49.49.4.m1.1c">79.8</annotation></semantics></math></td>
<td id="S5.T4.50.50.5" class="ltx_td ltx_align_center ltx_border_t"><math id="S5.T4.50.50.5.m1.1" class="ltx_Math" alttext="65.6" display="inline"><semantics id="S5.T4.50.50.5.m1.1a"><mn id="S5.T4.50.50.5.m1.1.1" xref="S5.T4.50.50.5.m1.1.1.cmml">65.6</mn><annotation-xml encoding="MathML-Content" id="S5.T4.50.50.5.m1.1b"><cn type="float" id="S5.T4.50.50.5.m1.1.1.cmml" xref="S5.T4.50.50.5.m1.1.1">65.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.50.50.5.m1.1c">65.6</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.54.54" class="ltx_tr">
<th id="S5.T4.54.54.5" class="ltx_td ltx_align_left ltx_th ltx_th_row">Graphix-T5</th>
<th id="S5.T4.54.54.6" class="ltx_td ltx_align_left ltx_th ltx_th_row">Approx. Query</th>
<td id="S5.T4.51.51.1" class="ltx_td ltx_align_center"><math id="S5.T4.51.51.1.m1.1" class="ltx_Math" alttext="92.3" display="inline"><semantics id="S5.T4.51.51.1.m1.1a"><mn id="S5.T4.51.51.1.m1.1.1" xref="S5.T4.51.51.1.m1.1.1.cmml">92.3</mn><annotation-xml encoding="MathML-Content" id="S5.T4.51.51.1.m1.1b"><cn type="float" id="S5.T4.51.51.1.m1.1.1.cmml" xref="S5.T4.51.51.1.m1.1.1">92.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.51.51.1.m1.1c">92.3</annotation></semantics></math></td>
<td id="S5.T4.52.52.2" class="ltx_td ltx_align_center"><math id="S5.T4.52.52.2.m1.1" class="ltx_Math" alttext="71.8" display="inline"><semantics id="S5.T4.52.52.2.m1.1a"><mn id="S5.T4.52.52.2.m1.1.1" xref="S5.T4.52.52.2.m1.1.1.cmml">71.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.52.52.2.m1.1b"><cn type="float" id="S5.T4.52.52.2.m1.1.1.cmml" xref="S5.T4.52.52.2.m1.1.1">71.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.52.52.2.m1.1c">71.8</annotation></semantics></math></td>
<td id="S5.T4.53.53.3" class="ltx_td ltx_align_center"><math id="S5.T4.53.53.3.m1.1" class="ltx_Math" alttext="81.8" display="inline"><semantics id="S5.T4.53.53.3.m1.1a"><mn id="S5.T4.53.53.3.m1.1.1" xref="S5.T4.53.53.3.m1.1.1.cmml">81.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.53.53.3.m1.1b"><cn type="float" id="S5.T4.53.53.3.m1.1.1.cmml" xref="S5.T4.53.53.3.m1.1.1">81.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.53.53.3.m1.1c">81.8</annotation></semantics></math></td>
<td id="S5.T4.54.54.4" class="ltx_td ltx_align_center"><math id="S5.T4.54.54.4.m1.1" class="ltx_Math" alttext="68.8" display="inline"><semantics id="S5.T4.54.54.4.m1.1a"><mn id="S5.T4.54.54.4.m1.1.1" xref="S5.T4.54.54.4.m1.1.1.cmml">68.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.54.54.4.m1.1b"><cn type="float" id="S5.T4.54.54.4.m1.1.1.cmml" xref="S5.T4.54.54.4.m1.1.1">68.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.54.54.4.m1.1c">68.8</annotation></semantics></math></td>
</tr>
<tr id="S5.T4.59.59" class="ltx_tr">
<th id="S5.T4.59.59.6" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">Graphix-T5</th>
<th id="S5.T4.55.55.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">Approx. Query + BM25 (dynamic top-<math id="S5.T4.55.55.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.T4.55.55.1.m1.1a"><mi id="S5.T4.55.55.1.m1.1.1" xref="S5.T4.55.55.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.T4.55.55.1.m1.1b"><ci id="S5.T4.55.55.1.m1.1.1.cmml" xref="S5.T4.55.55.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.55.55.1.m1.1c">k</annotation></semantics></math>)</th>
<td id="S5.T4.56.56.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.56.56.2.m1.1" class="ltx_Math" alttext="97.9" display="inline"><semantics id="S5.T4.56.56.2.m1.1a"><mn id="S5.T4.56.56.2.m1.1.1" xref="S5.T4.56.56.2.m1.1.1.cmml">97.9</mn><annotation-xml encoding="MathML-Content" id="S5.T4.56.56.2.m1.1b"><cn type="float" id="S5.T4.56.56.2.m1.1.1.cmml" xref="S5.T4.56.56.2.m1.1.1">97.9</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.56.56.2.m1.1c">97.9</annotation></semantics></math></td>
<td id="S5.T4.57.57.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.57.57.3.m1.1" class="ltx_Math" alttext="49.4" display="inline"><semantics id="S5.T4.57.57.3.m1.1a"><mn id="S5.T4.57.57.3.m1.1.1" xref="S5.T4.57.57.3.m1.1.1.cmml">49.4</mn><annotation-xml encoding="MathML-Content" id="S5.T4.57.57.3.m1.1b"><cn type="float" id="S5.T4.57.57.3.m1.1.1.cmml" xref="S5.T4.57.57.3.m1.1.1">49.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.57.57.3.m1.1c">49.4</annotation></semantics></math></td>
<td id="S5.T4.58.58.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.58.58.4.m1.1" class="ltx_Math" alttext="\mathbf{83.0}" display="inline"><semantics id="S5.T4.58.58.4.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.58.58.4.m1.1.1" xref="S5.T4.58.58.4.m1.1.1.cmml">83.0</mn><annotation-xml encoding="MathML-Content" id="S5.T4.58.58.4.m1.1b"><cn type="float" id="S5.T4.58.58.4.m1.1.1.cmml" xref="S5.T4.58.58.4.m1.1.1">83.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.58.58.4.m1.1c">\mathbf{83.0}</annotation></semantics></math></td>
<td id="S5.T4.59.59.5" class="ltx_td ltx_align_center ltx_border_bb"><math id="S5.T4.59.59.5.m1.1" class="ltx_Math" alttext="\mathbf{68.8}" display="inline"><semantics id="S5.T4.59.59.5.m1.1a"><mn class="ltx_mathvariant_bold" mathvariant="bold" id="S5.T4.59.59.5.m1.1.1" xref="S5.T4.59.59.5.m1.1.1.cmml">68.8</mn><annotation-xml encoding="MathML-Content" id="S5.T4.59.59.5.m1.1b"><cn type="float" id="S5.T4.59.59.5.m1.1.1.cmml" xref="S5.T4.59.59.5.m1.1.1">68.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.T4.59.59.5.m1.1c">\mathbf{68.8}</annotation></semantics></math></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 4: </span>Recall, Schema Shortening, EX and EM scores (using <span id="S5.T4.63.1" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span>) across different schema selection setups, on the development split of S<span id="S5.T4.64.2" class="ltx_text ltx_font_smallcaps">pider</span>. Value selection is enabled for the selected columns across all setups. For the oracle setup, we report performance upper-bounds using <span id="S5.T4.65.3" class="ltx_text ltx_font_italic">only</span> the schema elements from the gold query.</figcaption>
</figure>
<div id="S5.SS3.SSS1.p1" class="ltx_para">
<p id="S5.SS3.SSS1.p1.1" class="ltx_p">We evaluate our proposed schema selection strategy in a two-fold manner, given that value selection is applied for selected columns. Firstly, we use recall and schema shortening (rate) to compute averaged metrics across all samples showcasing the extent to which 
<span id="S5.I5" class="ltx_inline-enumerate">
<span id="S5.I5.i1" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(i)</span> <span id="S5.I5.i1.1" class="ltx_text">the most relevant schema elements are successfully retrieved, and </span></span>
<span id="S5.I5.i2" class="ltx_inline-item"><span class="ltx_tag ltx_tag_inline-item">(ii)</span> <span id="S5.I5.i2.1" class="ltx_text">the size of the resulting schema, after selection, with respect to its original size</span></span>
</span>. Secondly, we explore how the performance of the end-system changes across different schema pruning settings by reporting EX and EM scores. Recall is the percentage of samples for which all ground-truth schema elements are selected. Schema shortening is the number of schema elements that are excluded divided by the total number of schema elements. Results are summarised in Table <a href="#S5.T4" title="Table 4 ‣ 5.3.1 Schema Selection Evaluation ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>.</p>
</div>
<div id="S5.SS3.SSS1.p2" class="ltx_para">
<p id="S5.SS3.SSS1.p2.4" class="ltx_p">The benefits of schema selection are apparent in the oracle setup, in which only schema elements from the gold query are included in the prompt (cf. Table <a href="#A1.T8" title="Table 8 ‣ Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>). In this setup, the highest execution accuracy is achieved while filtering out <math id="S5.SS3.SSS1.p2.1.m1.1" class="ltx_Math" alttext="&gt;70\%" display="inline"><semantics id="S5.SS3.SSS1.p2.1.m1.1a"><mrow id="S5.SS3.SSS1.p2.1.m1.1.1" xref="S5.SS3.SSS1.p2.1.m1.1.1.cmml"><mi id="S5.SS3.SSS1.p2.1.m1.1.1.2" xref="S5.SS3.SSS1.p2.1.m1.1.1.2.cmml"></mi><mo id="S5.SS3.SSS1.p2.1.m1.1.1.1" xref="S5.SS3.SSS1.p2.1.m1.1.1.1.cmml">&gt;</mo><mrow id="S5.SS3.SSS1.p2.1.m1.1.1.3" xref="S5.SS3.SSS1.p2.1.m1.1.1.3.cmml"><mn id="S5.SS3.SSS1.p2.1.m1.1.1.3.2" xref="S5.SS3.SSS1.p2.1.m1.1.1.3.2.cmml">70</mn><mo id="S5.SS3.SSS1.p2.1.m1.1.1.3.1" xref="S5.SS3.SSS1.p2.1.m1.1.1.3.1.cmml">%</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS1.p2.1.m1.1b"><apply id="S5.SS3.SSS1.p2.1.m1.1.1.cmml" xref="S5.SS3.SSS1.p2.1.m1.1.1"><gt id="S5.SS3.SSS1.p2.1.m1.1.1.1.cmml" xref="S5.SS3.SSS1.p2.1.m1.1.1.1"></gt><csymbol cd="latexml" id="S5.SS3.SSS1.p2.1.m1.1.1.2.cmml" xref="S5.SS3.SSS1.p2.1.m1.1.1.2">absent</csymbol><apply id="S5.SS3.SSS1.p2.1.m1.1.1.3.cmml" xref="S5.SS3.SSS1.p2.1.m1.1.1.3"><csymbol cd="latexml" id="S5.SS3.SSS1.p2.1.m1.1.1.3.1.cmml" xref="S5.SS3.SSS1.p2.1.m1.1.1.3.1">percent</csymbol><cn type="integer" id="S5.SS3.SSS1.p2.1.m1.1.1.3.2.cmml" xref="S5.SS3.SSS1.p2.1.m1.1.1.3.2">70</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS1.p2.1.m1.1c">&gt;70\%</annotation></semantics></math> of the original schema on average. We note that our approach of coupling the schema elements returned in the approximated query with the ones returned by BM25 navigates a healthy trade-off between maximising recall and reducing processing of unnecessary schema elements. We also notice that our strategy of dynamically determining the number of retained schema elements per input (cf. Section <a href="#S4.SS1.SSS1" title="4.1.1 Incorporating for Approximated Query ‣ 4.1 Schema Selection ‣ 4 Database Context Selection ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1.1</span></a>) results in improvements compared to static top-<math id="S5.SS3.SSS1.p2.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.SS3.SSS1.p2.2.m2.1a"><mi id="S5.SS3.SSS1.p2.2.m2.1.1" xref="S5.SS3.SSS1.p2.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS1.p2.2.m2.1b"><ci id="S5.SS3.SSS1.p2.2.m2.1.1.cmml" xref="S5.SS3.SSS1.p2.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS1.p2.2.m2.1c">k</annotation></semantics></math> determination. For roughly the same extent of schema shortening (i.e. by comparing scores with dynamic top-<math id="S5.SS3.SSS1.p2.3.m3.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S5.SS3.SSS1.p2.3.m3.1a"><mi id="S5.SS3.SSS1.p2.3.m3.1.1" xref="S5.SS3.SSS1.p2.3.m3.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS1.p2.3.m3.1b"><ci id="S5.SS3.SSS1.p2.3.m3.1.1.cmml" xref="S5.SS3.SSS1.p2.3.m3.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS1.p2.3.m3.1c">k</annotation></semantics></math> against top-<math id="S5.SS3.SSS1.p2.4.m4.1" class="ltx_Math" alttext="7" display="inline"><semantics id="S5.SS3.SSS1.p2.4.m4.1a"><mn id="S5.SS3.SSS1.p2.4.m4.1.1" xref="S5.SS3.SSS1.p2.4.m4.1.1.cmml">7</mn><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS1.p2.4.m4.1b"><cn type="integer" id="S5.SS3.SSS1.p2.4.m4.1.1.cmml" xref="S5.SS3.SSS1.p2.4.m4.1.1">7</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS1.p2.4.m4.1c">7</annotation></semantics></math>), the results with the former are higher across all metrics.</p>
</div>
</section>
<section id="S5.SS3.SSS2" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.3.2 </span>Ablation Study</h4>

<div id="S5.SS3.SSS2.p1" class="ltx_para">
<p id="S5.SS3.SSS2.p1.2" class="ltx_p">Table <a href="#S5.T5" title="Table 5 ‣ 5.3.2 Ablation Study ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a> shows a comprehensive ablation study for the efficacy of our database context selection, and example selection methods including DAIL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite> and AST.
We consistently notice improvement when selecting examples using AST, for the same approximator. Interestingly, the performance gap is increasing the better the approximator becomes, leading to an improvement <math id="S5.SS3.SSS2.p1.1.m1.1" class="ltx_Math" alttext="&gt;2.4\%" display="inline"><semantics id="S5.SS3.SSS2.p1.1.m1.1a"><mrow id="S5.SS3.SSS2.p1.1.m1.1.1" xref="S5.SS3.SSS2.p1.1.m1.1.1.cmml"><mi id="S5.SS3.SSS2.p1.1.m1.1.1.2" xref="S5.SS3.SSS2.p1.1.m1.1.1.2.cmml"></mi><mo id="S5.SS3.SSS2.p1.1.m1.1.1.1" xref="S5.SS3.SSS2.p1.1.m1.1.1.1.cmml">&gt;</mo><mrow id="S5.SS3.SSS2.p1.1.m1.1.1.3" xref="S5.SS3.SSS2.p1.1.m1.1.1.3.cmml"><mn id="S5.SS3.SSS2.p1.1.m1.1.1.3.2" xref="S5.SS3.SSS2.p1.1.m1.1.1.3.2.cmml">2.4</mn><mo id="S5.SS3.SSS2.p1.1.m1.1.1.3.1" xref="S5.SS3.SSS2.p1.1.m1.1.1.3.1.cmml">%</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS2.p1.1.m1.1b"><apply id="S5.SS3.SSS2.p1.1.m1.1.1.cmml" xref="S5.SS3.SSS2.p1.1.m1.1.1"><gt id="S5.SS3.SSS2.p1.1.m1.1.1.1.cmml" xref="S5.SS3.SSS2.p1.1.m1.1.1.1"></gt><csymbol cd="latexml" id="S5.SS3.SSS2.p1.1.m1.1.1.2.cmml" xref="S5.SS3.SSS2.p1.1.m1.1.1.2">absent</csymbol><apply id="S5.SS3.SSS2.p1.1.m1.1.1.3.cmml" xref="S5.SS3.SSS2.p1.1.m1.1.1.3"><csymbol cd="latexml" id="S5.SS3.SSS2.p1.1.m1.1.1.3.1.cmml" xref="S5.SS3.SSS2.p1.1.m1.1.1.3.1">percent</csymbol><cn type="float" id="S5.SS3.SSS2.p1.1.m1.1.1.3.2.cmml" xref="S5.SS3.SSS2.p1.1.m1.1.1.3.2">2.4</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS2.p1.1.m1.1c">&gt;2.4\%</annotation></semantics></math> in the case of an oracle approximator. This finding is in agreement with our hypothesis that AST re-ranking can preserve structural information for more precise example selection when <math id="S5.SS3.SSS2.p1.2.m2.1" class="ltx_Math" alttext="\mathbf{s}^{\prime}\sim\mathbf{s}" display="inline"><semantics id="S5.SS3.SSS2.p1.2.m2.1a"><mrow id="S5.SS3.SSS2.p1.2.m2.1.1" xref="S5.SS3.SSS2.p1.2.m2.1.1.cmml"><msup id="S5.SS3.SSS2.p1.2.m2.1.1.2" xref="S5.SS3.SSS2.p1.2.m2.1.1.2.cmml"><mi id="S5.SS3.SSS2.p1.2.m2.1.1.2.2" xref="S5.SS3.SSS2.p1.2.m2.1.1.2.2.cmml">𝐬</mi><mo id="S5.SS3.SSS2.p1.2.m2.1.1.2.3" xref="S5.SS3.SSS2.p1.2.m2.1.1.2.3.cmml">′</mo></msup><mo id="S5.SS3.SSS2.p1.2.m2.1.1.1" xref="S5.SS3.SSS2.p1.2.m2.1.1.1.cmml">∼</mo><mi id="S5.SS3.SSS2.p1.2.m2.1.1.3" xref="S5.SS3.SSS2.p1.2.m2.1.1.3.cmml">𝐬</mi></mrow><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS2.p1.2.m2.1b"><apply id="S5.SS3.SSS2.p1.2.m2.1.1.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1"><csymbol cd="latexml" id="S5.SS3.SSS2.p1.2.m2.1.1.1.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1.1">similar-to</csymbol><apply id="S5.SS3.SSS2.p1.2.m2.1.1.2.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1.2"><csymbol cd="ambiguous" id="S5.SS3.SSS2.p1.2.m2.1.1.2.1.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1.2">superscript</csymbol><ci id="S5.SS3.SSS2.p1.2.m2.1.1.2.2.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1.2.2">𝐬</ci><ci id="S5.SS3.SSS2.p1.2.m2.1.1.2.3.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1.2.3">′</ci></apply><ci id="S5.SS3.SSS2.p1.2.m2.1.1.3.cmml" xref="S5.SS3.SSS2.p1.2.m2.1.1.3">𝐬</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS2.p1.2.m2.1c">\mathbf{s}^{\prime}\sim\mathbf{s}</annotation></semantics></math>. The inclusion of combined schema and value selection (SVS) leads to further improvements when coupled with example selection based on AST or DAIL.</p>
</div>
<figure id="S5.T5" class="ltx_table">
<table id="S5.T5.2" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S5.T5.2.3.1" class="ltx_tr">
<th id="S5.T5.2.3.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_tt"><span id="S5.T5.2.3.1.1.1" class="ltx_text ltx_font_bold">Approximator</span></th>
<th id="S5.T5.2.3.1.2" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt"><span id="S5.T5.2.3.1.2.1" class="ltx_text ltx_font_bold">Selection</span></th>
<th id="S5.T5.2.3.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S5.T5.2.3.1.3.1" class="ltx_text ltx_font_bold">EX</span></th>
<th id="S5.T5.2.3.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S5.T5.2.3.1.4.1" class="ltx_text ltx_font_bold">EM</span></th>
</tr>
<tr id="S5.T5.1.1" class="ltx_tr">
<th id="S5.T5.1.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_th_row ltx_border_t">N<math id="S5.T5.1.1.1.m1.1" class="ltx_Math" alttext="/" display="inline"><semantics id="S5.T5.1.1.1.m1.1a"><mo id="S5.T5.1.1.1.m1.1.1" xref="S5.T5.1.1.1.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="S5.T5.1.1.1.m1.1b"><divide id="S5.T5.1.1.1.m1.1.1.cmml" xref="S5.T5.1.1.1.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.1.1.1.m1.1c">/</annotation></semantics></math>A</th>
<th id="S5.T5.1.1.2" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_t">Question Similarity</th>
<th id="S5.T5.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t">74.7</th>
<th id="S5.T5.1.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t">52.3</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S5.T5.2.2" class="ltx_tr">
<th id="S5.T5.2.2.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" rowspan="6"><span id="S5.T5.2.2.1.1" class="ltx_text"><math id="S5.T5.2.2.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.T5.2.2.1.1.m1.1a"><msub id="S5.T5.2.2.1.1.m1.1.1" xref="S5.T5.2.2.1.1.m1.1.1.cmml"><mtext id="S5.T5.2.2.1.1.m1.1.1.2" xref="S5.T5.2.2.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.T5.2.2.1.1.m1.1.1.3" xref="S5.T5.2.2.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.T5.2.2.1.1.m1.1b"><apply id="S5.T5.2.2.1.1.m1.1.1.cmml" xref="S5.T5.2.2.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.T5.2.2.1.1.m1.1.1.1.cmml" xref="S5.T5.2.2.1.1.m1.1.1">subscript</csymbol><ci id="S5.T5.2.2.1.1.m1.1.1.2a.cmml" xref="S5.T5.2.2.1.1.m1.1.1.2"><mtext id="S5.T5.2.2.1.1.m1.1.1.2.cmml" xref="S5.T5.2.2.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S5.T5.2.2.1.1.m1.1.1.3a.cmml" xref="S5.T5.2.2.1.1.m1.1.1.3"><mtext mathsize="70%" id="S5.T5.2.2.1.1.m1.1.1.3.cmml" xref="S5.T5.2.2.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.T5.2.2.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></span></th>
<th id="S5.T5.2.2.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">DAIL</th>
<td id="S5.T5.2.2.3" class="ltx_td ltx_align_center ltx_border_t">78.6</td>
<td id="S5.T5.2.2.4" class="ltx_td ltx_align_center ltx_border_t">61.4</td>
</tr>
<tr id="S5.T5.2.4.1" class="ltx_tr">
<th id="S5.T5.2.4.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">DAIL + SVS</th>
<td id="S5.T5.2.4.1.2" class="ltx_td ltx_align_center">81.3</td>
<td id="S5.T5.2.4.1.3" class="ltx_td ltx_align_center">62.3</td>
</tr>
<tr id="S5.T5.2.5.2" class="ltx_tr">
<th id="S5.T5.2.5.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST</th>
<td id="S5.T5.2.5.2.2" class="ltx_td ltx_align_center">79.3</td>
<td id="S5.T5.2.5.2.3" class="ltx_td ltx_align_center">63.6</td>
</tr>
<tr id="S5.T5.2.6.3" class="ltx_tr">
<th id="S5.T5.2.6.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + VS</th>
<td id="S5.T5.2.6.3.2" class="ltx_td ltx_align_center">80.4</td>
<td id="S5.T5.2.6.3.3" class="ltx_td ltx_align_center">63.8</td>
</tr>
<tr id="S5.T5.2.7.4" class="ltx_tr">
<th id="S5.T5.2.7.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + SS</th>
<td id="S5.T5.2.7.4.2" class="ltx_td ltx_align_center">78.9</td>
<td id="S5.T5.2.7.4.3" class="ltx_td ltx_align_center">63.8</td>
</tr>
<tr id="S5.T5.2.8.5" class="ltx_tr">
<th id="S5.T5.2.8.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + SVS</th>
<td id="S5.T5.2.8.5.2" class="ltx_td ltx_align_center">82.0</td>
<td id="S5.T5.2.8.5.3" class="ltx_td ltx_align_center">65.7</td>
</tr>
<tr id="S5.T5.2.9.6" class="ltx_tr">
<th id="S5.T5.2.9.6.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" rowspan="6"><span id="S5.T5.2.9.6.1.1" class="ltx_text">Graphix-T5</span></th>
<th id="S5.T5.2.9.6.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">DAIL</th>
<td id="S5.T5.2.9.6.3" class="ltx_td ltx_align_center ltx_border_t">77.8</td>
<td id="S5.T5.2.9.6.4" class="ltx_td ltx_align_center ltx_border_t">61.9</td>
</tr>
<tr id="S5.T5.2.10.7" class="ltx_tr">
<th id="S5.T5.2.10.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">DAIL + SVS</th>
<td id="S5.T5.2.10.7.2" class="ltx_td ltx_align_center">81.0</td>
<td id="S5.T5.2.10.7.3" class="ltx_td ltx_align_center">63.7</td>
</tr>
<tr id="S5.T5.2.11.8" class="ltx_tr">
<th id="S5.T5.2.11.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST</th>
<td id="S5.T5.2.11.8.2" class="ltx_td ltx_align_center">79.8</td>
<td id="S5.T5.2.11.8.3" class="ltx_td ltx_align_center">65.6</td>
</tr>
<tr id="S5.T5.2.12.9" class="ltx_tr">
<th id="S5.T5.2.12.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + VS</th>
<td id="S5.T5.2.12.9.2" class="ltx_td ltx_align_center">81.4</td>
<td id="S5.T5.2.12.9.3" class="ltx_td ltx_align_center">66.4</td>
</tr>
<tr id="S5.T5.2.13.10" class="ltx_tr">
<th id="S5.T5.2.13.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + SS</th>
<td id="S5.T5.2.13.10.2" class="ltx_td ltx_align_center">80.2</td>
<td id="S5.T5.2.13.10.3" class="ltx_td ltx_align_center">66.2</td>
</tr>
<tr id="S5.T5.2.14.11" class="ltx_tr">
<th id="S5.T5.2.14.11.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + SVS</th>
<td id="S5.T5.2.14.11.2" class="ltx_td ltx_align_center">83.0</td>
<td id="S5.T5.2.14.11.3" class="ltx_td ltx_align_center">68.8</td>
</tr>
<tr id="S5.T5.2.15.12" class="ltx_tr">
<th id="S5.T5.2.15.12.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_bb ltx_border_t" rowspan="6"><span id="S5.T5.2.15.12.1.1" class="ltx_text ltx_font_italic">Oracle</span></th>
<th id="S5.T5.2.15.12.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">DAIL</th>
<td id="S5.T5.2.15.12.3" class="ltx_td ltx_align_center ltx_border_t">79.1</td>
<td id="S5.T5.2.15.12.4" class="ltx_td ltx_align_center ltx_border_t">63.2</td>
</tr>
<tr id="S5.T5.2.16.13" class="ltx_tr">
<th id="S5.T5.2.16.13.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">DAIL + SVS</th>
<td id="S5.T5.2.16.13.2" class="ltx_td ltx_align_center">82.5</td>
<td id="S5.T5.2.16.13.3" class="ltx_td ltx_align_center">66.0</td>
</tr>
<tr id="S5.T5.2.17.14" class="ltx_tr">
<th id="S5.T5.2.17.14.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST</th>
<td id="S5.T5.2.17.14.2" class="ltx_td ltx_align_center">81.0</td>
<td id="S5.T5.2.17.14.3" class="ltx_td ltx_align_center">67.6</td>
</tr>
<tr id="S5.T5.2.18.15" class="ltx_tr">
<th id="S5.T5.2.18.15.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + VS</th>
<td id="S5.T5.2.18.15.2" class="ltx_td ltx_align_center">82.6</td>
<td id="S5.T5.2.18.15.3" class="ltx_td ltx_align_center">68.1</td>
</tr>
<tr id="S5.T5.2.19.16" class="ltx_tr">
<th id="S5.T5.2.19.16.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">AST + SS</th>
<td id="S5.T5.2.19.16.2" class="ltx_td ltx_align_center">82.5</td>
<td id="S5.T5.2.19.16.3" class="ltx_td ltx_align_center">69.6</td>
</tr>
<tr id="S5.T5.2.20.17" class="ltx_tr">
<th id="S5.T5.2.20.17.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">AST + SVS</th>
<td id="S5.T5.2.20.17.2" class="ltx_td ltx_align_center ltx_border_bb">84.6</td>
<td id="S5.T5.2.20.17.3" class="ltx_td ltx_align_center ltx_border_bb">71.3</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 5: </span>EX and EM scores on the development set of S<span id="S5.T5.5.1" class="ltx_text ltx_font_smallcaps">pider</span>, with <span id="S5.T5.6.2" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span>, across different approximators, and selection setups: example selection (with DAIL or AST), schema selection (SS), value selection (VS), and schema &amp; value selection (SVS). We report results from oracle approximator using gold queries.</figcaption>
</figure>
</section>
</section>
<section id="S5.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.4 </span>Schema Splitting</h3>

<div id="S5.SS4.p1" class="ltx_para">
<p id="S5.SS4.p1.10" class="ltx_p">We evaluate the effect of splitting a schema into <math id="S5.SS4.p1.1.m1.1" class="ltx_Math" alttext="r_{m}" display="inline"><semantics id="S5.SS4.p1.1.m1.1a"><msub id="S5.SS4.p1.1.m1.1.1" xref="S5.SS4.p1.1.m1.1.1.cmml"><mi id="S5.SS4.p1.1.m1.1.1.2" xref="S5.SS4.p1.1.m1.1.1.2.cmml">r</mi><mi id="S5.SS4.p1.1.m1.1.1.3" xref="S5.SS4.p1.1.m1.1.1.3.cmml">m</mi></msub><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.1.m1.1b"><apply id="S5.SS4.p1.1.m1.1.1.cmml" xref="S5.SS4.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS4.p1.1.m1.1.1.1.cmml" xref="S5.SS4.p1.1.m1.1.1">subscript</csymbol><ci id="S5.SS4.p1.1.m1.1.1.2.cmml" xref="S5.SS4.p1.1.m1.1.1.2">𝑟</ci><ci id="S5.SS4.p1.1.m1.1.1.3.cmml" xref="S5.SS4.p1.1.m1.1.1.3">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.1.m1.1c">r_{m}</annotation></semantics></math> splits, using <math id="S5.SS4.p1.2.m2.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.SS4.p1.2.m2.1a"><msub id="S5.SS4.p1.2.m2.1.1" xref="S5.SS4.p1.2.m2.1.1.cmml"><mtext id="S5.SS4.p1.2.m2.1.1.2" xref="S5.SS4.p1.2.m2.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.SS4.p1.2.m2.1.1.3" xref="S5.SS4.p1.2.m2.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.2.m2.1b"><apply id="S5.SS4.p1.2.m2.1.1.cmml" xref="S5.SS4.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S5.SS4.p1.2.m2.1.1.1.cmml" xref="S5.SS4.p1.2.m2.1.1">subscript</csymbol><ci id="S5.SS4.p1.2.m2.1.1.2a.cmml" xref="S5.SS4.p1.2.m2.1.1.2"><mtext id="S5.SS4.p1.2.m2.1.1.2.cmml" xref="S5.SS4.p1.2.m2.1.1.2">FastRAT</mtext></ci><ci id="S5.SS4.p1.2.m2.1.1.3a.cmml" xref="S5.SS4.p1.2.m2.1.1.3"><mtext mathsize="70%" id="S5.SS4.p1.2.m2.1.1.3.cmml" xref="S5.SS4.p1.2.m2.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.2.m2.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> for schema selection. Figure <a href="#S5.F1" title="Figure 1 ‣ 5.4 Schema Splitting ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> shows EX scores across different maximum number of columns per schema split (<math id="S5.SS4.p1.3.m3.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.SS4.p1.3.m3.1a"><mi id="S5.SS4.p1.3.m3.1.1" xref="S5.SS4.p1.3.m3.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.3.m3.1b"><ci id="S5.SS4.p1.3.m3.1.1.cmml" xref="S5.SS4.p1.3.m3.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.3.m3.1c">r</annotation></semantics></math>), on the development set of S<span id="S5.SS4.p1.10.1" class="ltx_text ltx_font_smallcaps">pider</span>. We see that the EX scores of our approach remain consistent across different <math id="S5.SS4.p1.4.m4.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.SS4.p1.4.m4.1a"><mi id="S5.SS4.p1.4.m4.1.1" xref="S5.SS4.p1.4.m4.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.4.m4.1b"><ci id="S5.SS4.p1.4.m4.1.1.cmml" xref="S5.SS4.p1.4.m4.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.4.m4.1c">r</annotation></semantics></math>. The performance in the case where particular schemata from the development set are split into <math id="S5.SS4.p1.5.m5.1" class="ltx_Math" alttext="r_{m}=3" display="inline"><semantics id="S5.SS4.p1.5.m5.1a"><mrow id="S5.SS4.p1.5.m5.1.1" xref="S5.SS4.p1.5.m5.1.1.cmml"><msub id="S5.SS4.p1.5.m5.1.1.2" xref="S5.SS4.p1.5.m5.1.1.2.cmml"><mi id="S5.SS4.p1.5.m5.1.1.2.2" xref="S5.SS4.p1.5.m5.1.1.2.2.cmml">r</mi><mi id="S5.SS4.p1.5.m5.1.1.2.3" xref="S5.SS4.p1.5.m5.1.1.2.3.cmml">m</mi></msub><mo id="S5.SS4.p1.5.m5.1.1.1" xref="S5.SS4.p1.5.m5.1.1.1.cmml">=</mo><mn id="S5.SS4.p1.5.m5.1.1.3" xref="S5.SS4.p1.5.m5.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.5.m5.1b"><apply id="S5.SS4.p1.5.m5.1.1.cmml" xref="S5.SS4.p1.5.m5.1.1"><eq id="S5.SS4.p1.5.m5.1.1.1.cmml" xref="S5.SS4.p1.5.m5.1.1.1"></eq><apply id="S5.SS4.p1.5.m5.1.1.2.cmml" xref="S5.SS4.p1.5.m5.1.1.2"><csymbol cd="ambiguous" id="S5.SS4.p1.5.m5.1.1.2.1.cmml" xref="S5.SS4.p1.5.m5.1.1.2">subscript</csymbol><ci id="S5.SS4.p1.5.m5.1.1.2.2.cmml" xref="S5.SS4.p1.5.m5.1.1.2.2">𝑟</ci><ci id="S5.SS4.p1.5.m5.1.1.2.3.cmml" xref="S5.SS4.p1.5.m5.1.1.2.3">𝑚</ci></apply><cn type="integer" id="S5.SS4.p1.5.m5.1.1.3.cmml" xref="S5.SS4.p1.5.m5.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.5.m5.1c">r_{m}=3</annotation></semantics></math> or <math id="S5.SS4.p1.6.m6.1" class="ltx_Math" alttext="r_{m}=4" display="inline"><semantics id="S5.SS4.p1.6.m6.1a"><mrow id="S5.SS4.p1.6.m6.1.1" xref="S5.SS4.p1.6.m6.1.1.cmml"><msub id="S5.SS4.p1.6.m6.1.1.2" xref="S5.SS4.p1.6.m6.1.1.2.cmml"><mi id="S5.SS4.p1.6.m6.1.1.2.2" xref="S5.SS4.p1.6.m6.1.1.2.2.cmml">r</mi><mi id="S5.SS4.p1.6.m6.1.1.2.3" xref="S5.SS4.p1.6.m6.1.1.2.3.cmml">m</mi></msub><mo id="S5.SS4.p1.6.m6.1.1.1" xref="S5.SS4.p1.6.m6.1.1.1.cmml">=</mo><mn id="S5.SS4.p1.6.m6.1.1.3" xref="S5.SS4.p1.6.m6.1.1.3.cmml">4</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.6.m6.1b"><apply id="S5.SS4.p1.6.m6.1.1.cmml" xref="S5.SS4.p1.6.m6.1.1"><eq id="S5.SS4.p1.6.m6.1.1.1.cmml" xref="S5.SS4.p1.6.m6.1.1.1"></eq><apply id="S5.SS4.p1.6.m6.1.1.2.cmml" xref="S5.SS4.p1.6.m6.1.1.2"><csymbol cd="ambiguous" id="S5.SS4.p1.6.m6.1.1.2.1.cmml" xref="S5.SS4.p1.6.m6.1.1.2">subscript</csymbol><ci id="S5.SS4.p1.6.m6.1.1.2.2.cmml" xref="S5.SS4.p1.6.m6.1.1.2.2">𝑟</ci><ci id="S5.SS4.p1.6.m6.1.1.2.3.cmml" xref="S5.SS4.p1.6.m6.1.1.2.3">𝑚</ci></apply><cn type="integer" id="S5.SS4.p1.6.m6.1.1.3.cmml" xref="S5.SS4.p1.6.m6.1.1.3">4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.6.m6.1c">r_{m}=4</annotation></semantics></math> splits (i.e. for <math id="S5.SS4.p1.7.m7.1" class="ltx_Math" alttext="r=24" display="inline"><semantics id="S5.SS4.p1.7.m7.1a"><mrow id="S5.SS4.p1.7.m7.1.1" xref="S5.SS4.p1.7.m7.1.1.cmml"><mi id="S5.SS4.p1.7.m7.1.1.2" xref="S5.SS4.p1.7.m7.1.1.2.cmml">r</mi><mo id="S5.SS4.p1.7.m7.1.1.1" xref="S5.SS4.p1.7.m7.1.1.1.cmml">=</mo><mn id="S5.SS4.p1.7.m7.1.1.3" xref="S5.SS4.p1.7.m7.1.1.3.cmml">24</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.7.m7.1b"><apply id="S5.SS4.p1.7.m7.1.1.cmml" xref="S5.SS4.p1.7.m7.1.1"><eq id="S5.SS4.p1.7.m7.1.1.1.cmml" xref="S5.SS4.p1.7.m7.1.1.1"></eq><ci id="S5.SS4.p1.7.m7.1.1.2.cmml" xref="S5.SS4.p1.7.m7.1.1.2">𝑟</ci><cn type="integer" id="S5.SS4.p1.7.m7.1.1.3.cmml" xref="S5.SS4.p1.7.m7.1.1.3">24</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.7.m7.1c">r=24</annotation></semantics></math> or <math id="S5.SS4.p1.8.m8.1" class="ltx_Math" alttext="r=16" display="inline"><semantics id="S5.SS4.p1.8.m8.1a"><mrow id="S5.SS4.p1.8.m8.1.1" xref="S5.SS4.p1.8.m8.1.1.cmml"><mi id="S5.SS4.p1.8.m8.1.1.2" xref="S5.SS4.p1.8.m8.1.1.2.cmml">r</mi><mo id="S5.SS4.p1.8.m8.1.1.1" xref="S5.SS4.p1.8.m8.1.1.1.cmml">=</mo><mn id="S5.SS4.p1.8.m8.1.1.3" xref="S5.SS4.p1.8.m8.1.1.3.cmml">16</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.8.m8.1b"><apply id="S5.SS4.p1.8.m8.1.1.cmml" xref="S5.SS4.p1.8.m8.1.1"><eq id="S5.SS4.p1.8.m8.1.1.1.cmml" xref="S5.SS4.p1.8.m8.1.1.1"></eq><ci id="S5.SS4.p1.8.m8.1.1.2.cmml" xref="S5.SS4.p1.8.m8.1.1.2">𝑟</ci><cn type="integer" id="S5.SS4.p1.8.m8.1.1.3.cmml" xref="S5.SS4.p1.8.m8.1.1.3">16</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.8.m8.1c">r=16</annotation></semantics></math> respectively) is identical to the scores where schemata are split using the default <math id="S5.SS4.p1.9.m9.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.SS4.p1.9.m9.1a"><mi id="S5.SS4.p1.9.m9.1.1" xref="S5.SS4.p1.9.m9.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.9.m9.1b"><ci id="S5.SS4.p1.9.m9.1.1.cmml" xref="S5.SS4.p1.9.m9.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.9.m9.1c">r</annotation></semantics></math> with which <math id="S5.SS4.p1.10.m10.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S5.SS4.p1.10.m10.1a"><msub id="S5.SS4.p1.10.m10.1.1" xref="S5.SS4.p1.10.m10.1.1.cmml"><mtext id="S5.SS4.p1.10.m10.1.1.2" xref="S5.SS4.p1.10.m10.1.1.2a.cmml">FastRAT</mtext><mtext id="S5.SS4.p1.10.m10.1.1.3" xref="S5.SS4.p1.10.m10.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.10.m10.1b"><apply id="S5.SS4.p1.10.m10.1.1.cmml" xref="S5.SS4.p1.10.m10.1.1"><csymbol cd="ambiguous" id="S5.SS4.p1.10.m10.1.1.1.cmml" xref="S5.SS4.p1.10.m10.1.1">subscript</csymbol><ci id="S5.SS4.p1.10.m10.1.1.2a.cmml" xref="S5.SS4.p1.10.m10.1.1.2"><mtext id="S5.SS4.p1.10.m10.1.1.2.cmml" xref="S5.SS4.p1.10.m10.1.1.2">FastRAT</mtext></ci><ci id="S5.SS4.p1.10.m10.1.1.3a.cmml" xref="S5.SS4.p1.10.m10.1.1.3"><mtext mathsize="70%" id="S5.SS4.p1.10.m10.1.1.3.cmml" xref="S5.SS4.p1.10.m10.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.10.m10.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> has been trained.</p>
</div>
<figure id="S5.F1" class="ltx_figure"><img src="/html/2407.03227/assets/x1.png" id="S5.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="438" height="340" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>Execution accuracy scores on on the development set of S<span id="S5.F1.6.1" class="ltx_text ltx_font_smallcaps">pider</span> across different maximum numbers of columns per schema split, <math id="S5.F1.2.m1.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S5.F1.2.m1.1b"><mi id="S5.F1.2.m1.1.1" xref="S5.F1.2.m1.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S5.F1.2.m1.1c"><ci id="S5.F1.2.m1.1.1.cmml" xref="S5.F1.2.m1.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.F1.2.m1.1d">r</annotation></semantics></math>. The results of our approach, using <span id="S5.F1.7.2" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span>, are presented across different S<span id="S5.F1.8.3" class="ltx_text ltx_font_smallcaps">pider</span>-query difficulty levels.</figcaption>
</figure>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Discussion</h2>

<section id="S6.SS0.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Are there any theoretical performance upper limits for example selection using AST?</h5>

<div id="S6.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px1.p1.1" class="ltx_p">For each data instance in the development set of Spider, we compute the average AST similarity between the approximated query and each SQL query that is included (after example selection) in the corresponding prompt. In Table <a href="#S6.T6" title="Table 6 ‣ Are there any theoretical performance upper limits for example selection using AST? ‣ 6 Discussion ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>, we measure EX scores on the development set of S<span id="S6.SS0.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_smallcaps">pider</span>, across different AST-similarity intervals. We see an obvious correlation between execution accuracy and AST scores–execution accuracy is higher for higher AST scores. Besides highlighting an empirical, execution accuracy upper-bound, in the case of test questions whose SQL structure is well-covered (i.e. with high AST score) in the examples space, our approach can hint data instances that might be challenging for the current configuration, without even requiring to prompt the target LLM or executing the resulting SQL against a database instance.
Challenging data instances can be taken into consideration with respect to the existence of insufficient examples in <math id="S6.SS0.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="\mathbb{X}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.1.m1.1a"><mi id="S6.SS0.SSS0.Px1.p1.1.m1.1.1" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.cmml">𝕏</mi><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.1.m1.1b"><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1">𝕏</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.1.m1.1c">\mathbb{X}</annotation></semantics></math> to support the expected SQL structure or a potentially harmful approximator.</p>
</div>
<figure id="S6.T6" class="ltx_table">
<table id="S6.T6.25" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S6.T6.25.26.1" class="ltx_tr">
<th id="S6.T6.25.26.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" rowspan="2"><span id="S6.T6.25.26.1.1.1" class="ltx_text ltx_font_bold">AST Interval</span></th>
<th id="S6.T6.25.26.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" colspan="3"><span id="S6.T6.25.26.1.2.1" class="ltx_text ltx_font_bold">Approximator</span></th>
</tr>
<tr id="S6.T6.1.1" class="ltx_tr">
<th id="S6.T6.1.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><span id="S6.T6.1.1.2.1" class="ltx_text ltx_font_italic">Oracle</span></th>
<th id="S6.T6.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t">Graphix</th>
<th id="S6.T6.1.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><math id="S6.T6.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S6.T6.1.1.1.m1.1a"><msub id="S6.T6.1.1.1.m1.1.1" xref="S6.T6.1.1.1.m1.1.1.cmml"><mtext id="S6.T6.1.1.1.m1.1.1.2" xref="S6.T6.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S6.T6.1.1.1.m1.1.1.3" xref="S6.T6.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S6.T6.1.1.1.m1.1b"><apply id="S6.T6.1.1.1.m1.1.1.cmml" xref="S6.T6.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S6.T6.1.1.1.m1.1.1.1.cmml" xref="S6.T6.1.1.1.m1.1.1">subscript</csymbol><ci id="S6.T6.1.1.1.m1.1.1.2a.cmml" xref="S6.T6.1.1.1.m1.1.1.2"><mtext id="S6.T6.1.1.1.m1.1.1.2.cmml" xref="S6.T6.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S6.T6.1.1.1.m1.1.1.3a.cmml" xref="S6.T6.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="S6.T6.1.1.1.m1.1.1.3.cmml" xref="S6.T6.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
</tr>
<tr id="S6.T6.5.5" class="ltx_tr">
<th id="S6.T6.2.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_t"><math id="S6.T6.2.2.1.m1.2" class="ltx_Math" alttext="[0.0,1.0]" display="inline"><semantics id="S6.T6.2.2.1.m1.2a"><mrow id="S6.T6.2.2.1.m1.2.3.2" xref="S6.T6.2.2.1.m1.2.3.1.cmml"><mo stretchy="false" id="S6.T6.2.2.1.m1.2.3.2.1" xref="S6.T6.2.2.1.m1.2.3.1.cmml">[</mo><mn id="S6.T6.2.2.1.m1.1.1" xref="S6.T6.2.2.1.m1.1.1.cmml">0.0</mn><mo id="S6.T6.2.2.1.m1.2.3.2.2" xref="S6.T6.2.2.1.m1.2.3.1.cmml">,</mo><mn id="S6.T6.2.2.1.m1.2.2" xref="S6.T6.2.2.1.m1.2.2.cmml">1.0</mn><mo stretchy="false" id="S6.T6.2.2.1.m1.2.3.2.3" xref="S6.T6.2.2.1.m1.2.3.1.cmml">]</mo></mrow><annotation-xml encoding="MathML-Content" id="S6.T6.2.2.1.m1.2b"><interval closure="closed" id="S6.T6.2.2.1.m1.2.3.1.cmml" xref="S6.T6.2.2.1.m1.2.3.2"><cn type="float" id="S6.T6.2.2.1.m1.1.1.cmml" xref="S6.T6.2.2.1.m1.1.1">0.0</cn><cn type="float" id="S6.T6.2.2.1.m1.2.2.cmml" xref="S6.T6.2.2.1.m1.2.2">1.0</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.2.2.1.m1.2c">[0.0,1.0]</annotation></semantics></math></th>
<th id="S6.T6.3.3.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><math id="S6.T6.3.3.2.m1.1" class="ltx_Math" alttext="84.6" display="inline"><semantics id="S6.T6.3.3.2.m1.1a"><mn id="S6.T6.3.3.2.m1.1.1" xref="S6.T6.3.3.2.m1.1.1.cmml">84.6</mn><annotation-xml encoding="MathML-Content" id="S6.T6.3.3.2.m1.1b"><cn type="float" id="S6.T6.3.3.2.m1.1.1.cmml" xref="S6.T6.3.3.2.m1.1.1">84.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.3.3.2.m1.1c">84.6</annotation></semantics></math></th>
<th id="S6.T6.4.4.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><math id="S6.T6.4.4.3.m1.1" class="ltx_Math" alttext="83.0" display="inline"><semantics id="S6.T6.4.4.3.m1.1a"><mn id="S6.T6.4.4.3.m1.1.1" xref="S6.T6.4.4.3.m1.1.1.cmml">83.0</mn><annotation-xml encoding="MathML-Content" id="S6.T6.4.4.3.m1.1b"><cn type="float" id="S6.T6.4.4.3.m1.1.1.cmml" xref="S6.T6.4.4.3.m1.1.1">83.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.4.4.3.m1.1c">83.0</annotation></semantics></math></th>
<th id="S6.T6.5.5.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_t"><math id="S6.T6.5.5.4.m1.1" class="ltx_Math" alttext="82.0" display="inline"><semantics id="S6.T6.5.5.4.m1.1a"><mn id="S6.T6.5.5.4.m1.1.1" xref="S6.T6.5.5.4.m1.1.1.cmml">82.0</mn><annotation-xml encoding="MathML-Content" id="S6.T6.5.5.4.m1.1b"><cn type="float" id="S6.T6.5.5.4.m1.1.1.cmml" xref="S6.T6.5.5.4.m1.1.1">82.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.5.5.4.m1.1c">82.0</annotation></semantics></math></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S6.T6.9.9" class="ltx_tr">
<th id="S6.T6.6.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><math id="S6.T6.6.6.1.m1.2" class="ltx_Math" alttext="[0.95,1.0]" display="inline"><semantics id="S6.T6.6.6.1.m1.2a"><mrow id="S6.T6.6.6.1.m1.2.3.2" xref="S6.T6.6.6.1.m1.2.3.1.cmml"><mo stretchy="false" id="S6.T6.6.6.1.m1.2.3.2.1" xref="S6.T6.6.6.1.m1.2.3.1.cmml">[</mo><mn id="S6.T6.6.6.1.m1.1.1" xref="S6.T6.6.6.1.m1.1.1.cmml">0.95</mn><mo id="S6.T6.6.6.1.m1.2.3.2.2" xref="S6.T6.6.6.1.m1.2.3.1.cmml">,</mo><mn id="S6.T6.6.6.1.m1.2.2" xref="S6.T6.6.6.1.m1.2.2.cmml">1.0</mn><mo stretchy="false" id="S6.T6.6.6.1.m1.2.3.2.3" xref="S6.T6.6.6.1.m1.2.3.1.cmml">]</mo></mrow><annotation-xml encoding="MathML-Content" id="S6.T6.6.6.1.m1.2b"><interval closure="closed" id="S6.T6.6.6.1.m1.2.3.1.cmml" xref="S6.T6.6.6.1.m1.2.3.2"><cn type="float" id="S6.T6.6.6.1.m1.1.1.cmml" xref="S6.T6.6.6.1.m1.1.1">0.95</cn><cn type="float" id="S6.T6.6.6.1.m1.2.2.cmml" xref="S6.T6.6.6.1.m1.2.2">1.0</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.6.6.1.m1.2c">[0.95,1.0]</annotation></semantics></math></th>
<td id="S6.T6.7.7.2" class="ltx_td ltx_align_center ltx_border_t"><math id="S6.T6.7.7.2.m1.1" class="ltx_Math" alttext="90.8" display="inline"><semantics id="S6.T6.7.7.2.m1.1a"><mn id="S6.T6.7.7.2.m1.1.1" xref="S6.T6.7.7.2.m1.1.1.cmml">90.8</mn><annotation-xml encoding="MathML-Content" id="S6.T6.7.7.2.m1.1b"><cn type="float" id="S6.T6.7.7.2.m1.1.1.cmml" xref="S6.T6.7.7.2.m1.1.1">90.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.7.7.2.m1.1c">90.8</annotation></semantics></math></td>
<td id="S6.T6.8.8.3" class="ltx_td ltx_align_center ltx_border_t"><math id="S6.T6.8.8.3.m1.1" class="ltx_Math" alttext="88.4" display="inline"><semantics id="S6.T6.8.8.3.m1.1a"><mn id="S6.T6.8.8.3.m1.1.1" xref="S6.T6.8.8.3.m1.1.1.cmml">88.4</mn><annotation-xml encoding="MathML-Content" id="S6.T6.8.8.3.m1.1b"><cn type="float" id="S6.T6.8.8.3.m1.1.1.cmml" xref="S6.T6.8.8.3.m1.1.1">88.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.8.8.3.m1.1c">88.4</annotation></semantics></math></td>
<td id="S6.T6.9.9.4" class="ltx_td ltx_align_center ltx_border_t"><math id="S6.T6.9.9.4.m1.1" class="ltx_Math" alttext="88.7" display="inline"><semantics id="S6.T6.9.9.4.m1.1a"><mn id="S6.T6.9.9.4.m1.1.1" xref="S6.T6.9.9.4.m1.1.1.cmml">88.7</mn><annotation-xml encoding="MathML-Content" id="S6.T6.9.9.4.m1.1b"><cn type="float" id="S6.T6.9.9.4.m1.1.1.cmml" xref="S6.T6.9.9.4.m1.1.1">88.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.9.9.4.m1.1c">88.7</annotation></semantics></math></td>
</tr>
<tr id="S6.T6.13.13" class="ltx_tr">
<th id="S6.T6.10.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S6.T6.10.10.1.m1.2" class="ltx_Math" alttext="[0.9,0.95)" display="inline"><semantics id="S6.T6.10.10.1.m1.2a"><mrow id="S6.T6.10.10.1.m1.2.3.2" xref="S6.T6.10.10.1.m1.2.3.1.cmml"><mo stretchy="false" id="S6.T6.10.10.1.m1.2.3.2.1" xref="S6.T6.10.10.1.m1.2.3.1.cmml">[</mo><mn id="S6.T6.10.10.1.m1.1.1" xref="S6.T6.10.10.1.m1.1.1.cmml">0.9</mn><mo id="S6.T6.10.10.1.m1.2.3.2.2" xref="S6.T6.10.10.1.m1.2.3.1.cmml">,</mo><mn id="S6.T6.10.10.1.m1.2.2" xref="S6.T6.10.10.1.m1.2.2.cmml">0.95</mn><mo stretchy="false" id="S6.T6.10.10.1.m1.2.3.2.3" xref="S6.T6.10.10.1.m1.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S6.T6.10.10.1.m1.2b"><interval closure="closed-open" id="S6.T6.10.10.1.m1.2.3.1.cmml" xref="S6.T6.10.10.1.m1.2.3.2"><cn type="float" id="S6.T6.10.10.1.m1.1.1.cmml" xref="S6.T6.10.10.1.m1.1.1">0.9</cn><cn type="float" id="S6.T6.10.10.1.m1.2.2.cmml" xref="S6.T6.10.10.1.m1.2.2">0.95</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.10.10.1.m1.2c">[0.9,0.95)</annotation></semantics></math></th>
<td id="S6.T6.11.11.2" class="ltx_td ltx_align_center"><math id="S6.T6.11.11.2.m1.1" class="ltx_Math" alttext="80.7" display="inline"><semantics id="S6.T6.11.11.2.m1.1a"><mn id="S6.T6.11.11.2.m1.1.1" xref="S6.T6.11.11.2.m1.1.1.cmml">80.7</mn><annotation-xml encoding="MathML-Content" id="S6.T6.11.11.2.m1.1b"><cn type="float" id="S6.T6.11.11.2.m1.1.1.cmml" xref="S6.T6.11.11.2.m1.1.1">80.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.11.11.2.m1.1c">80.7</annotation></semantics></math></td>
<td id="S6.T6.12.12.3" class="ltx_td ltx_align_center"><math id="S6.T6.12.12.3.m1.1" class="ltx_Math" alttext="74.1" display="inline"><semantics id="S6.T6.12.12.3.m1.1a"><mn id="S6.T6.12.12.3.m1.1.1" xref="S6.T6.12.12.3.m1.1.1.cmml">74.1</mn><annotation-xml encoding="MathML-Content" id="S6.T6.12.12.3.m1.1b"><cn type="float" id="S6.T6.12.12.3.m1.1.1.cmml" xref="S6.T6.12.12.3.m1.1.1">74.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.12.12.3.m1.1c">74.1</annotation></semantics></math></td>
<td id="S6.T6.13.13.4" class="ltx_td ltx_align_center"><math id="S6.T6.13.13.4.m1.1" class="ltx_Math" alttext="76.8" display="inline"><semantics id="S6.T6.13.13.4.m1.1a"><mn id="S6.T6.13.13.4.m1.1.1" xref="S6.T6.13.13.4.m1.1.1.cmml">76.8</mn><annotation-xml encoding="MathML-Content" id="S6.T6.13.13.4.m1.1b"><cn type="float" id="S6.T6.13.13.4.m1.1.1.cmml" xref="S6.T6.13.13.4.m1.1.1">76.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.13.13.4.m1.1c">76.8</annotation></semantics></math></td>
</tr>
<tr id="S6.T6.17.17" class="ltx_tr">
<th id="S6.T6.14.14.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S6.T6.14.14.1.m1.2" class="ltx_Math" alttext="[0.85,0.9)" display="inline"><semantics id="S6.T6.14.14.1.m1.2a"><mrow id="S6.T6.14.14.1.m1.2.3.2" xref="S6.T6.14.14.1.m1.2.3.1.cmml"><mo stretchy="false" id="S6.T6.14.14.1.m1.2.3.2.1" xref="S6.T6.14.14.1.m1.2.3.1.cmml">[</mo><mn id="S6.T6.14.14.1.m1.1.1" xref="S6.T6.14.14.1.m1.1.1.cmml">0.85</mn><mo id="S6.T6.14.14.1.m1.2.3.2.2" xref="S6.T6.14.14.1.m1.2.3.1.cmml">,</mo><mn id="S6.T6.14.14.1.m1.2.2" xref="S6.T6.14.14.1.m1.2.2.cmml">0.9</mn><mo stretchy="false" id="S6.T6.14.14.1.m1.2.3.2.3" xref="S6.T6.14.14.1.m1.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S6.T6.14.14.1.m1.2b"><interval closure="closed-open" id="S6.T6.14.14.1.m1.2.3.1.cmml" xref="S6.T6.14.14.1.m1.2.3.2"><cn type="float" id="S6.T6.14.14.1.m1.1.1.cmml" xref="S6.T6.14.14.1.m1.1.1">0.85</cn><cn type="float" id="S6.T6.14.14.1.m1.2.2.cmml" xref="S6.T6.14.14.1.m1.2.2">0.9</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.14.14.1.m1.2c">[0.85,0.9)</annotation></semantics></math></th>
<td id="S6.T6.15.15.2" class="ltx_td ltx_align_center"><math id="S6.T6.15.15.2.m1.1" class="ltx_Math" alttext="73.0" display="inline"><semantics id="S6.T6.15.15.2.m1.1a"><mn id="S6.T6.15.15.2.m1.1.1" xref="S6.T6.15.15.2.m1.1.1.cmml">73.0</mn><annotation-xml encoding="MathML-Content" id="S6.T6.15.15.2.m1.1b"><cn type="float" id="S6.T6.15.15.2.m1.1.1.cmml" xref="S6.T6.15.15.2.m1.1.1">73.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.15.15.2.m1.1c">73.0</annotation></semantics></math></td>
<td id="S6.T6.16.16.3" class="ltx_td ltx_align_center"><math id="S6.T6.16.16.3.m1.1" class="ltx_Math" alttext="68.3" display="inline"><semantics id="S6.T6.16.16.3.m1.1a"><mn id="S6.T6.16.16.3.m1.1.1" xref="S6.T6.16.16.3.m1.1.1.cmml">68.3</mn><annotation-xml encoding="MathML-Content" id="S6.T6.16.16.3.m1.1b"><cn type="float" id="S6.T6.16.16.3.m1.1.1.cmml" xref="S6.T6.16.16.3.m1.1.1">68.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.16.16.3.m1.1c">68.3</annotation></semantics></math></td>
<td id="S6.T6.17.17.4" class="ltx_td ltx_align_center"><math id="S6.T6.17.17.4.m1.1" class="ltx_Math" alttext="59.4" display="inline"><semantics id="S6.T6.17.17.4.m1.1a"><mn id="S6.T6.17.17.4.m1.1.1" xref="S6.T6.17.17.4.m1.1.1.cmml">59.4</mn><annotation-xml encoding="MathML-Content" id="S6.T6.17.17.4.m1.1b"><cn type="float" id="S6.T6.17.17.4.m1.1.1.cmml" xref="S6.T6.17.17.4.m1.1.1">59.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.17.17.4.m1.1c">59.4</annotation></semantics></math></td>
</tr>
<tr id="S6.T6.21.21" class="ltx_tr">
<th id="S6.T6.18.18.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><math id="S6.T6.18.18.1.m1.2" class="ltx_Math" alttext="[0.8,0.85)" display="inline"><semantics id="S6.T6.18.18.1.m1.2a"><mrow id="S6.T6.18.18.1.m1.2.3.2" xref="S6.T6.18.18.1.m1.2.3.1.cmml"><mo stretchy="false" id="S6.T6.18.18.1.m1.2.3.2.1" xref="S6.T6.18.18.1.m1.2.3.1.cmml">[</mo><mn id="S6.T6.18.18.1.m1.1.1" xref="S6.T6.18.18.1.m1.1.1.cmml">0.8</mn><mo id="S6.T6.18.18.1.m1.2.3.2.2" xref="S6.T6.18.18.1.m1.2.3.1.cmml">,</mo><mn id="S6.T6.18.18.1.m1.2.2" xref="S6.T6.18.18.1.m1.2.2.cmml">0.85</mn><mo stretchy="false" id="S6.T6.18.18.1.m1.2.3.2.3" xref="S6.T6.18.18.1.m1.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S6.T6.18.18.1.m1.2b"><interval closure="closed-open" id="S6.T6.18.18.1.m1.2.3.1.cmml" xref="S6.T6.18.18.1.m1.2.3.2"><cn type="float" id="S6.T6.18.18.1.m1.1.1.cmml" xref="S6.T6.18.18.1.m1.1.1">0.8</cn><cn type="float" id="S6.T6.18.18.1.m1.2.2.cmml" xref="S6.T6.18.18.1.m1.2.2">0.85</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.18.18.1.m1.2c">[0.8,0.85)</annotation></semantics></math></th>
<td id="S6.T6.19.19.2" class="ltx_td ltx_align_center"><math id="S6.T6.19.19.2.m1.1" class="ltx_Math" alttext="62.4" display="inline"><semantics id="S6.T6.19.19.2.m1.1a"><mn id="S6.T6.19.19.2.m1.1.1" xref="S6.T6.19.19.2.m1.1.1.cmml">62.4</mn><annotation-xml encoding="MathML-Content" id="S6.T6.19.19.2.m1.1b"><cn type="float" id="S6.T6.19.19.2.m1.1.1.cmml" xref="S6.T6.19.19.2.m1.1.1">62.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.19.19.2.m1.1c">62.4</annotation></semantics></math></td>
<td id="S6.T6.20.20.3" class="ltx_td ltx_align_center"><math id="S6.T6.20.20.3.m1.1" class="ltx_Math" alttext="66.8" display="inline"><semantics id="S6.T6.20.20.3.m1.1a"><mn id="S6.T6.20.20.3.m1.1.1" xref="S6.T6.20.20.3.m1.1.1.cmml">66.8</mn><annotation-xml encoding="MathML-Content" id="S6.T6.20.20.3.m1.1b"><cn type="float" id="S6.T6.20.20.3.m1.1.1.cmml" xref="S6.T6.20.20.3.m1.1.1">66.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.20.20.3.m1.1c">66.8</annotation></semantics></math></td>
<td id="S6.T6.21.21.4" class="ltx_td ltx_align_center"><math id="S6.T6.21.21.4.m1.1" class="ltx_Math" alttext="63.5" display="inline"><semantics id="S6.T6.21.21.4.m1.1a"><mn id="S6.T6.21.21.4.m1.1.1" xref="S6.T6.21.21.4.m1.1.1.cmml">63.5</mn><annotation-xml encoding="MathML-Content" id="S6.T6.21.21.4.m1.1b"><cn type="float" id="S6.T6.21.21.4.m1.1.1.cmml" xref="S6.T6.21.21.4.m1.1.1">63.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.21.21.4.m1.1c">63.5</annotation></semantics></math></td>
</tr>
<tr id="S6.T6.25.25" class="ltx_tr">
<th id="S6.T6.22.22.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb"><math id="S6.T6.22.22.1.m1.2" class="ltx_Math" alttext="[0.0,0.8)" display="inline"><semantics id="S6.T6.22.22.1.m1.2a"><mrow id="S6.T6.22.22.1.m1.2.3.2" xref="S6.T6.22.22.1.m1.2.3.1.cmml"><mo stretchy="false" id="S6.T6.22.22.1.m1.2.3.2.1" xref="S6.T6.22.22.1.m1.2.3.1.cmml">[</mo><mn id="S6.T6.22.22.1.m1.1.1" xref="S6.T6.22.22.1.m1.1.1.cmml">0.0</mn><mo id="S6.T6.22.22.1.m1.2.3.2.2" xref="S6.T6.22.22.1.m1.2.3.1.cmml">,</mo><mn id="S6.T6.22.22.1.m1.2.2" xref="S6.T6.22.22.1.m1.2.2.cmml">0.8</mn><mo stretchy="false" id="S6.T6.22.22.1.m1.2.3.2.3" xref="S6.T6.22.22.1.m1.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S6.T6.22.22.1.m1.2b"><interval closure="closed-open" id="S6.T6.22.22.1.m1.2.3.1.cmml" xref="S6.T6.22.22.1.m1.2.3.2"><cn type="float" id="S6.T6.22.22.1.m1.1.1.cmml" xref="S6.T6.22.22.1.m1.1.1">0.0</cn><cn type="float" id="S6.T6.22.22.1.m1.2.2.cmml" xref="S6.T6.22.22.1.m1.2.2">0.8</cn></interval></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.22.22.1.m1.2c">[0.0,0.8)</annotation></semantics></math></th>
<td id="S6.T6.23.23.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="S6.T6.23.23.2.m1.1" class="ltx_Math" alttext="50.0" display="inline"><semantics id="S6.T6.23.23.2.m1.1a"><mn id="S6.T6.23.23.2.m1.1.1" xref="S6.T6.23.23.2.m1.1.1.cmml">50.0</mn><annotation-xml encoding="MathML-Content" id="S6.T6.23.23.2.m1.1b"><cn type="float" id="S6.T6.23.23.2.m1.1.1.cmml" xref="S6.T6.23.23.2.m1.1.1">50.0</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.23.23.2.m1.1c">50.0</annotation></semantics></math></td>
<td id="S6.T6.24.24.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="S6.T6.24.24.3.m1.1" class="ltx_Math" alttext="54.1" display="inline"><semantics id="S6.T6.24.24.3.m1.1a"><mn id="S6.T6.24.24.3.m1.1.1" xref="S6.T6.24.24.3.m1.1.1.cmml">54.1</mn><annotation-xml encoding="MathML-Content" id="S6.T6.24.24.3.m1.1b"><cn type="float" id="S6.T6.24.24.3.m1.1.1.cmml" xref="S6.T6.24.24.3.m1.1.1">54.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.24.24.3.m1.1c">54.1</annotation></semantics></math></td>
<td id="S6.T6.25.25.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="S6.T6.25.25.4.m1.1" class="ltx_Math" alttext="58.7" display="inline"><semantics id="S6.T6.25.25.4.m1.1a"><mn id="S6.T6.25.25.4.m1.1.1" xref="S6.T6.25.25.4.m1.1.1.cmml">58.7</mn><annotation-xml encoding="MathML-Content" id="S6.T6.25.25.4.m1.1b"><cn type="float" id="S6.T6.25.25.4.m1.1.1.cmml" xref="S6.T6.25.25.4.m1.1.1">58.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.T6.25.25.4.m1.1c">58.7</annotation></semantics></math></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 6: </span>EX scores on the S<span id="S6.T6.28.1" class="ltx_text ltx_font_smallcaps">pider</span> development set using <span id="S6.T6.29.2" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span>, across different average AST-similarity intervals.</figcaption>
</figure>
</section>
<section id="S6.SS0.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Is the choice of approximator critical?</h5>

<div id="S6.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px2.p1.4" class="ltx_p">Although our AST re-ranking and schema selection benefit from more accurate SQL predicted by a stronger approximator, the choice of approximator depends on the desired trade-off between effectiveness and efficiency in practice. <math id="S6.SS0.SSS0.Px2.p1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="S6.SS0.SSS0.Px2.p1.1.m1.1a"><msub id="S6.SS0.SSS0.Px2.p1.1.m1.1.1" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.cmml"><mtext id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.2" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.3" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px2.p1.1.m1.1b"><apply id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.cmml" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.1.cmml" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1">subscript</csymbol><ci id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.2a.cmml" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.2"><mtext id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.2.cmml" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.3a.cmml" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.3"><mtext mathsize="70%" id="S6.SS0.SSS0.Px2.p1.1.m1.1.1.3.cmml" xref="S6.SS0.SSS0.Px2.p1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px2.p1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> is over <math id="S6.SS0.SSS0.Px2.p1.2.m2.1" class="ltx_Math" alttext="600" display="inline"><semantics id="S6.SS0.SSS0.Px2.p1.2.m2.1a"><mn id="S6.SS0.SSS0.Px2.p1.2.m2.1.1" xref="S6.SS0.SSS0.Px2.p1.2.m2.1.1.cmml">600</mn><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px2.p1.2.m2.1b"><cn type="integer" id="S6.SS0.SSS0.Px2.p1.2.m2.1.1.cmml" xref="S6.SS0.SSS0.Px2.p1.2.m2.1.1">600</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px2.p1.2.m2.1c">600</annotation></semantics></math> times faster than Graphix-T5 <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib13" title="" class="ltx_ref">2023b</a>)</cite> on an A100 <math id="S6.SS0.SSS0.Px2.p1.3.m3.1" class="ltx_Math" alttext="80" display="inline"><semantics id="S6.SS0.SSS0.Px2.p1.3.m3.1a"><mn id="S6.SS0.SSS0.Px2.p1.3.m3.1.1" xref="S6.SS0.SSS0.Px2.p1.3.m3.1.1.cmml">80</mn><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px2.p1.3.m3.1b"><cn type="integer" id="S6.SS0.SSS0.Px2.p1.3.m3.1.1.cmml" xref="S6.SS0.SSS0.Px2.p1.3.m3.1.1">80</cn></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px2.p1.3.m3.1c">80</annotation></semantics></math>G GPU, while the resulting difference, within our framework, in EX on the S<span id="S6.SS0.SSS0.Px2.p1.4.1" class="ltx_text ltx_font_smallcaps">pider</span> development set is <math id="S6.SS0.SSS0.Px2.p1.4.m4.1" class="ltx_Math" alttext="\leq 1\%" display="inline"><semantics id="S6.SS0.SSS0.Px2.p1.4.m4.1a"><mrow id="S6.SS0.SSS0.Px2.p1.4.m4.1.1" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.cmml"><mi id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.2" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.2.cmml"></mi><mo id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.1" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.1.cmml">≤</mo><mrow id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.cmml"><mn id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.2" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.2.cmml">1</mn><mo id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.1" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.1.cmml">%</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px2.p1.4.m4.1b"><apply id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.cmml" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1"><leq id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.1.cmml" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.1"></leq><csymbol cd="latexml" id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.2.cmml" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.2">absent</csymbol><apply id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.cmml" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3"><csymbol cd="latexml" id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.1.cmml" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.1">percent</csymbol><cn type="integer" id="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.2.cmml" xref="S6.SS0.SSS0.Px2.p1.4.m4.1.1.3.2">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px2.p1.4.m4.1c">\leq 1\%</annotation></semantics></math> (Table <a href="#S5.T5" title="Table 5 ‣ 5.3.2 Ablation Study ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>).</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Does schema selection improve the performance?</h5>

<div id="S6.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px3.p1.1" class="ltx_p">In Table <a href="#S5.T5" title="Table 5 ‣ 5.3.2 Ablation Study ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, we noted that performing schema selection (i.e. SS) without DB value selection does not necessarily lead to performance improvements. This is in partial agreement with <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib25" title="" class="ltx_ref">Sun et al.</a></cite> that hard column selection can be harmful for the end-to-end performance, and can be attributed to the drop of recall, when less capable approximators are involved. Nonetheless, as we note in Section <a href="#S5.SS3.SSS2" title="5.3.2 Ablation Study ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5.3.2</span></a>, the combination of schema and value selection (SVS) can consistently improve EX and EM, while significantly reducing the LLM token-processing cost due to shortened schema.</p>
</div>
</section>
</section>
<section id="S7" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">7 </span>Related Work</h2>

<div id="S7.p1" class="ltx_para">
<p id="S7.p1.1" class="ltx_p">Significant number of recent works have looked at how LLMs can be employed in Text-to-SQL scenarios <cite class="ltx_cite ltx_citemacro_cite">Rajkumar et al. (<a href="#bib.bib20" title="" class="ltx_ref">2022</a>); Chang and Fosler-Lussier (<a href="#bib.bib3" title="" class="ltx_ref">2023</a>); Liu et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>); Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>); Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>); Guo et al. (<a href="#bib.bib11" title="" class="ltx_ref">2024</a>)</cite>.
More recent works have looked at how incorporating examples in the prompt could benefit the performance of LLMs in the end task <cite class="ltx_cite ltx_citemacro_cite">Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>); Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>); Guo et al. (<a href="#bib.bib11" title="" class="ltx_ref">2024</a>); Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>.
In spite of its underlying benefits, conventional solutions for example selection have focused on retrieving pairs using question similarity <cite class="ltx_cite ltx_citemacro_cite">Nan et al. (<a href="#bib.bib16" title="" class="ltx_ref">2023</a>)</cite>. Other approaches have sought to <span id="S7.p1.1.1" class="ltx_text ltx_font_italic">approximate</span> expected SQL queries, and either directly use these approximations in the prompt, in a few-shot setting <cite class="ltx_cite ltx_citemacro_cite">Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite> or to filter candidate (question, SQL) pairs by taking into consideration the similarity of their corresponding SQL query skeletons <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib12" title="" class="ltx_ref">2023a</a>)</cite> against the skeleton of the approximated SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>. We argue that such example selection strategies can result in information loss, and we propose an approach for re-ranking examples using similarity of normalised SQL ASTs.</p>
</div>
<div id="S7.p2" class="ltx_para">
<p id="S7.p2.1" class="ltx_p">The benefits of schema selection for Text-to-SQL have been highlighted across the relevant bibliography <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib29" title="" class="ltx_ref">2020</a>); Li et al. (<a href="#bib.bib13" title="" class="ltx_ref">2023b</a>); Pourreza and Rafiei (<a href="#bib.bib18" title="" class="ltx_ref">2023</a>)</cite>. From the LLMs perspective, pruning schema elements from the prompts has been usually leading to performance degradation <cite class="ltx_cite ltx_citemacro_cite">Sun et al. (<a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>.
Inspired by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib10" title="" class="ltx_ref">Gao et al.</a></cite>, we compute a preliminary query for a given <math id="S7.p2.1.m1.2" class="ltx_Math" alttext="(\mathbf{q},\mathbf{D})" display="inline"><semantics id="S7.p2.1.m1.2a"><mrow id="S7.p2.1.m1.2.3.2" xref="S7.p2.1.m1.2.3.1.cmml"><mo stretchy="false" id="S7.p2.1.m1.2.3.2.1" xref="S7.p2.1.m1.2.3.1.cmml">(</mo><mi id="S7.p2.1.m1.1.1" xref="S7.p2.1.m1.1.1.cmml">𝐪</mi><mo id="S7.p2.1.m1.2.3.2.2" xref="S7.p2.1.m1.2.3.1.cmml">,</mo><mi id="S7.p2.1.m1.2.2" xref="S7.p2.1.m1.2.2.cmml">𝐃</mi><mo stretchy="false" id="S7.p2.1.m1.2.3.2.3" xref="S7.p2.1.m1.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S7.p2.1.m1.2b"><interval closure="open" id="S7.p2.1.m1.2.3.1.cmml" xref="S7.p2.1.m1.2.3.2"><ci id="S7.p2.1.m1.1.1.cmml" xref="S7.p2.1.m1.1.1">𝐪</ci><ci id="S7.p2.1.m1.2.2.cmml" xref="S7.p2.1.m1.2.2">𝐃</ci></interval></annotation-xml><annotation encoding="application/x-tex" id="S7.p2.1.m1.2c">(\mathbf{q},\mathbf{D})</annotation></semantics></math> by we adapting FastRAT <cite class="ltx_cite ltx_citemacro_cite">Vougiouklis et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite>, to the requirements of processing longer schemata, in a parallelised manner. We couple the resulting <span id="S7.p2.1.1" class="ltx_text ltx_font_italic">approximator</span> with a sparse retriever, and we propose a dynamic strategy for reducing the computational cost of the task while achieving performance improvements.</p>
</div>
</section>
<section id="S8" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">8 </span>Conclusion</h2>

<div id="S8.p1" class="ltx_para">
<p id="S8.p1.1" class="ltx_p">In this paper, we augment LLMs for Text-to-SQL semantic parsing by selecting suitable examples and database information. We present a novel AST-based metric to rank examples by similarity of SQL queries. Our hybrid search strategy for schema selection reuses a preliminary query to reduce irrelevant schema elements while maintaining high recall. Extensive experiments demonstrate that our AST-based ranking outperforms previous approaches of example selection and that a symbiotic combination of schema and value selection can further enhance the end-to-end performance of both closed- and open-source LLM solutions.</p>
</div>
</section>
<section id="Sx1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Limitations</h2>

<div id="Sx1.p1" class="ltx_para">
<p id="Sx1.p1.1" class="ltx_p">There are limitations with regards to both of our example selection and schema selection. Our AST-based ranking can be biased when an approximated SQL deviates significantly from structurally correct answers. To address the failure of approximators, a future direction is to sensibly diversify selected examples such that LLMs can generalise compositionally. As for schema selection, our semantic search relies on an approximator which is essentially a parser with high precision in schema linking but lack mechanisms to control recall as a standalone model. Therefore, it is worth extending cross-encoder architecture such as <span id="Sx1.p1.1.1" class="ltx_text ltx_markedasmath">FastRAT</span> to support ranking schema elements while being a SQL approximator in the meantime.</p>
</div>
<div id="Sx1.p2" class="ltx_para">
<p id="Sx1.p2.1" class="ltx_p">We demonstrate that schema splitting strategies within our framework can be applied across various numbers of splits without noticeable performance degradation. Nonetheless, given the lack of available datasets that incorporate longer commercial schemata, we focus our experiments on the cross-database setting provided by CS<span id="Sx1.p2.1.1" class="ltx_text ltx_font_smallcaps">pider</span> and S<span id="Sx1.p2.1.2" class="ltx_text ltx_font_smallcaps">pider</span> variants.</p>
</div>
</section>
<section id="Sx2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Ethics Statement</h2>

<div id="Sx2.p1" class="ltx_para">
<p id="Sx2.p1.1" class="ltx_p">We do not make use of any private, proprietary, or sensitive data. <math id="Sx2.p1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="Sx2.p1.1.m1.1a"><msub id="Sx2.p1.1.m1.1.1" xref="Sx2.p1.1.m1.1.1.cmml"><mtext id="Sx2.p1.1.m1.1.1.2" xref="Sx2.p1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="Sx2.p1.1.m1.1.1.3" xref="Sx2.p1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="Sx2.p1.1.m1.1b"><apply id="Sx2.p1.1.m1.1.1.cmml" xref="Sx2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="Sx2.p1.1.m1.1.1.1.cmml" xref="Sx2.p1.1.m1.1.1">subscript</csymbol><ci id="Sx2.p1.1.m1.1.1.2a.cmml" xref="Sx2.p1.1.m1.1.1.2"><mtext id="Sx2.p1.1.m1.1.1.2.cmml" xref="Sx2.p1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="Sx2.p1.1.m1.1.1.3a.cmml" xref="Sx2.p1.1.m1.1.1.3"><mtext mathsize="70%" id="Sx2.p1.1.m1.1.1.3.cmml" xref="Sx2.p1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="Sx2.p1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> is trained on publicly available Text-to-SQL datasets, using publicly available encoder-models as base. Our framework for retrieval-augmented generation builds on-top of large, pre-trained language models, which may have been trained using proprietary data (e.g. in the case of the OpenAI models). Given the nature of pre-training schemes, it is possible that our system could carry forward biases present in the datasets and/or the involved LLMs.</p>
</div>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">An et al. (2023)</span>
<span class="ltx_bibblock">
Shengnan An, Bo Zhou, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Weizhu Chen,
and Jian-Guang Lou. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2023.emnlp-main.831" title="" class="ltx_ref ltx_href">Skill-based
few-shot selection for in-context learning</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2023 Conference on Empirical Methods in
Natural Language Processing</em>, pages 13472–13492, Singapore. Association for
Computational Linguistics.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cao et al. (2023)</span>
<span class="ltx_bibblock">
Ruisheng Cao, Lu Chen, Jieyu Li, Hanchong Zhang, Hongshen Xu, Wangyou Zhang,
and Kai Yu. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.1109/TPAMI.2023.3298895" title="" class="ltx_ref ltx_href">A heterogeneous
graph to abstract syntax tree framework for text-to-SQL</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">IEEE Transactions on Pattern Analysis and Machine
Intelligence</em>, 45(11):13796–13813.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chang and Fosler-Lussier (2023)</span>
<span class="ltx_bibblock">
Shuaichen Chang and Eric Fosler-Lussier. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2305.11853" title="" class="ltx_ref ltx_href">How to prompt llms for
text-to-SQL: A study in zero-shot, single-domain, and cross-domain
settings</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2305.11853.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2024)</span>
<span class="ltx_bibblock">
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2024.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://openreview.net/forum?id=KuPixIqPiq" title="" class="ltx_ref ltx_href">Teaching large
language models to self-debug</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning
Representations</em>.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Deng et al. (2021)</span>
<span class="ltx_bibblock">
Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan
Sun, and Matthew Richardson. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2021.naacl-main.105" title="" class="ltx_ref ltx_href">Structure-grounded pretraining for text-to-SQL</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language
Technologies</em>, pages 1337–1350, Online. Association for Computational
Linguistics.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dong et al. (2023)</span>
<span class="ltx_bibblock">
Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, lu Chen, Jinshu Lin,
and Dongfang Lou. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2307.07306" title="" class="ltx_ref ltx_href">C3: Zero-shot text-to-SQL
with ChatGPT</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2307.07306.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Fluri et al. (2007)</span>
<span class="ltx_bibblock">
Beat Fluri, Michael Wursch, Martin PInzger, and Harald Gall. 2007.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.1109/TSE.2007.70731" title="" class="ltx_ref ltx_href">Change
distilling:tree differencing for fine-grained source code change extraction</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">IEEE Transactions on Software Engineering</em>, 33(11):725–743.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gan et al. (2021a)</span>
<span class="ltx_bibblock">
Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward,
Jinxia Xie, and Pengsheng Huang. 2021a.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2021.acl-long.195" title="" class="ltx_ref ltx_href">Towards
robustness of text-to-SQL models against synonym substitution</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 59th Annual Meeting of the Association
for Computational Linguistics and the 11th International Joint Conference on
Natural Language Processing (Volume 1: Long Papers)</em>, pages 2505–2515,
Online. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gan et al. (2021b)</span>
<span class="ltx_bibblock">
Yujian Gan, Xinyun Chen, and Matthew Purver. 2021b.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2021.emnlp-main.702" title="" class="ltx_ref ltx_href">Exploring
underexplored limitations of cross-domain text-to-SQL generalization</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference on Empirical Methods in
Natural Language Processing</em>, pages 8926–8931, Online and Punta Cana,
Dominican Republic. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gao et al. (2023)</span>
<span class="ltx_bibblock">
Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and
Jingren Zhou. 2023.

</span>
<span class="ltx_bibblock">Text-to-SQL empowered by large language models: A benchmark
evaluation.

</span>
<span class="ltx_bibblock"><em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">CoRR</em>, abs/2308.15363.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Guo et al. (2024)</span>
<span class="ltx_bibblock">
Chunxi Guo, Zhiliang Tian, Jintao Tang, Shasha Li, Zhihua Wen, Kaixuan Wang,
and Ting Wang. 2024.

</span>
<span class="ltx_bibblock">Retrieval-augmented GPT-3.5-based text-to-SQL framework
with sample-aware prompting and dynamic revision chain.

</span>
<span class="ltx_bibblock">In <em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">Neural Information Processing</em>, pages 341–356, Singapore.
Springer Nature Singapore.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023a)</span>
<span class="ltx_bibblock">
Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. 2023a.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.1609/aaai.v37i11.26535" title="" class="ltx_ref ltx_href">RESDSQL:
Decoupling schema linking and skeleton parsing for text-to-sql</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">Proceedings of the AAAI Conference on Artificial Intelligence</em>,
37(11):13067–13075.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023b)</span>
<span class="ltx_bibblock">
Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei
Huang, Wenyu Du, Luo Si, and Yongbin Li. 2023b.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.1609/aaai.v37i11.26536" title="" class="ltx_ref ltx_href">Graphix-T5:
mixing pre-trained transformers with graph-aware layers for text-to-SQL
parsing</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">Proceedings of the Thirty-Seventh AAAI Conference on
Artificial Intelligence and Thirty-Fifth Conference on Innovative
Applications of Artificial Intelligence and Thirteenth Symposium on
Educational Advances in Artificial Intelligence</em>, AAAI’23/IAAI’23/EAAI’23.
AAAI Press.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. (2023)</span>
<span class="ltx_bibblock">
Aiwei Liu, Xuming Hu, Lijie Wen, and Philip S. Yu. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2303.13547" title="" class="ltx_ref ltx_href">A comprehensive evaluation
of ChatGPT’s zero-shot text-to-SQL capability</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2303.13547.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Min et al. (2019)</span>
<span class="ltx_bibblock">
Qingkai Min, Yuefeng Shi, and Yue Zhang. 2019.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/D19-1377" title="" class="ltx_ref ltx_href">A pilot study for
Chinese SQL semantic parsing</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint Conference on
Natural Language Processing (EMNLP-IJCNLP)</em>, pages 3652–3658, Hong Kong,
China. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Nan et al. (2023)</span>
<span class="ltx_bibblock">
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang,
Arman Cohan, and Dragomir Radev. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2023.findings-emnlp.996" title="" class="ltx_ref ltx_href">Enhancing text-to-SQL capabilities of large language models: A study on
prompt design strategies</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">Findings of the Association for Computational Linguistics:
EMNLP 2023</em>, pages 14935–14956, Singapore. Association for Computational
Linguistics.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI et al. (2024)</span>
<span class="ltx_bibblock">
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya,
Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman,
Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom,
Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake
Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg
Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles
Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany
Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis
Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben
Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah
Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien
Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien
Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix,
Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges,
Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes,
Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross,
Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen
He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey,
Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost
Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang,
Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan,
Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar,
Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim,
Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz
Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen
Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade
Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin,
Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim
Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie
Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey,
Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke
Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel
Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro
Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long
Ouyang, Cullen O’Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley
Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex
Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila
Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael,
Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power,
Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya
Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob
Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani
Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman,
Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker,
Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin,
Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher,
Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas
Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth
Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe,
Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright,
Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann,
Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave
Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin
Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan,
Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao,
Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. 2024.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2303.08774" title="" class="ltx_ref ltx_href">GPT-4 technical report</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2303.08774.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Pourreza and Rafiei (2023)</span>
<span class="ltx_bibblock">
Mohammadreza Pourreza and Davood Rafiei. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2304.11015" title="" class="ltx_ref ltx_href">DIN-SQL: Decomposed
in-context learning of text-to-SQL with self-correction</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2304.11015.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Raffel et al. (2020)</span>
<span class="ltx_bibblock">
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael
Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.

</span>
<span class="ltx_bibblock">Exploring the limits of transfer learning with a unified text-to-text
transformer.

</span>
<span class="ltx_bibblock"><em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">J. Mach. Learn. Res.</em>, 21(1).

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Rajkumar et al. (2022)</span>
<span class="ltx_bibblock">
Nitarshan Rajkumar, Raymond Li, and Dzmitry Bahdanau. 2022.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2204.00498" title="" class="ltx_ref ltx_href">Evaluating the
text-to-SQL capabilities of large language models</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2204.00498.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Reimers and Gurevych (2019)</span>
<span class="ltx_bibblock">
Nils Reimers and Iryna Gurevych. 2019.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/D19-1410" title="" class="ltx_ref ltx_href">Sentence-BERT:
Sentence embeddings using Siamese BERT-networks</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint Conference on
Natural Language Processing (EMNLP-IJCNLP)</em>, pages 3982–3992, Hong Kong,
China. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ren et al. (2020)</span>
<span class="ltx_bibblock">
Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel
Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2009.10297" title="" class="ltx_ref ltx_href">CodeBLEU: a method for
automatic evaluation of code synthesis</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2009.10297.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Robertson et al. (1994)</span>
<span class="ltx_bibblock">
Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu,
and Mike Gatford. 1994.

</span>
<span class="ltx_bibblock"><a target="_blank" href="http://trec.nist.gov/pubs/trec3/papers/city.ps.gz" title="" class="ltx_ref ltx_href">Okapi at
TREC-3</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">Proceedings of The Third Text REtrieval Conference, TREC
1994, Gaithersburg, Maryland, USA, November 2-4, 1994</em>, volume 500-225 of
<em id="bib.bib23.2.2" class="ltx_emph ltx_font_italic">NIST Special Publication</em>, pages 109–126. National Institute of
Standards and Technology (NIST).

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Song et al. (2020)</span>
<span class="ltx_bibblock">
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://proceedings.neurips.cc/paper_files/paper/2020/file/c3a690be93aa602ee2dc0ccab5b7b67e-Paper.pdf" title="" class="ltx_ref ltx_href">MPNet: Masked and permuted pre-training for language understanding</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>,
volume 33, pages 16857–16867. Curran Associates, Inc.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sun et al. (2024)</span>
<span class="ltx_bibblock">
Ruoxi Sun, Sercan Ö. Arik, Alex Muzio, Lesly Miculicich, Satya Gundabathula,
Pengcheng Yin, Hanjun Dai, Hootan Nakhost, Rajarishi Sinha, Zifeng Wang, and
Tomas Pfister. 2024.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2306.00739" title="" class="ltx_ref ltx_href">SQL-PaLM: Improved
large language model adaptation for text-to-sql (extended)</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2306.00739.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Tran et al. (2019)</span>
<span class="ltx_bibblock">
Ngoc Tran, Hieu Tran, Son Nguyen, Hoan Nguyen, and Tien Nguyen. 2019.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.1109/ICPC.2019.00034" title="" class="ltx_ref ltx_href">Does bleu score work
for code migration?</a>

</span>
<span class="ltx_bibblock">In <em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">2019 IEEE/ACM 27th International Conference on Program
Comprehension (ICPC)</em>, pages 165–176.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Vougiouklis et al. (2023)</span>
<span class="ltx_bibblock">
Pavlos Vougiouklis, Nikos Papasarantopoulos, Danna Zheng, David Tuckey, Chenxin
Diao, Zhili Shen, and Jeff Pan. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2023.ijcnlp-main.38" title="" class="ltx_ref ltx_href">FastRAT:
Fast and efficient cross-lingual text-to-SQL semantic parsing</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 13th International Joint Conference on
Natural Language Processing and the 3rd Conference of the Asia-Pacific
Chapter of the Association for Computational Linguistics (Volume 1: Long
Papers)</em>, pages 564–576, Nusa Dua, Bali. Association for Computational
Linguistics.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2021)</span>
<span class="ltx_bibblock">
Bailin Wang, Mirella Lapata, and Ivan Titov. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2021.naacl-main.33" title="" class="ltx_ref ltx_href">Meta-learning
for domain generalization in semantic parsing</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2021 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language
Technologies</em>, pages 366–379, Online. Association for Computational
Linguistics.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2020)</span>
<span class="ltx_bibblock">
Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew
Richardson. 2020.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2020.acl-main.677" title="" class="ltx_ref ltx_href">RAT-SQL:
Relation-aware schema encoding and linking for text-to-SQL parsers</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib29.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 58th Annual Meeting of the Association
for Computational Linguistics</em>, pages 7567–7578, Online. Association for
Computational Linguistics.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ye et al. (2023)</span>
<span class="ltx_bibblock">
Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui,
Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang,
and Xuanjing Huang. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2303.10420" title="" class="ltx_ref ltx_href">A comprehensive capability
analysis of GPT-3 and GPT-3.5 series models</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib30.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2303.10420.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yu et al. (2021)</span>
<span class="ltx_bibblock">
Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang,
Dragomir Radev, Richard Socher, and Caiming Xiong. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2009.13845" title="" class="ltx_ref ltx_href">GraPPa: Grammar-augmented
pre-training for table semantic parsing</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yu et al. (2018)</span>
<span class="ltx_bibblock">
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James
Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev.
2018.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/D18-1425" title="" class="ltx_ref ltx_href">Spider: A large-scale
human-labeled dataset for complex and cross-domain semantic parsing and
text-to-SQL task</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib32.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2018 Conference on Empirical Methods in
Natural Language Processing</em>, pages 3911–3921, Brussels, Belgium.
Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zelle and Mooney (1996)</span>
<span class="ltx_bibblock">
John M. Zelle and Raymond J. Mooney. 1996.

</span>
<span class="ltx_bibblock">Learning to parse database queries using inductive logic programming.

</span>
<span class="ltx_bibblock">In <em id="bib.bib33.1.1" class="ltx_emph ltx_font_italic">Proceedings of the Thirteenth National Conference on
Artificial Intelligence - Volume 2</em>, AAAI’96, page 1050–1055, Portland,
Oregon. AAAI Press.

</span>
</li>
</ul>
</section>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>SQL Similarity using Normalised Abstract Syntax Trees</h2>

<div id="A1.p1" class="ltx_para">
<p id="A1.p1.1" class="ltx_p">Table <a href="#A1.T7" title="Table 7 ‣ Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> shows the corresponding SQL queries after each step of our AST normalisation as explained in Section <a href="#S3" title="3 Example Selection using Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>. An example of the similarity between normalised ASTs is provided in Figure <a href="#A1.F2" title="Figure 2 ‣ Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, where tables, columns and values are masked out for cross-domain settings.</p>
</div>
<figure id="A1.T7" class="ltx_table">
<table id="A1.T7.1" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A1.T7.1.1.1" class="ltx_tr">
<th id="A1.T7.1.1.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_th ltx_th_column ltx_border_tt" style="width:41.3pt;">
<span id="A1.T7.1.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.1.1.1.1.1" class="ltx_p"><span id="A1.T7.1.1.1.1.1.1.1" class="ltx_text ltx_font_bold">SQL</span></span>
</span>
</th>
<th id="A1.T7.1.1.1.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_th ltx_th_column ltx_border_tt" style="width:155.1pt;">
<span id="A1.T7.1.1.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.1.1.2.1.1" class="ltx_p"><span id="A1.T7.1.1.1.2.1.1.1" class="ltx_text ltx_font_typewriter" style="color:#FF0000;">SELECT<span id="A1.T7.1.1.1.2.1.1.1.1" class="ltx_text" style="color:#000000;"> T1.Category, </span>COUNT<span id="A1.T7.1.1.1.2.1.1.1.2" class="ltx_text" style="color:#000000;">(*) </span>AS<span id="A1.T7.1.1.1.2.1.1.1.3" class="ltx_text" style="color:#000000;"> Num </span>FROM<span id="A1.T7.1.1.1.2.1.1.1.4" class="ltx_text" style="color:#000000;"> Products </span>AS<span id="A1.T7.1.1.1.2.1.1.1.5" class="ltx_text" style="color:#000000;"> T1 </span>JOIN<span id="A1.T7.1.1.1.2.1.1.1.6" class="ltx_text" style="color:#000000;"> Orders </span>AS<span id="A1.T7.1.1.1.2.1.1.1.7" class="ltx_text" style="color:#000000;"> T2 </span>ON<span id="A1.T7.1.1.1.2.1.1.1.8" class="ltx_text" style="color:#000000;"> T1.id = T2.pid </span>GROUP BY<span id="A1.T7.1.1.1.2.1.1.1.9" class="ltx_text" style="color:#000000;"> T1.Category </span>ORDER BY<span id="A1.T7.1.1.1.2.1.1.1.10" class="ltx_text" style="color:#000000;"> Num </span>ASC</span></span>
</span>
</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A1.T7.1.2.1" class="ltx_tr">
<td id="A1.T7.1.2.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:41.3pt;">
<span id="A1.T7.1.2.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.2.1.1.1.1" class="ltx_p"><span id="A1.T7.1.2.1.1.1.1.1" class="ltx_text ltx_font_bold">1. Unify Identifiers</span></span>
</span>
</td>
<td id="A1.T7.1.2.1.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:155.1pt;">
<span id="A1.T7.1.2.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.2.1.2.1.1" class="ltx_p"><span id="A1.T7.1.2.1.2.1.1.1" class="ltx_text ltx_font_typewriter" style="color:#FF0000;">SELECT<span id="A1.T7.1.2.1.2.1.1.1.1" class="ltx_text" style="color:#000000;"> t1.category, </span>COUNT<span id="A1.T7.1.2.1.2.1.1.1.2" class="ltx_text" style="color:#000000;">(*) </span>AS<span id="A1.T7.1.2.1.2.1.1.1.3" class="ltx_text" style="color:#000000;"> num </span>FROM<span id="A1.T7.1.2.1.2.1.1.1.4" class="ltx_text" style="color:#000000;"> products </span>AS<span id="A1.T7.1.2.1.2.1.1.1.5" class="ltx_text" style="color:#000000;"> t1 </span>JOIN<span id="A1.T7.1.2.1.2.1.1.1.6" class="ltx_text" style="color:#000000;"> orders </span>AS<span id="A1.T7.1.2.1.2.1.1.1.7" class="ltx_text" style="color:#000000;"> t2 </span>ON<span id="A1.T7.1.2.1.2.1.1.1.8" class="ltx_text" style="color:#000000;"> t1.id = t2.pid </span>GROUP BY<span id="A1.T7.1.2.1.2.1.1.1.9" class="ltx_text" style="color:#000000;"> t1.category </span>ORDER BY<span id="A1.T7.1.2.1.2.1.1.1.10" class="ltx_text" style="color:#000000;"> num </span>ASC</span></span>
</span>
</td>
</tr>
<tr id="A1.T7.1.3.2" class="ltx_tr">
<td id="A1.T7.1.3.2.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:41.3pt;">
<span id="A1.T7.1.3.2.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.3.2.1.1.1" class="ltx_p"><span id="A1.T7.1.3.2.1.1.1.1" class="ltx_text ltx_font_bold">2. Resolve Aliases</span></span>
</span>
</td>
<td id="A1.T7.1.3.2.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:155.1pt;">
<span id="A1.T7.1.3.2.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.3.2.2.1.1" class="ltx_p"><span id="A1.T7.1.3.2.2.1.1.1" class="ltx_text ltx_font_typewriter" style="color:#FF0000;">SELECT<span id="A1.T7.1.3.2.2.1.1.1.1" class="ltx_text" style="color:#000000;"> products.category, </span>COUNT<span id="A1.T7.1.3.2.2.1.1.1.2" class="ltx_text" style="color:#000000;">(*) </span>FROM<span id="A1.T7.1.3.2.2.1.1.1.3" class="ltx_text" style="color:#000000;"> products </span>JOIN<span id="A1.T7.1.3.2.2.1.1.1.4" class="ltx_text" style="color:#000000;"> orders </span>ON<span id="A1.T7.1.3.2.2.1.1.1.5" class="ltx_text" style="color:#000000;"> products.id = orders.pid </span>GROUP BY<span id="A1.T7.1.3.2.2.1.1.1.6" class="ltx_text" style="color:#000000;"> products.category </span>ORDER BY<span id="A1.T7.1.3.2.2.1.1.1.7" class="ltx_text" style="color:#000000;"> </span>COUNT<span id="A1.T7.1.3.2.2.1.1.1.8" class="ltx_text" style="color:#000000;">(*) </span>ASC</span></span>
</span>
</td>
</tr>
<tr id="A1.T7.1.4.3" class="ltx_tr">
<td id="A1.T7.1.4.3.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:41.3pt;">
<span id="A1.T7.1.4.3.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.4.3.1.1.1" class="ltx_p"><span id="A1.T7.1.4.3.1.1.1.1" class="ltx_text ltx_font_bold">3. Reorder JOIN</span></span>
</span>
</td>
<td id="A1.T7.1.4.3.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_t" style="width:155.1pt;">
<span id="A1.T7.1.4.3.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.4.3.2.1.1" class="ltx_p"><span id="A1.T7.1.4.3.2.1.1.1" class="ltx_text ltx_font_typewriter" style="color:#FF0000;">SELECT<span id="A1.T7.1.4.3.2.1.1.1.1" class="ltx_text" style="color:#000000;"> products.category, </span>COUNT<span id="A1.T7.1.4.3.2.1.1.1.2" class="ltx_text" style="color:#000000;">(*) </span>FROM<span id="A1.T7.1.4.3.2.1.1.1.3" class="ltx_text" style="color:#000000;"> orders </span>JOIN<span id="A1.T7.1.4.3.2.1.1.1.4" class="ltx_text" style="color:#000000;"> products </span>ON<span id="A1.T7.1.4.3.2.1.1.1.5" class="ltx_text" style="color:#000000;"> orders.id = products.pid </span>GROUP BY<span id="A1.T7.1.4.3.2.1.1.1.6" class="ltx_text" style="color:#000000;"> products.category </span>ORDER BY COUNT<span id="A1.T7.1.4.3.2.1.1.1.7" class="ltx_text" style="color:#000000;">(*) </span>ASC</span></span>
</span>
</td>
</tr>
<tr id="A1.T7.1.5.4" class="ltx_tr">
<td id="A1.T7.1.5.4.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb ltx_border_t" style="width:41.3pt;">
<span id="A1.T7.1.5.4.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.5.4.1.1.1" class="ltx_p"><span id="A1.T7.1.5.4.1.1.1.1" class="ltx_text ltx_font_bold">4. Mask Identifiers &amp; Values</span> (<span id="A1.T7.1.5.4.1.1.1.2" class="ltx_text" style="color:#0000FF;">cross-domain only</span>)</span>
</span>
</td>
<td id="A1.T7.1.5.4.2" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb ltx_border_t" style="width:155.1pt;">
<span id="A1.T7.1.5.4.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T7.1.5.4.2.1.1" class="ltx_p"><span id="A1.T7.1.5.4.2.1.1.1" class="ltx_text ltx_font_typewriter" style="color:#FF0000;">SELECT<span id="A1.T7.1.5.4.2.1.1.1.1" class="ltx_text" style="color:#000000;"> _, </span>COUNT<span id="A1.T7.1.5.4.2.1.1.1.2" class="ltx_text" style="color:#000000;">(*) </span>FROM<span id="A1.T7.1.5.4.2.1.1.1.3" class="ltx_text" style="color:#000000;"> _ </span>JOIN<span id="A1.T7.1.5.4.2.1.1.1.4" class="ltx_text" style="color:#000000;"> _ </span>ON<span id="A1.T7.1.5.4.2.1.1.1.5" class="ltx_text" style="color:#000000;"> _ = _ </span>GROUP BY<span id="A1.T7.1.5.4.2.1.1.1.6" class="ltx_text" style="color:#000000;"> _ </span>ORDER BY COUNT<span id="A1.T7.1.5.4.2.1.1.1.7" class="ltx_text" style="color:#000000;">(*) </span>ASC</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 7: </span>An example of the effects to corresponding SQL after each step of our AST normalisation.</figcaption>
</figure>
<figure id="A1.F2" class="ltx_figure"><img src="/html/2407.03227/assets/x2.png" id="A1.F2.g1" class="ltx_graphics ltx_centering ltx_img_portrait" width="438" height="787" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 2: </span>Example of how the similarity between two different SQL queries is computed using normalised ASTs.</figcaption>
</figure>
<figure id="A1.T8" class="ltx_table">
<table id="A1.T8.1" class="ltx_tabular ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A1.T8.1.2.1" class="ltx_tr">
<td id="A1.T8.1.2.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_r ltx_border_tt" style="width:28.5pt;"></td>
<td id="A1.T8.1.2.1.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="A1.T8.1.2.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.2.1.2.1.1" class="ltx_p" style="width:165.0pt;"># Given SQLite database schema student_transcripts:</span>
</span>
</td>
</tr>
<tr id="A1.T8.1.1" class="ltx_tr">
<td id="A1.T8.1.1.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_r" style="width:28.5pt;">
<span id="A1.T8.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.1.1.1.1" class="ltx_block ltx_parbox ltx_align_middle" style="width:0.0pt;">
<span id="A1.T8.1.1.1.1.1.2" class="ltx_p"><span id="A1.T8.1.1.1.1.1.2.1" class="ltx_text" style="background-color:#FDC883;">Selected</span></span>
<span id="A1.T8.1.1.1.1.1.3" class="ltx_p"><span id="A1.T8.1.1.1.1.1.3.1" class="ltx_text" style="background-color:#FDC883;">Schema</span></span>
<span id="A1.T8.1.1.1.1.1.1" class="ltx_p">w<math id="A1.T8.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="/" display="inline"><semantics id="A1.T8.1.1.1.1.1.1.m1.1a"><mo id="A1.T8.1.1.1.1.1.1.m1.1.1" xref="A1.T8.1.1.1.1.1.1.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="A1.T8.1.1.1.1.1.1.m1.1b"><divide id="A1.T8.1.1.1.1.1.1.m1.1.1.cmml" xref="A1.T8.1.1.1.1.1.1.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="A1.T8.1.1.1.1.1.1.m1.1c">/</annotation></semantics></math> <span id="A1.T8.1.1.1.1.1.1.1" class="ltx_text" style="background-color:#FFBFBF;">Selected</span></span>
<span id="A1.T8.1.1.1.1.1.4" class="ltx_p"><span id="A1.T8.1.1.1.1.1.4.1" class="ltx_text" style="background-color:#FFBFBF;">Values</span></span>
</span>
</span>
</td>
<td id="A1.T8.1.1.2" class="ltx_td ltx_align_justify ltx_align_top" style="background-color:#FDC883;">
<span id="A1.T8.1.1.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.1.2.1.1" class="ltx_p" style="width:165.0pt;">
<span id="A1.T8.1.1.2.1.1.1" class="ltx_inline-block ltx_parbox ltx_align_middle" style="width:411.9pt;background-color:#FDC883;">
<span id="A1.T8.1.1.2.1.1.1.1" class="ltx_p"><span id="A1.T8.1.1.2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">CREATE TABLE Departments(
<br class="ltx_break">      department_id number,
<br class="ltx_break">      department_name text COMMENT ’department name <span id="A1.T8.1.1.2.1.1.1.1.1.1" class="ltx_text" style="background-color:#FFBFBF;">(e.g. engineer, statistics, medical)</span>’, 
<br class="ltx_break">      ...);
<br class="ltx_break">CREATE TABLE Degree_Programs(
<br class="ltx_break">      degree_program_id number,
<br class="ltx_break">      degree_summary_name text COMMENT ’degree summary name <span id="A1.T8.1.1.2.1.1.1.1.1.2" class="ltx_text" style="background-color:#FFBFBF;">(e.g. PHD, Master, Bachelor)</span>’,
      ...
<br class="ltx_break">      PRIMARY KEY (degree_program_id), 
<br class="ltx_break">      FOREIGN KEY (department_id) REFERENCES Departments(department_id));</span></span>
</span></span>
</span>
</td>
</tr>
<tr id="A1.T8.1.3.2" class="ltx_tr">
<td id="A1.T8.1.3.2.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_r" style="width:28.5pt;"></td>
<td id="A1.T8.1.3.2.2" class="ltx_td ltx_align_justify ltx_align_top">
<span id="A1.T8.1.3.2.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.3.2.2.1.1" class="ltx_p" style="width:165.0pt;"># Your task is to translate Question into SQL.</span>
</span>
</td>
</tr>
<tr id="A1.T8.1.4.3" class="ltx_tr">
<td id="A1.T8.1.4.3.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_r" style="width:28.5pt;"></td>
<td id="A1.T8.1.4.3.2" class="ltx_td ltx_align_justify ltx_align_top">
<span id="A1.T8.1.4.3.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.4.3.2.1.1" class="ltx_p" style="width:165.0pt;"># Some examples are provided based on similar problems:</span>
</span>
</td>
</tr>
<tr id="A1.T8.1.5.4" class="ltx_tr">
<td id="A1.T8.1.5.4.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_r" style="width:28.5pt;">
<span id="A1.T8.1.5.4.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.5.4.1.1.1" class="ltx_block ltx_parbox ltx_align_middle" style="width:0.0pt;">
<span id="A1.T8.1.5.4.1.1.1.1" class="ltx_p"><span id="A1.T8.1.5.4.1.1.1.1.1" class="ltx_text" style="background-color:#98FB98;">Selected</span></span>
<span id="A1.T8.1.5.4.1.1.1.2" class="ltx_p"><span id="A1.T8.1.5.4.1.1.1.2.1" class="ltx_text" style="background-color:#98FB98;">Examples</span></span>
</span>
</span>
</td>
<td id="A1.T8.1.5.4.2" class="ltx_td ltx_align_justify ltx_align_top" style="background-color:#98FB98;">
<span id="A1.T8.1.5.4.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.5.4.2.1.1" class="ltx_p" style="width:165.0pt;">
<span id="A1.T8.1.5.4.2.1.1.1" class="ltx_inline-block ltx_parbox ltx_align_middle" style="width:411.9pt;background-color:#98FB98;">
<span id="A1.T8.1.5.4.2.1.1.1.1" class="ltx_p">Question: How many courses does the department of Computer Information Systems offer?</span>
<span id="A1.T8.1.5.4.2.1.1.1.2" class="ltx_p">SQL: <span id="A1.T8.1.5.4.2.1.1.1.2.1" class="ltx_text ltx_font_typewriter">SELECT count(*) FROM department AS T1 JOIN course AS T2 ON T1.dept_code = T2.dept_code WHERE dept_name = ‘‘Computer Info.Systems’’
<br class="ltx_break"></span>Question: …</span>
<span id="A1.T8.1.5.4.2.1.1.1.3" class="ltx_p">SQL: …</span>
</span></span>
</span>
</td>
</tr>
<tr id="A1.T8.1.6.5" class="ltx_tr">
<td id="A1.T8.1.6.5.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_r" style="width:28.5pt;"></td>
<td id="A1.T8.1.6.5.2" class="ltx_td ltx_align_justify ltx_align_top">
<span id="A1.T8.1.6.5.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.6.5.2.1.1" class="ltx_p" style="width:165.0pt;"># Complete the following SQL for schema student_transcripts:</span>
</span>
</td>
</tr>
<tr id="A1.T8.1.7.6" class="ltx_tr">
<td id="A1.T8.1.7.6.1" class="ltx_td ltx_align_justify ltx_align_middle ltx_border_bb ltx_border_r" style="width:28.5pt;">
<span id="A1.T8.1.7.6.1.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.7.6.1.1.1" class="ltx_block ltx_parbox ltx_align_middle" style="width:0.0pt;">
<span id="A1.T8.1.7.6.1.1.1.1" class="ltx_p"><span id="A1.T8.1.7.6.1.1.1.1.1" class="ltx_text" style="background-color:#D1EDF2;">Test</span></span>
<span id="A1.T8.1.7.6.1.1.1.2" class="ltx_p"><span id="A1.T8.1.7.6.1.1.1.2.1" class="ltx_text" style="background-color:#D1EDF2;">Question</span></span>
</span>
</span>
</td>
<td id="A1.T8.1.7.6.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb" style="background-color:#D1EDF2;">
<span id="A1.T8.1.7.6.2.1" class="ltx_inline-block ltx_align_top">
<span id="A1.T8.1.7.6.2.1.1" class="ltx_p" style="width:165.0pt;">
<span id="A1.T8.1.7.6.2.1.1.1" class="ltx_inline-block ltx_parbox ltx_align_middle" style="width:411.9pt;background-color:#D1EDF2;">
<span id="A1.T8.1.7.6.2.1.1.1.1" class="ltx_p">Question: How many degrees does the engineering department have?</span>
<span id="A1.T8.1.7.6.2.1.1.1.2" class="ltx_p">SQL:</span>
</span></span>
</span>
</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 8: </span>An example of the resulting prompt, after example and schema and DB content selection.</figcaption>
</figure>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Prompt Formulation</h2>

<div id="A2.p1" class="ltx_para">
<p id="A2.p1.1" class="ltx_p">Table <a href="#A1.T8" title="Table 8 ‣ Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a> shows an example of our prompt, after example and DB context selection (i.e. schema and value selection). This prompt is provided as input to LLMs. Following the latest OpenAI example<span id="footnote7" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">7</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">7</sup><span class="ltx_tag ltx_tag_note">7</span><a target="_blank" href="https://platform.openai.com/examples/default-sql-translate" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://platform.openai.com/examples/default-sql-translate</a></span></span></span> for Text-to-SQL parsing, we represent a schema with <span id="A2.p1.1.1" class="ltx_text ltx_font_typewriter">CREATE TABLE</span> statements in SQL. Semantic names or descriptions of tables and columns are included as <span id="A2.p1.1.2" class="ltx_text ltx_font_typewriter">COMMENT</span> along with the corresponding columns or tables. Note that we
filter out comments that can be obtained by simply lowercasing original names and/or removing underscores. To maintain a compact representation of database information, we append selected values of columns into their <span id="A2.p1.1.3" class="ltx_text ltx_font_typewriter">COMMENT</span> rather than introducing additional lines as in the work by <cite class="ltx_cite ltx_citemacro_citet">Chen et al. (<a href="#bib.bib4" title="" class="ltx_ref">2024</a>)</cite>. Example (question, SQL) pairs are provided in a similar manner to DAIL-SQL <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>, followed by an instruction to prompt LLMs to generate SQL for the test question.</p>
</div>
</section>
<section id="A3" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix C </span>Implementation Details</h2>

<div id="A3.p1" class="ltx_para">
<p id="A3.p1.1" class="ltx_p">We use this section to provide further details about the implementation of our approach.</p>
</div>
<section id="A3.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">C.1 </span>Example Selection</h3>

<div id="A3.SS1.p1" class="ltx_para">
<p id="A3.SS1.p1.1" class="ltx_p">Following <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>, we employ the pre-trained <span id="A3.SS1.p1.1.1" class="ltx_text ltx_font_typewriter">all-mpnet-base-v2</span> model <cite class="ltx_cite ltx_citemacro_cite">Song et al. (<a href="#bib.bib24" title="" class="ltx_ref">2020</a>)</cite> from Sentence Transformer <cite class="ltx_cite ltx_citemacro_cite">Reimers and Gurevych (<a href="#bib.bib21" title="" class="ltx_ref">2019</a>)</cite> to compute dense question embeddings for English datasets including S<span id="A3.SS1.p1.1.2" class="ltx_text ltx_font_smallcaps">pider</span>, S<span id="A3.SS1.p1.1.3" class="ltx_text ltx_font_smallcaps">pider</span>-DK, S<span id="A3.SS1.p1.1.4" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="A3.SS1.p1.1.5" class="ltx_text ltx_font_smallcaps">eal</span>, and S<span id="A3.SS1.p1.1.6" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="A3.SS1.p1.1.7" class="ltx_text ltx_font_smallcaps">yn</span>. For CS<span id="A3.SS1.p1.1.8" class="ltx_text ltx_font_smallcaps">pider</span>, <span id="A3.SS1.p1.1.9" class="ltx_text ltx_font_typewriter">paraphrase-multilingual-MiniLM-L12-v2</span> is used instead. SQL queries are parsed into AST by using SQLGlot<span id="footnote8" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">8</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">8</sup><span class="ltx_tag ltx_tag_note">8</span><a target="_blank" href="https://github.com/tobymao/sqlglot" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/tobymao/sqlglot</a></span></span></span> and are then normalised as explained in Section <a href="#S3" title="3 Example Selection using Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>. SQLGlot provides an implementation of the Change Distilling algorithm for AST differencing. We refer readers to SQLGlot’s documentation<span id="footnote9" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">9</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">9</sup><span class="ltx_tag ltx_tag_note">9</span><a target="_blank" href="https://github.com/tobymao/sqlglot/blob/main/posts/sql_diff.md" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/tobymao/sqlglot/blob/main/posts/sql_diff.md</a></span></span></span> for more details. For selecting example question-SQL pairs, we first retrieve top 500 examples by question similarity and rerank them in terms of the similarity of normalised SQL ASTs. For relevant experiments in Table <a href="#S5.T5" title="Table 5 ‣ 5.3.2 Ablation Study ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, we reproduced the implementation of DAIL selection from the original paper <cite class="ltx_cite ltx_citemacro_cite">Gao et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>. The number of few-shot examples is set to 5 across all experiments.</p>
</div>
</section>
<section id="A3.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">C.2 </span>Schema &amp; Value Selection</h3>

<div id="A3.SS2.p1" class="ltx_para">
<p id="A3.SS2.p1.2" class="ltx_p">Each database schema is treated as an independent collection of columns that are analogous to documents to be retrieved by using BM25.
As mentioned in Section <a href="#S4.SS1" title="4.1 Schema Selection ‣ 4 Database Context Selection ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1</span></a>, we represent a column by concatenating semantic names of both the column and its table, and the column values in the database.
Semantic names and values are tokenized using spaCy<span id="footnote10" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">10</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">10</sup><span class="ltx_tag ltx_tag_note">10</span><a target="_blank" href="https://github.com/explosion/spaCy" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/explosion/spaCy</a></span></span></span> and preprocessed by lowercasing and stemming<span id="footnote11" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">11</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">11</sup><span class="ltx_tag ltx_tag_note">11</span><a target="_blank" href="https://www.nltk.org/api/nltk.stem.porter.html" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://www.nltk.org/api/nltk.stem.porter.html</a></span></span></span>. At inference time, the same processing is applied to questions. We adopt the implementation of Okapi BM25 <cite class="ltx_cite ltx_citemacro_cite">Robertson et al. (<a href="#bib.bib23" title="" class="ltx_ref">1994</a>)</cite> from Rank-BM25<span id="footnote12" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">12</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">12</sup><span class="ltx_tag ltx_tag_note">12</span><a target="_blank" href="https://github.com/dorianbrown/rank_bm25" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/dorianbrown/rank_bm25</a></span></span></span>.
The number of columns to retrieve is dynamically set to <math id="A3.SS2.p1.1.m1.1" class="ltx_Math" alttext="\lfloor 1.5\times\gamma\rfloor" display="inline"><semantics id="A3.SS2.p1.1.m1.1a"><mrow id="A3.SS2.p1.1.m1.1.1.1" xref="A3.SS2.p1.1.m1.1.1.2.cmml"><mo stretchy="false" id="A3.SS2.p1.1.m1.1.1.1.2" xref="A3.SS2.p1.1.m1.1.1.2.1.cmml">⌊</mo><mrow id="A3.SS2.p1.1.m1.1.1.1.1" xref="A3.SS2.p1.1.m1.1.1.1.1.cmml"><mn id="A3.SS2.p1.1.m1.1.1.1.1.2" xref="A3.SS2.p1.1.m1.1.1.1.1.2.cmml">1.5</mn><mo lspace="0.222em" rspace="0.222em" id="A3.SS2.p1.1.m1.1.1.1.1.1" xref="A3.SS2.p1.1.m1.1.1.1.1.1.cmml">×</mo><mi id="A3.SS2.p1.1.m1.1.1.1.1.3" xref="A3.SS2.p1.1.m1.1.1.1.1.3.cmml">γ</mi></mrow><mo stretchy="false" id="A3.SS2.p1.1.m1.1.1.1.3" xref="A3.SS2.p1.1.m1.1.1.2.1.cmml">⌋</mo></mrow><annotation-xml encoding="MathML-Content" id="A3.SS2.p1.1.m1.1b"><apply id="A3.SS2.p1.1.m1.1.1.2.cmml" xref="A3.SS2.p1.1.m1.1.1.1"><floor id="A3.SS2.p1.1.m1.1.1.2.1.cmml" xref="A3.SS2.p1.1.m1.1.1.1.2"></floor><apply id="A3.SS2.p1.1.m1.1.1.1.1.cmml" xref="A3.SS2.p1.1.m1.1.1.1.1"><times id="A3.SS2.p1.1.m1.1.1.1.1.1.cmml" xref="A3.SS2.p1.1.m1.1.1.1.1.1"></times><cn type="float" id="A3.SS2.p1.1.m1.1.1.1.1.2.cmml" xref="A3.SS2.p1.1.m1.1.1.1.1.2">1.5</cn><ci id="A3.SS2.p1.1.m1.1.1.1.1.3.cmml" xref="A3.SS2.p1.1.m1.1.1.1.1.3">𝛾</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS2.p1.1.m1.1c">\lfloor 1.5\times\gamma\rfloor</annotation></semantics></math> where <math id="A3.SS2.p1.2.m2.1" class="ltx_Math" alttext="\gamma" display="inline"><semantics id="A3.SS2.p1.2.m2.1a"><mi id="A3.SS2.p1.2.m2.1.1" xref="A3.SS2.p1.2.m2.1.1.cmml">γ</mi><annotation-xml encoding="MathML-Content" id="A3.SS2.p1.2.m2.1b"><ci id="A3.SS2.p1.2.m2.1.1.cmml" xref="A3.SS2.p1.2.m2.1.1">𝛾</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.SS2.p1.2.m2.1c">\gamma</annotation></semantics></math> is the number of unique columns in an approximated query.
We limit the resulting number to a range between 6 and 20.
By retrieving at column level, a table is selected if any of its columns are selected. We merge retrieved schema elements with schema elements from the approximated query to construct a sub-schema.
To further increase the recall, we add additional primary keys and foreign keys that are not selected but valid based on selected tables, except for experiments where only approximated queries are used (see Table <a href="#S5.T4" title="Table 4 ‣ 5.3.1 Schema Selection Evaluation ‣ 5.3 Text-to-SQL Evaluation ‣ 5 Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>). In such cases, however, if the SQL query involves only tables (e.g. <span id="A3.SS2.p1.2.1" class="ltx_text ltx_font_typewriter">SELECT * FROM books</span>), primary keys of selected tables are still included to ensure that corresponding <span id="A3.SS2.p1.2.2" class="ltx_text ltx_font_typewriter">CREATE TABLE</span> statements (see Table <a href="#A1.T8" title="Table 8 ‣ Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>) are meaningful and consistent.</p>
</div>
<div id="A3.SS2.p2" class="ltx_para">
<p id="A3.SS2.p2.1" class="ltx_p">For selecting values, similarly, we match the input question and the set of values for each (selected) column that has a non-numeric type. The top 3 results are added to the prompt as exemplified in Table <a href="#A1.T8" title="Table 8 ‣ Appendix A SQL Similarity using Normalised Abstract Syntax Trees ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>. The same setting of schema and value selection is used for all datasets we experimented with except CS<span id="A3.SS2.p2.1.1" class="ltx_text ltx_font_smallcaps">pider</span>. Due to the cross-lingual nature of CS<span id="A3.SS2.p2.1.2" class="ltx_text ltx_font_smallcaps">pider</span>, schema selection and value selection are simply disabled.</p>
</div>
<section id="A3.SS2.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Training <math id="A3.SS2.SSS0.Px1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A3.SS2.SSS0.Px1.1.m1.1b"><msub id="A3.SS2.SSS0.Px1.1.m1.1.1" xref="A3.SS2.SSS0.Px1.1.m1.1.1.cmml"><mtext id="A3.SS2.SSS0.Px1.1.m1.1.1.2" xref="A3.SS2.SSS0.Px1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A3.SS2.SSS0.Px1.1.m1.1.1.3" xref="A3.SS2.SSS0.Px1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.SS2.SSS0.Px1.1.m1.1c"><apply id="A3.SS2.SSS0.Px1.1.m1.1.1.cmml" xref="A3.SS2.SSS0.Px1.1.m1.1.1"><csymbol cd="ambiguous" id="A3.SS2.SSS0.Px1.1.m1.1.1.1.cmml" xref="A3.SS2.SSS0.Px1.1.m1.1.1">subscript</csymbol><ci id="A3.SS2.SSS0.Px1.1.m1.1.1.2a.cmml" xref="A3.SS2.SSS0.Px1.1.m1.1.1.2"><mtext id="A3.SS2.SSS0.Px1.1.m1.1.1.2.cmml" xref="A3.SS2.SSS0.Px1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A3.SS2.SSS0.Px1.1.m1.1.1.3a.cmml" xref="A3.SS2.SSS0.Px1.1.m1.1.1.3"><mtext mathsize="70%" id="A3.SS2.SSS0.Px1.1.m1.1.1.3.cmml" xref="A3.SS2.SSS0.Px1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS2.SSS0.Px1.1.m1.1d">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>
</h5>

<div id="A3.SS2.SSS0.Px1.p1" class="ltx_para">
<p id="A3.SS2.SSS0.Px1.p1.2" class="ltx_p">We follow the original hyper-parameters provided by <cite class="ltx_cite ltx_citemacro_cite">Vougiouklis et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite> for training <math id="A3.SS2.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A3.SS2.SSS0.Px1.p1.1.m1.1a"><msub id="A3.SS2.SSS0.Px1.p1.1.m1.1.1" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.cmml"><mtext id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.2" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.3" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.SS2.SSS0.Px1.p1.1.m1.1b"><apply id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.cmml" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.1.cmml" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1">subscript</csymbol><ci id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.2a.cmml" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.2"><mtext id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.2.cmml" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.3a.cmml" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.3"><mtext mathsize="70%" id="A3.SS2.SSS0.Px1.p1.1.m1.1.1.3.cmml" xref="A3.SS2.SSS0.Px1.p1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS2.SSS0.Px1.p1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>. The monolingual version of FastRAT is based on <math id="A3.SS2.SSS0.Px1.p1.2.m2.1" class="ltx_Math" alttext="\text{BERT}_{\text{LARGE}}" display="inline"><semantics id="A3.SS2.SSS0.Px1.p1.2.m2.1a"><msub id="A3.SS2.SSS0.Px1.p1.2.m2.1.1" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.cmml"><mtext id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.2" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.2a.cmml">BERT</mtext><mtext id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.3" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.3a.cmml">LARGE</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.SS2.SSS0.Px1.p1.2.m2.1b"><apply id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.cmml" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.1.cmml" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1">subscript</csymbol><ci id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.2a.cmml" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.2"><mtext id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.2.cmml" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.2">BERT</mtext></ci><ci id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.3a.cmml" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.3"><mtext mathsize="70%" id="A3.SS2.SSS0.Px1.p1.2.m2.1.1.3.cmml" xref="A3.SS2.SSS0.Px1.p1.2.m2.1.1.3">LARGE</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS2.SSS0.Px1.p1.2.m2.1c">\text{BERT}_{\text{LARGE}}</annotation></semantics></math> while its cross-lingual variant on XLM-RoBERTa-large.</p>
</div>
</section>
</section>
<section id="A3.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">C.3 </span>OpenAI Models</h3>

<div id="A3.SS3.p1" class="ltx_para">
<p id="A3.SS3.p1.3" class="ltx_p">We use <span id="A3.SS3.p1.3.1" class="ltx_text ltx_font_typewriter">gpt-4</span> (<span id="A3.SS3.p1.3.2" class="ltx_text ltx_font_typewriter">gpt-4-0613</span>) and <span id="A3.SS3.p1.3.3" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo</span> (<span id="A3.SS3.p1.3.4" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo-0613</span>) for our experiments. For decoding, sampling is disabled and the maximum number of tokens to generate is set to 256. A single experiment, on the S<span id="A3.SS3.p1.3.5" class="ltx_text ltx_font_smallcaps">pider</span> development set using our approach with <math id="A3.SS3.p1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A3.SS3.p1.1.m1.1a"><msub id="A3.SS3.p1.1.m1.1.1" xref="A3.SS3.p1.1.m1.1.1.cmml"><mtext id="A3.SS3.p1.1.m1.1.1.2" xref="A3.SS3.p1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A3.SS3.p1.1.m1.1.1.3" xref="A3.SS3.p1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.SS3.p1.1.m1.1b"><apply id="A3.SS3.p1.1.m1.1.1.cmml" xref="A3.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="A3.SS3.p1.1.m1.1.1.1.cmml" xref="A3.SS3.p1.1.m1.1.1">subscript</csymbol><ci id="A3.SS3.p1.1.m1.1.1.2a.cmml" xref="A3.SS3.p1.1.m1.1.1.2"><mtext id="A3.SS3.p1.1.m1.1.1.2.cmml" xref="A3.SS3.p1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A3.SS3.p1.1.m1.1.1.3a.cmml" xref="A3.SS3.p1.1.m1.1.1.3"><mtext mathsize="70%" id="A3.SS3.p1.1.m1.1.1.3.cmml" xref="A3.SS3.p1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS3.p1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> as the approximator and dynamic database context selection costs around <math id="A3.SS3.p1.2.m2.1" class="ltx_Math" alttext="\$0.8" display="inline"><semantics id="A3.SS3.p1.2.m2.1a"><mrow id="A3.SS3.p1.2.m2.1.1" xref="A3.SS3.p1.2.m2.1.1.cmml"><mo rspace="0.167em" id="A3.SS3.p1.2.m2.1.1.1" xref="A3.SS3.p1.2.m2.1.1.1.cmml">$</mo><mn id="A3.SS3.p1.2.m2.1.1.2" xref="A3.SS3.p1.2.m2.1.1.2.cmml">0.8</mn></mrow><annotation-xml encoding="MathML-Content" id="A3.SS3.p1.2.m2.1b"><apply id="A3.SS3.p1.2.m2.1.1.cmml" xref="A3.SS3.p1.2.m2.1.1"><csymbol cd="latexml" id="A3.SS3.p1.2.m2.1.1.1.cmml" xref="A3.SS3.p1.2.m2.1.1.1">currency-dollar</csymbol><cn type="float" id="A3.SS3.p1.2.m2.1.1.2.cmml" xref="A3.SS3.p1.2.m2.1.1.2">0.8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS3.p1.2.m2.1c">\$0.8</annotation></semantics></math> and <math id="A3.SS3.p1.3.m3.1" class="ltx_Math" alttext="\$16.5" display="inline"><semantics id="A3.SS3.p1.3.m3.1a"><mrow id="A3.SS3.p1.3.m3.1.1" xref="A3.SS3.p1.3.m3.1.1.cmml"><mo rspace="0.167em" id="A3.SS3.p1.3.m3.1.1.1" xref="A3.SS3.p1.3.m3.1.1.1.cmml">$</mo><mn id="A3.SS3.p1.3.m3.1.1.2" xref="A3.SS3.p1.3.m3.1.1.2.cmml">16.5</mn></mrow><annotation-xml encoding="MathML-Content" id="A3.SS3.p1.3.m3.1b"><apply id="A3.SS3.p1.3.m3.1.1.cmml" xref="A3.SS3.p1.3.m3.1.1"><csymbol cd="latexml" id="A3.SS3.p1.3.m3.1.1.1.cmml" xref="A3.SS3.p1.3.m3.1.1.1">currency-dollar</csymbol><cn type="float" id="A3.SS3.p1.3.m3.1.1.2.cmml" xref="A3.SS3.p1.3.m3.1.1.2">16.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.SS3.p1.3.m3.1c">\$16.5</annotation></semantics></math> in the case of <span id="A3.SS3.p1.3.6" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo-0613</span> and <span id="A3.SS3.p1.3.7" class="ltx_text ltx_font_typewriter">gpt-4-0613</span> respectively.</p>
</div>
</section>
<section id="A3.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">C.4 </span>Experiments with Open-Source LLMs</h3>

<div id="A3.SS4.p1" class="ltx_para">
<p id="A3.SS4.p1.1" class="ltx_p">We further conduct experiments with open-source models from the DeepSeek family<span id="footnote13" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">13</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">13</sup><span class="ltx_tag ltx_tag_note">13</span>We use the implementations provided by <a target="_blank" href="https://huggingface.co/deepseek-ai" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://huggingface.co/deepseek-ai</a>.</span></span></span>, that specialise in code generation. Prompting and decoding setups remain consistent across all LLMs (cf. Section <a href="#A3" title="Appendix C Implementation Details ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a> of the Appendix). Table <a href="#A3.T9" title="Table 9 ‣ C.4 Experiments with Open-Source LLMs ‣ Appendix C Implementation Details ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a> summarises the results. We see that our approach can generalise even in the case of open-source LLM alternatives. Interestingly, our scores using <span id="A3.SS4.p1.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span> are comparable to the scores when using <span id="A3.SS4.p1.1.2" class="ltx_text ltx_font_typewriter">gpt-3.5-turbo-0613</span> across all approximators. Inference experiments are conducted on a machine using <math id="A3.SS4.p1.1.m1.1" class="ltx_math_unparsed" alttext="8\times" display="inline"><semantics id="A3.SS4.p1.1.m1.1a"><mrow id="A3.SS4.p1.1.m1.1b"><mn id="A3.SS4.p1.1.m1.1.1">8</mn><mo lspace="0.222em" id="A3.SS4.p1.1.m1.1.2">×</mo></mrow><annotation encoding="application/x-tex" id="A3.SS4.p1.1.m1.1c">8\times</annotation></semantics></math>NVIDIA-V100 32G GPUs.</p>
</div>
<figure id="A3.T9" class="ltx_table">
<table id="A3.T9.38" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A3.T9.38.39.1" class="ltx_tr">
<th id="A3.T9.38.39.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt" rowspan="2"><span id="A3.T9.38.39.1.1.1" class="ltx_text ltx_font_bold">Model</span></th>
<td id="A3.T9.38.39.1.2" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A3.T9.38.39.1.2.1" class="ltx_text ltx_font_bold">S<span id="A3.T9.38.39.1.2.1.1" class="ltx_text ltx_font_smallcaps">pider</span></span></td>
<td id="A3.T9.38.39.1.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A3.T9.38.39.1.3.1" class="ltx_text ltx_font_bold">S<span id="A3.T9.38.39.1.3.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-DK</span></td>
<td id="A3.T9.38.39.1.4" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A3.T9.38.39.1.4.1" class="ltx_text ltx_font_bold">S<span id="A3.T9.38.39.1.4.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="A3.T9.38.39.1.4.1.2" class="ltx_text ltx_font_smallcaps">eal</span></span></td>
<td id="A3.T9.38.39.1.5" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A3.T9.38.39.1.5.1" class="ltx_text ltx_font_bold">S<span id="A3.T9.38.39.1.5.1.1" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="A3.T9.38.39.1.5.1.2" class="ltx_text ltx_font_smallcaps">yn</span></span></td>
<td id="A3.T9.38.39.1.6" class="ltx_td ltx_align_center ltx_border_tt"><span id="A3.T9.38.39.1.6.1" class="ltx_text ltx_font_bold">CS<span id="A3.T9.38.39.1.6.1.1" class="ltx_text ltx_font_smallcaps">pider</span></span></td>
</tr>
<tr id="A3.T9.38.40.2" class="ltx_tr">
<td id="A3.T9.38.40.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.1.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A3.T9.38.40.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.2.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A3.T9.38.40.2.3" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.3.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A3.T9.38.40.2.4" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.4.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A3.T9.38.40.2.5" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.5.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A3.T9.38.40.2.6" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.6.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A3.T9.38.40.2.7" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.7.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A3.T9.38.40.2.8" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.8.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A3.T9.38.40.2.9" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T9.38.40.2.9.1" class="ltx_text ltx_font_bold">EM</span></td>
</tr>
<tr id="A3.T9.38.41.3" class="ltx_tr">
<th id="A3.T9.38.41.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="A3.T9.38.41.3.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-6.7b-instruct</span></th>
<td id="A3.T9.38.41.3.2" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.41.3.3" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.41.3.4" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.41.3.5" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.41.3.6" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.41.3.7" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.41.3.8" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="A3.T9.10.10" class="ltx_tr">
<th id="A3.T9.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ <math id="A3.T9.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A3.T9.1.1.1.m1.1a"><msub id="A3.T9.1.1.1.m1.1.1" xref="A3.T9.1.1.1.m1.1.1.cmml"><mtext id="A3.T9.1.1.1.m1.1.1.2" xref="A3.T9.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A3.T9.1.1.1.m1.1.1.3" xref="A3.T9.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.T9.1.1.1.m1.1b"><apply id="A3.T9.1.1.1.m1.1.1.cmml" xref="A3.T9.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A3.T9.1.1.1.m1.1.1.1.cmml" xref="A3.T9.1.1.1.m1.1.1">subscript</csymbol><ci id="A3.T9.1.1.1.m1.1.1.2a.cmml" xref="A3.T9.1.1.1.m1.1.1.2"><mtext id="A3.T9.1.1.1.m1.1.1.2.cmml" xref="A3.T9.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A3.T9.1.1.1.m1.1.1.3a.cmml" xref="A3.T9.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="A3.T9.1.1.1.m1.1.1.3.cmml" xref="A3.T9.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="A3.T9.2.2.2" class="ltx_td ltx_align_center"><math id="A3.T9.2.2.2.m1.1" class="ltx_Math" alttext="78.6" display="inline"><semantics id="A3.T9.2.2.2.m1.1a"><mn id="A3.T9.2.2.2.m1.1.1" xref="A3.T9.2.2.2.m1.1.1.cmml">78.6</mn><annotation-xml encoding="MathML-Content" id="A3.T9.2.2.2.m1.1b"><cn type="float" id="A3.T9.2.2.2.m1.1.1.cmml" xref="A3.T9.2.2.2.m1.1.1">78.6</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.2.2.2.m1.1c">78.6</annotation></semantics></math></td>
<td id="A3.T9.3.3.3" class="ltx_td ltx_align_center"><math id="A3.T9.3.3.3.m1.1" class="ltx_Math" alttext="64.8" display="inline"><semantics id="A3.T9.3.3.3.m1.1a"><mn id="A3.T9.3.3.3.m1.1.1" xref="A3.T9.3.3.3.m1.1.1.cmml">64.8</mn><annotation-xml encoding="MathML-Content" id="A3.T9.3.3.3.m1.1b"><cn type="float" id="A3.T9.3.3.3.m1.1.1.cmml" xref="A3.T9.3.3.3.m1.1.1">64.8</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.3.3.3.m1.1c">64.8</annotation></semantics></math></td>
<td id="A3.T9.4.4.4" class="ltx_td ltx_align_center"><math id="A3.T9.4.4.4.m1.1" class="ltx_Math" alttext="66.7" display="inline"><semantics id="A3.T9.4.4.4.m1.1a"><mn id="A3.T9.4.4.4.m1.1.1" xref="A3.T9.4.4.4.m1.1.1.cmml">66.7</mn><annotation-xml encoding="MathML-Content" id="A3.T9.4.4.4.m1.1b"><cn type="float" id="A3.T9.4.4.4.m1.1.1.cmml" xref="A3.T9.4.4.4.m1.1.1">66.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.4.4.4.m1.1c">66.7</annotation></semantics></math></td>
<td id="A3.T9.5.5.5" class="ltx_td ltx_align_center"><math id="A3.T9.5.5.5.m1.1" class="ltx_Math" alttext="46.4" display="inline"><semantics id="A3.T9.5.5.5.m1.1a"><mn id="A3.T9.5.5.5.m1.1.1" xref="A3.T9.5.5.5.m1.1.1.cmml">46.4</mn><annotation-xml encoding="MathML-Content" id="A3.T9.5.5.5.m1.1b"><cn type="float" id="A3.T9.5.5.5.m1.1.1.cmml" xref="A3.T9.5.5.5.m1.1.1">46.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.5.5.5.m1.1c">46.4</annotation></semantics></math></td>
<td id="A3.T9.6.6.6" class="ltx_td ltx_align_center"><math id="A3.T9.6.6.6.m1.1" class="ltx_Math" alttext="73.2" display="inline"><semantics id="A3.T9.6.6.6.m1.1a"><mn id="A3.T9.6.6.6.m1.1.1" xref="A3.T9.6.6.6.m1.1.1.cmml">73.2</mn><annotation-xml encoding="MathML-Content" id="A3.T9.6.6.6.m1.1b"><cn type="float" id="A3.T9.6.6.6.m1.1.1.cmml" xref="A3.T9.6.6.6.m1.1.1">73.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.6.6.6.m1.1c">73.2</annotation></semantics></math></td>
<td id="A3.T9.7.7.7" class="ltx_td ltx_align_center"><math id="A3.T9.7.7.7.m1.1" class="ltx_Math" alttext="55.7" display="inline"><semantics id="A3.T9.7.7.7.m1.1a"><mn id="A3.T9.7.7.7.m1.1.1" xref="A3.T9.7.7.7.m1.1.1.cmml">55.7</mn><annotation-xml encoding="MathML-Content" id="A3.T9.7.7.7.m1.1b"><cn type="float" id="A3.T9.7.7.7.m1.1.1.cmml" xref="A3.T9.7.7.7.m1.1.1">55.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.7.7.7.m1.1c">55.7</annotation></semantics></math></td>
<td id="A3.T9.8.8.8" class="ltx_td ltx_align_center"><math id="A3.T9.8.8.8.m1.1" class="ltx_Math" alttext="66.0" display="inline"><semantics id="A3.T9.8.8.8.m1.1a"><mn id="A3.T9.8.8.8.m1.1.1" xref="A3.T9.8.8.8.m1.1.1.cmml">66.0</mn><annotation-xml encoding="MathML-Content" id="A3.T9.8.8.8.m1.1b"><cn type="float" id="A3.T9.8.8.8.m1.1.1.cmml" xref="A3.T9.8.8.8.m1.1.1">66.0</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.8.8.8.m1.1c">66.0</annotation></semantics></math></td>
<td id="A3.T9.9.9.9" class="ltx_td ltx_align_center"><math id="A3.T9.9.9.9.m1.1" class="ltx_Math" alttext="49.5" display="inline"><semantics id="A3.T9.9.9.9.m1.1a"><mn id="A3.T9.9.9.9.m1.1.1" xref="A3.T9.9.9.9.m1.1.1.cmml">49.5</mn><annotation-xml encoding="MathML-Content" id="A3.T9.9.9.9.m1.1b"><cn type="float" id="A3.T9.9.9.9.m1.1.1.cmml" xref="A3.T9.9.9.9.m1.1.1">49.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.9.9.9.m1.1c">49.5</annotation></semantics></math></td>
<td id="A3.T9.10.10.10" class="ltx_td ltx_align_center"><math id="A3.T9.10.10.10.m1.1" class="ltx_Math" alttext="53.0" display="inline"><semantics id="A3.T9.10.10.10.m1.1a"><mn id="A3.T9.10.10.10.m1.1.1" xref="A3.T9.10.10.10.m1.1.1.cmml">53.0</mn><annotation-xml encoding="MathML-Content" id="A3.T9.10.10.10.m1.1b"><cn type="float" id="A3.T9.10.10.10.m1.1.1.cmml" xref="A3.T9.10.10.10.m1.1.1">53.0</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.10.10.10.m1.1c">53.0</annotation></semantics></math></td>
</tr>
<tr id="A3.T9.19.19" class="ltx_tr">
<th id="A3.T9.19.19.10" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ Graphix-T5)</th>
<td id="A3.T9.11.11.1" class="ltx_td ltx_align_center"><math id="A3.T9.11.11.1.m1.1" class="ltx_Math" alttext="79.5" display="inline"><semantics id="A3.T9.11.11.1.m1.1a"><mn id="A3.T9.11.11.1.m1.1.1" xref="A3.T9.11.11.1.m1.1.1.cmml">79.5</mn><annotation-xml encoding="MathML-Content" id="A3.T9.11.11.1.m1.1b"><cn type="float" id="A3.T9.11.11.1.m1.1.1.cmml" xref="A3.T9.11.11.1.m1.1.1">79.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.11.11.1.m1.1c">79.5</annotation></semantics></math></td>
<td id="A3.T9.12.12.2" class="ltx_td ltx_align_center"><math id="A3.T9.12.12.2.m1.1" class="ltx_Math" alttext="64.8" display="inline"><semantics id="A3.T9.12.12.2.m1.1a"><mn id="A3.T9.12.12.2.m1.1.1" xref="A3.T9.12.12.2.m1.1.1.cmml">64.8</mn><annotation-xml encoding="MathML-Content" id="A3.T9.12.12.2.m1.1b"><cn type="float" id="A3.T9.12.12.2.m1.1.1.cmml" xref="A3.T9.12.12.2.m1.1.1">64.8</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.12.12.2.m1.1c">64.8</annotation></semantics></math></td>
<td id="A3.T9.13.13.3" class="ltx_td ltx_align_center"><math id="A3.T9.13.13.3.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.13.13.3.m1.1a"><mo id="A3.T9.13.13.3.m1.1.1" xref="A3.T9.13.13.3.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.13.13.3.m1.1b"><minus id="A3.T9.13.13.3.m1.1.1.cmml" xref="A3.T9.13.13.3.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.13.13.3.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.14.14.4" class="ltx_td ltx_align_center"><math id="A3.T9.14.14.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.14.14.4.m1.1a"><mo id="A3.T9.14.14.4.m1.1.1" xref="A3.T9.14.14.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.14.14.4.m1.1b"><minus id="A3.T9.14.14.4.m1.1.1.cmml" xref="A3.T9.14.14.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.14.14.4.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.15.15.5" class="ltx_td ltx_align_center"><math id="A3.T9.15.15.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.15.15.5.m1.1a"><mo id="A3.T9.15.15.5.m1.1.1" xref="A3.T9.15.15.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.15.15.5.m1.1b"><minus id="A3.T9.15.15.5.m1.1.1.cmml" xref="A3.T9.15.15.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.15.15.5.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.16.16.6" class="ltx_td ltx_align_center"><math id="A3.T9.16.16.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.16.16.6.m1.1a"><mo id="A3.T9.16.16.6.m1.1.1" xref="A3.T9.16.16.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.16.16.6.m1.1b"><minus id="A3.T9.16.16.6.m1.1.1.cmml" xref="A3.T9.16.16.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.16.16.6.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.17.17.7" class="ltx_td ltx_align_center"><math id="A3.T9.17.17.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.17.17.7.m1.1a"><mo id="A3.T9.17.17.7.m1.1.1" xref="A3.T9.17.17.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.17.17.7.m1.1b"><minus id="A3.T9.17.17.7.m1.1.1.cmml" xref="A3.T9.17.17.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.17.17.7.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.18.18.8" class="ltx_td ltx_align_center"><math id="A3.T9.18.18.8.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.18.18.8.m1.1a"><mo id="A3.T9.18.18.8.m1.1.1" xref="A3.T9.18.18.8.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.18.18.8.m1.1b"><minus id="A3.T9.18.18.8.m1.1.1.cmml" xref="A3.T9.18.18.8.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.18.18.8.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.19.19.9" class="ltx_td ltx_align_center"><math id="A3.T9.19.19.9.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.19.19.9.m1.1a"><mo id="A3.T9.19.19.9.m1.1.1" xref="A3.T9.19.19.9.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.19.19.9.m1.1b"><minus id="A3.T9.19.19.9.m1.1.1.cmml" xref="A3.T9.19.19.9.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.19.19.9.m1.1c">-</annotation></semantics></math></td>
</tr>
<tr id="A3.T9.38.42.4" class="ltx_tr">
<th id="A3.T9.38.42.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="A3.T9.38.42.4.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span></th>
<td id="A3.T9.38.42.4.2" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.42.4.3" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.42.4.4" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.42.4.5" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.42.4.6" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.42.4.7" class="ltx_td ltx_border_t"></td>
<td id="A3.T9.38.42.4.8" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="A3.T9.29.29" class="ltx_tr">
<th id="A3.T9.20.20.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ <math id="A3.T9.20.20.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A3.T9.20.20.1.m1.1a"><msub id="A3.T9.20.20.1.m1.1.1" xref="A3.T9.20.20.1.m1.1.1.cmml"><mtext id="A3.T9.20.20.1.m1.1.1.2" xref="A3.T9.20.20.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A3.T9.20.20.1.m1.1.1.3" xref="A3.T9.20.20.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.T9.20.20.1.m1.1b"><apply id="A3.T9.20.20.1.m1.1.1.cmml" xref="A3.T9.20.20.1.m1.1.1"><csymbol cd="ambiguous" id="A3.T9.20.20.1.m1.1.1.1.cmml" xref="A3.T9.20.20.1.m1.1.1">subscript</csymbol><ci id="A3.T9.20.20.1.m1.1.1.2a.cmml" xref="A3.T9.20.20.1.m1.1.1.2"><mtext id="A3.T9.20.20.1.m1.1.1.2.cmml" xref="A3.T9.20.20.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A3.T9.20.20.1.m1.1.1.3a.cmml" xref="A3.T9.20.20.1.m1.1.1.3"><mtext mathsize="70%" id="A3.T9.20.20.1.m1.1.1.3.cmml" xref="A3.T9.20.20.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.20.20.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="A3.T9.21.21.2" class="ltx_td ltx_align_center"><math id="A3.T9.21.21.2.m1.1" class="ltx_Math" alttext="81.5" display="inline"><semantics id="A3.T9.21.21.2.m1.1a"><mn id="A3.T9.21.21.2.m1.1.1" xref="A3.T9.21.21.2.m1.1.1.cmml">81.5</mn><annotation-xml encoding="MathML-Content" id="A3.T9.21.21.2.m1.1b"><cn type="float" id="A3.T9.21.21.2.m1.1.1.cmml" xref="A3.T9.21.21.2.m1.1.1">81.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.21.21.2.m1.1c">81.5</annotation></semantics></math></td>
<td id="A3.T9.22.22.3" class="ltx_td ltx_align_center"><math id="A3.T9.22.22.3.m1.1" class="ltx_Math" alttext="62.1" display="inline"><semantics id="A3.T9.22.22.3.m1.1a"><mn id="A3.T9.22.22.3.m1.1.1" xref="A3.T9.22.22.3.m1.1.1.cmml">62.1</mn><annotation-xml encoding="MathML-Content" id="A3.T9.22.22.3.m1.1b"><cn type="float" id="A3.T9.22.22.3.m1.1.1.cmml" xref="A3.T9.22.22.3.m1.1.1">62.1</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.22.22.3.m1.1c">62.1</annotation></semantics></math></td>
<td id="A3.T9.23.23.4" class="ltx_td ltx_align_center"><math id="A3.T9.23.23.4.m1.1" class="ltx_Math" alttext="70.5" display="inline"><semantics id="A3.T9.23.23.4.m1.1a"><mn id="A3.T9.23.23.4.m1.1.1" xref="A3.T9.23.23.4.m1.1.1.cmml">70.5</mn><annotation-xml encoding="MathML-Content" id="A3.T9.23.23.4.m1.1b"><cn type="float" id="A3.T9.23.23.4.m1.1.1.cmml" xref="A3.T9.23.23.4.m1.1.1">70.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.23.23.4.m1.1c">70.5</annotation></semantics></math></td>
<td id="A3.T9.24.24.5" class="ltx_td ltx_align_center"><math id="A3.T9.24.24.5.m1.1" class="ltx_Math" alttext="46.4" display="inline"><semantics id="A3.T9.24.24.5.m1.1a"><mn id="A3.T9.24.24.5.m1.1.1" xref="A3.T9.24.24.5.m1.1.1.cmml">46.4</mn><annotation-xml encoding="MathML-Content" id="A3.T9.24.24.5.m1.1b"><cn type="float" id="A3.T9.24.24.5.m1.1.1.cmml" xref="A3.T9.24.24.5.m1.1.1">46.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.24.24.5.m1.1c">46.4</annotation></semantics></math></td>
<td id="A3.T9.25.25.6" class="ltx_td ltx_align_center"><math id="A3.T9.25.25.6.m1.1" class="ltx_Math" alttext="77.4" display="inline"><semantics id="A3.T9.25.25.6.m1.1a"><mn id="A3.T9.25.25.6.m1.1.1" xref="A3.T9.25.25.6.m1.1.1.cmml">77.4</mn><annotation-xml encoding="MathML-Content" id="A3.T9.25.25.6.m1.1b"><cn type="float" id="A3.T9.25.25.6.m1.1.1.cmml" xref="A3.T9.25.25.6.m1.1.1">77.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.25.25.6.m1.1c">77.4</annotation></semantics></math></td>
<td id="A3.T9.26.26.7" class="ltx_td ltx_align_center"><math id="A3.T9.26.26.7.m1.1" class="ltx_Math" alttext="59.3" display="inline"><semantics id="A3.T9.26.26.7.m1.1a"><mn id="A3.T9.26.26.7.m1.1.1" xref="A3.T9.26.26.7.m1.1.1.cmml">59.3</mn><annotation-xml encoding="MathML-Content" id="A3.T9.26.26.7.m1.1b"><cn type="float" id="A3.T9.26.26.7.m1.1.1.cmml" xref="A3.T9.26.26.7.m1.1.1">59.3</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.26.26.7.m1.1c">59.3</annotation></semantics></math></td>
<td id="A3.T9.27.27.8" class="ltx_td ltx_align_center"><math id="A3.T9.27.27.8.m1.1" class="ltx_Math" alttext="68.7" display="inline"><semantics id="A3.T9.27.27.8.m1.1a"><mn id="A3.T9.27.27.8.m1.1.1" xref="A3.T9.27.27.8.m1.1.1.cmml">68.7</mn><annotation-xml encoding="MathML-Content" id="A3.T9.27.27.8.m1.1b"><cn type="float" id="A3.T9.27.27.8.m1.1.1.cmml" xref="A3.T9.27.27.8.m1.1.1">68.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.27.27.8.m1.1c">68.7</annotation></semantics></math></td>
<td id="A3.T9.28.28.9" class="ltx_td ltx_align_center"><math id="A3.T9.28.28.9.m1.1" class="ltx_Math" alttext="49.5" display="inline"><semantics id="A3.T9.28.28.9.m1.1a"><mn id="A3.T9.28.28.9.m1.1.1" xref="A3.T9.28.28.9.m1.1.1.cmml">49.5</mn><annotation-xml encoding="MathML-Content" id="A3.T9.28.28.9.m1.1b"><cn type="float" id="A3.T9.28.28.9.m1.1.1.cmml" xref="A3.T9.28.28.9.m1.1.1">49.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.28.28.9.m1.1c">49.5</annotation></semantics></math></td>
<td id="A3.T9.29.29.10" class="ltx_td ltx_align_center"><math id="A3.T9.29.29.10.m1.1" class="ltx_Math" alttext="55.9" display="inline"><semantics id="A3.T9.29.29.10.m1.1a"><mn id="A3.T9.29.29.10.m1.1.1" xref="A3.T9.29.29.10.m1.1.1.cmml">55.9</mn><annotation-xml encoding="MathML-Content" id="A3.T9.29.29.10.m1.1b"><cn type="float" id="A3.T9.29.29.10.m1.1.1.cmml" xref="A3.T9.29.29.10.m1.1.1">55.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.29.29.10.m1.1c">55.9</annotation></semantics></math></td>
</tr>
<tr id="A3.T9.38.38" class="ltx_tr">
<th id="A3.T9.38.38.10" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">Ours (w/ Graphix-T5)</th>
<td id="A3.T9.30.30.1" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.30.30.1.m1.1" class="ltx_Math" alttext="83.4" display="inline"><semantics id="A3.T9.30.30.1.m1.1a"><mn id="A3.T9.30.30.1.m1.1.1" xref="A3.T9.30.30.1.m1.1.1.cmml">83.4</mn><annotation-xml encoding="MathML-Content" id="A3.T9.30.30.1.m1.1b"><cn type="float" id="A3.T9.30.30.1.m1.1.1.cmml" xref="A3.T9.30.30.1.m1.1.1">83.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.30.30.1.m1.1c">83.4</annotation></semantics></math></td>
<td id="A3.T9.31.31.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.31.31.2.m1.1" class="ltx_Math" alttext="64.7" display="inline"><semantics id="A3.T9.31.31.2.m1.1a"><mn id="A3.T9.31.31.2.m1.1.1" xref="A3.T9.31.31.2.m1.1.1.cmml">64.7</mn><annotation-xml encoding="MathML-Content" id="A3.T9.31.31.2.m1.1b"><cn type="float" id="A3.T9.31.31.2.m1.1.1.cmml" xref="A3.T9.31.31.2.m1.1.1">64.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.31.31.2.m1.1c">64.7</annotation></semantics></math></td>
<td id="A3.T9.32.32.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.32.32.3.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.32.32.3.m1.1a"><mo id="A3.T9.32.32.3.m1.1.1" xref="A3.T9.32.32.3.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.32.32.3.m1.1b"><minus id="A3.T9.32.32.3.m1.1.1.cmml" xref="A3.T9.32.32.3.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.32.32.3.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.33.33.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.33.33.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.33.33.4.m1.1a"><mo id="A3.T9.33.33.4.m1.1.1" xref="A3.T9.33.33.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.33.33.4.m1.1b"><minus id="A3.T9.33.33.4.m1.1.1.cmml" xref="A3.T9.33.33.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.33.33.4.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.34.34.5" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.34.34.5.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.34.34.5.m1.1a"><mo id="A3.T9.34.34.5.m1.1.1" xref="A3.T9.34.34.5.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.34.34.5.m1.1b"><minus id="A3.T9.34.34.5.m1.1.1.cmml" xref="A3.T9.34.34.5.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.34.34.5.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.35.35.6" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.35.35.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.35.35.6.m1.1a"><mo id="A3.T9.35.35.6.m1.1.1" xref="A3.T9.35.35.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.35.35.6.m1.1b"><minus id="A3.T9.35.35.6.m1.1.1.cmml" xref="A3.T9.35.35.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.35.35.6.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.36.36.7" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.36.36.7.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.36.36.7.m1.1a"><mo id="A3.T9.36.36.7.m1.1.1" xref="A3.T9.36.36.7.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.36.36.7.m1.1b"><minus id="A3.T9.36.36.7.m1.1.1.cmml" xref="A3.T9.36.36.7.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.36.36.7.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.37.37.8" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.37.37.8.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.37.37.8.m1.1a"><mo id="A3.T9.37.37.8.m1.1.1" xref="A3.T9.37.37.8.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.37.37.8.m1.1b"><minus id="A3.T9.37.37.8.m1.1.1.cmml" xref="A3.T9.37.37.8.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.37.37.8.m1.1c">-</annotation></semantics></math></td>
<td id="A3.T9.38.38.9" class="ltx_td ltx_align_center ltx_border_bb"><math id="A3.T9.38.38.9.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A3.T9.38.38.9.m1.1a"><mo id="A3.T9.38.38.9.m1.1.1" xref="A3.T9.38.38.9.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A3.T9.38.38.9.m1.1b"><minus id="A3.T9.38.38.9.m1.1.1.cmml" xref="A3.T9.38.38.9.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.38.38.9.m1.1c">-</annotation></semantics></math></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 9: </span>Execution (EX) and exact match (EM) accuracy scores of our approach using DeepSeek family models, on the development splits of S<span id="A3.T9.49.1" class="ltx_text ltx_font_smallcaps">pider</span> and CS<span id="A3.T9.50.2" class="ltx_text ltx_font_smallcaps">pider</span>, and the S<span id="A3.T9.51.3" class="ltx_text ltx_font_smallcaps">pider</span>-DK, S<span id="A3.T9.52.4" class="ltx_text ltx_font_smallcaps">pider</span>-R<span id="A3.T9.53.5" class="ltx_text ltx_font_smallcaps">eal</span> and S<span id="A3.T9.54.6" class="ltx_text ltx_font_smallcaps">pider</span>-S<span id="A3.T9.55.7" class="ltx_text ltx_font_smallcaps">yn</span> test splits. CS<span id="A3.T9.56.8" class="ltx_text ltx_font_smallcaps">pider</span> results are using only <math id="A3.T9.40.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A3.T9.40.m1.1b"><msub id="A3.T9.40.m1.1.1" xref="A3.T9.40.m1.1.1.cmml"><mtext id="A3.T9.40.m1.1.1.2" xref="A3.T9.40.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A3.T9.40.m1.1.1.3" xref="A3.T9.40.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A3.T9.40.m1.1c"><apply id="A3.T9.40.m1.1.1.cmml" xref="A3.T9.40.m1.1.1"><csymbol cd="ambiguous" id="A3.T9.40.m1.1.1.1.cmml" xref="A3.T9.40.m1.1.1">subscript</csymbol><ci id="A3.T9.40.m1.1.1.2a.cmml" xref="A3.T9.40.m1.1.1.2"><mtext id="A3.T9.40.m1.1.1.2.cmml" xref="A3.T9.40.m1.1.1.2">FastRAT</mtext></ci><ci id="A3.T9.40.m1.1.1.3a.cmml" xref="A3.T9.40.m1.1.1.3"><mtext mathsize="70%" id="A3.T9.40.m1.1.1.3.cmml" xref="A3.T9.40.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.T9.40.m1.1d">\text{FastRAT}_{\text{ext}}</annotation></semantics></math> as approximator.</figcaption>
</figure>
</section>
</section>
<section id="A4" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix D </span>S<span id="A4.1.1" class="ltx_text ltx_font_smallcaps">pider</span> and CS<span id="A4.2.2" class="ltx_text ltx_font_smallcaps">pider</span> Experiments</h2>

<div id="A4.p1" class="ltx_para">
<p id="A4.p1.6" class="ltx_p">We report experiments on CS<span id="A4.p1.6.1" class="ltx_text ltx_font_smallcaps">pider</span> <cite class="ltx_cite ltx_citemacro_cite">Min et al. (<a href="#bib.bib15" title="" class="ltx_ref">2019</a>)</cite> and S<span id="A4.p1.6.2" class="ltx_text ltx_font_smallcaps">pider</span> <cite class="ltx_cite ltx_citemacro_cite">Yu et al. (<a href="#bib.bib32" title="" class="ltx_ref">2018</a>)</cite>, which contain database schema information and examples in Chinese and English respectively.
Since CS<span id="A4.p1.6.3" class="ltx_text ltx_font_smallcaps">pider</span> is a translated version of the S<span id="A4.p1.6.4" class="ltx_text ltx_font_smallcaps">pider</span> dataset, the characteristics of the two with respect to structure and number of examples are identical. Both datasets contain <math id="A4.p1.1.m1.2" class="ltx_Math" alttext="8,659" display="inline"><semantics id="A4.p1.1.m1.2a"><mrow id="A4.p1.1.m1.2.3.2" xref="A4.p1.1.m1.2.3.1.cmml"><mn id="A4.p1.1.m1.1.1" xref="A4.p1.1.m1.1.1.cmml">8</mn><mo id="A4.p1.1.m1.2.3.2.1" xref="A4.p1.1.m1.2.3.1.cmml">,</mo><mn id="A4.p1.1.m1.2.2" xref="A4.p1.1.m1.2.2.cmml">659</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.p1.1.m1.2b"><list id="A4.p1.1.m1.2.3.1.cmml" xref="A4.p1.1.m1.2.3.2"><cn type="integer" id="A4.p1.1.m1.1.1.cmml" xref="A4.p1.1.m1.1.1">8</cn><cn type="integer" id="A4.p1.1.m1.2.2.cmml" xref="A4.p1.1.m1.2.2">659</cn></list></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.1.m1.2c">8,659</annotation></semantics></math> examples of questions and SQL queries along with their relevant SQL schemata (i.e. <math id="A4.p1.2.m2.1" class="ltx_Math" alttext="146" display="inline"><semantics id="A4.p1.2.m2.1a"><mn id="A4.p1.2.m2.1.1" xref="A4.p1.2.m2.1.1.cmml">146</mn><annotation-xml encoding="MathML-Content" id="A4.p1.2.m2.1b"><cn type="integer" id="A4.p1.2.m2.1.1.cmml" xref="A4.p1.2.m2.1.1">146</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.2.m2.1c">146</annotation></semantics></math> unique databases). The development and test<span id="footnote14" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">14</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">14</sup><span class="ltx_tag ltx_tag_note">14</span>Since the <math id="footnote14.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="footnote14.m1.1b"><mn id="footnote14.m1.1.1" xref="footnote14.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="footnote14.m1.1c"><cn type="integer" id="footnote14.m1.1.1.cmml" xref="footnote14.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="footnote14.m1.1d">1</annotation></semantics></math>st of March <math id="footnote14.m2.1" class="ltx_Math" alttext="2024" display="inline"><semantics id="footnote14.m2.1b"><mn id="footnote14.m2.1.1" xref="footnote14.m2.1.1.cmml">2024</mn><annotation-xml encoding="MathML-Content" id="footnote14.m2.1c"><cn type="integer" id="footnote14.m2.1.1.cmml" xref="footnote14.m2.1.1">2024</cn></annotation-xml><annotation encoding="application/x-tex" id="footnote14.m2.1d">2024</annotation></semantics></math>, the test sets of both Spider and CSpider have become publicly available.</span></span></span> sets consist of <math id="A4.p1.3.m3.2" class="ltx_Math" alttext="1,034" display="inline"><semantics id="A4.p1.3.m3.2a"><mrow id="A4.p1.3.m3.2.3.2" xref="A4.p1.3.m3.2.3.1.cmml"><mn id="A4.p1.3.m3.1.1" xref="A4.p1.3.m3.1.1.cmml">1</mn><mo id="A4.p1.3.m3.2.3.2.1" xref="A4.p1.3.m3.2.3.1.cmml">,</mo><mn id="A4.p1.3.m3.2.2" xref="A4.p1.3.m3.2.2.cmml">034</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.p1.3.m3.2b"><list id="A4.p1.3.m3.2.3.1.cmml" xref="A4.p1.3.m3.2.3.2"><cn type="integer" id="A4.p1.3.m3.1.1.cmml" xref="A4.p1.3.m3.1.1">1</cn><cn type="integer" id="A4.p1.3.m3.2.2.cmml" xref="A4.p1.3.m3.2.2">034</cn></list></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.3.m3.2c">1,034</annotation></semantics></math>, on <math id="A4.p1.4.m4.1" class="ltx_Math" alttext="20" display="inline"><semantics id="A4.p1.4.m4.1a"><mn id="A4.p1.4.m4.1.1" xref="A4.p1.4.m4.1.1.cmml">20</mn><annotation-xml encoding="MathML-Content" id="A4.p1.4.m4.1b"><cn type="integer" id="A4.p1.4.m4.1.1.cmml" xref="A4.p1.4.m4.1.1">20</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.4.m4.1c">20</annotation></semantics></math> unique databases and <math id="A4.p1.5.m5.2" class="ltx_Math" alttext="2,147" display="inline"><semantics id="A4.p1.5.m5.2a"><mrow id="A4.p1.5.m5.2.3.2" xref="A4.p1.5.m5.2.3.1.cmml"><mn id="A4.p1.5.m5.1.1" xref="A4.p1.5.m5.1.1.cmml">2</mn><mo id="A4.p1.5.m5.2.3.2.1" xref="A4.p1.5.m5.2.3.1.cmml">,</mo><mn id="A4.p1.5.m5.2.2" xref="A4.p1.5.m5.2.2.cmml">147</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.p1.5.m5.2b"><list id="A4.p1.5.m5.2.3.1.cmml" xref="A4.p1.5.m5.2.3.2"><cn type="integer" id="A4.p1.5.m5.1.1.cmml" xref="A4.p1.5.m5.1.1">2</cn><cn type="integer" id="A4.p1.5.m5.2.2.cmml" xref="A4.p1.5.m5.2.2">147</cn></list></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.5.m5.2c">2,147</annotation></semantics></math>, on <math id="A4.p1.6.m6.1" class="ltx_Math" alttext="40" display="inline"><semantics id="A4.p1.6.m6.1a"><mn id="A4.p1.6.m6.1.1" xref="A4.p1.6.m6.1.1.cmml">40</mn><annotation-xml encoding="MathML-Content" id="A4.p1.6.m6.1b"><cn type="integer" id="A4.p1.6.m6.1.1.cmml" xref="A4.p1.6.m6.1.1">40</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.6.m6.1c">40</annotation></semantics></math> unique databases, respectively, and none of the relevant databases are seen in the training set. Due to the scarcity of works reporting test scores on these benchmarks, we chose not to include our results in the main body of our manuscript. Table <a href="#A4.T10" title="Table 10 ‣ Appendix D Spider and CSpider Experiments ‣ Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a> shows the performance of our framework with respect to execution and exact match accuracy scores on the test splits of S<span id="A4.p1.6.5" class="ltx_text ltx_font_smallcaps">pider</span> and CS<span id="A4.p1.6.6" class="ltx_text ltx_font_smallcaps">pider</span>.</p>
</div>
<figure id="A4.T10" class="ltx_table">
<table id="A4.T10.64" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="A4.T10.64.65.1" class="ltx_tr">
<th id="A4.T10.64.65.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt" rowspan="2"><span id="A4.T10.64.65.1.1.1" class="ltx_text ltx_font_bold">Model</span></th>
<td id="A4.T10.64.65.1.2" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A4.T10.64.65.1.2.1" class="ltx_text ltx_font_bold">Easy</span></td>
<td id="A4.T10.64.65.1.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A4.T10.64.65.1.3.1" class="ltx_text ltx_font_bold">Medium</span></td>
<td id="A4.T10.64.65.1.4" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A4.T10.64.65.1.4.1" class="ltx_text ltx_font_bold">Hard</span></td>
<td id="A4.T10.64.65.1.5" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A4.T10.64.65.1.5.1" class="ltx_text ltx_font_bold">Extra</span></td>
<td id="A4.T10.64.65.1.6" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="A4.T10.64.65.1.6.1" class="ltx_text ltx_font_bold">All</span></td>
</tr>
<tr id="A4.T10.64.66.2" class="ltx_tr">
<td id="A4.T10.64.66.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.1.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A4.T10.64.66.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.2.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A4.T10.64.66.2.3" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.3.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A4.T10.64.66.2.4" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.4.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A4.T10.64.66.2.5" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.5.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A4.T10.64.66.2.6" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.6.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A4.T10.64.66.2.7" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.7.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A4.T10.64.66.2.8" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.8.1" class="ltx_text ltx_font_bold">EM</span></td>
<td id="A4.T10.64.66.2.9" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.9.1" class="ltx_text ltx_font_bold">EX</span></td>
<td id="A4.T10.64.66.2.10" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.T10.64.66.2.10.1" class="ltx_text ltx_font_bold">EM</span></td>
</tr>
<tr id="A4.T10.64.67.3" class="ltx_tr">
<th id="A4.T10.64.67.3.1" class="ltx_td ltx_th ltx_th_row ltx_border_t"></th>
<td id="A4.T10.64.67.3.2" class="ltx_td ltx_align_center ltx_border_t" colspan="10"><span id="A4.T10.64.67.3.2.1" class="ltx_text ltx_font_bold">S<span id="A4.T10.64.67.3.2.1.1" class="ltx_text ltx_font_smallcaps">pider</span></span></td>
</tr>
<tr id="A4.T10.11.11" class="ltx_tr">
<th id="A4.T10.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><math id="A4.T10.1.1.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A4.T10.1.1.1.m1.1a"><msub id="A4.T10.1.1.1.m1.1.1" xref="A4.T10.1.1.1.m1.1.1.cmml"><mtext id="A4.T10.1.1.1.m1.1.1.2" xref="A4.T10.1.1.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A4.T10.1.1.1.m1.1.1.3" xref="A4.T10.1.1.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A4.T10.1.1.1.m1.1b"><apply id="A4.T10.1.1.1.m1.1.1.cmml" xref="A4.T10.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T10.1.1.1.m1.1.1.1.cmml" xref="A4.T10.1.1.1.m1.1.1">subscript</csymbol><ci id="A4.T10.1.1.1.m1.1.1.2a.cmml" xref="A4.T10.1.1.1.m1.1.1.2"><mtext id="A4.T10.1.1.1.m1.1.1.2.cmml" xref="A4.T10.1.1.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A4.T10.1.1.1.m1.1.1.3a.cmml" xref="A4.T10.1.1.1.m1.1.1.3"><mtext mathsize="70%" id="A4.T10.1.1.1.m1.1.1.3.cmml" xref="A4.T10.1.1.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.1.1.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<td id="A4.T10.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.2.2.2.m1.1" class="ltx_Math" alttext="86.2" display="inline"><semantics id="A4.T10.2.2.2.m1.1a"><mn id="A4.T10.2.2.2.m1.1.1" xref="A4.T10.2.2.2.m1.1.1.cmml">86.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.2.2.2.m1.1b"><cn type="float" id="A4.T10.2.2.2.m1.1.1.cmml" xref="A4.T10.2.2.2.m1.1.1">86.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.2.2.2.m1.1c">86.2</annotation></semantics></math></td>
<td id="A4.T10.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.3.3.3.m1.1" class="ltx_Math" alttext="81.3" display="inline"><semantics id="A4.T10.3.3.3.m1.1a"><mn id="A4.T10.3.3.3.m1.1.1" xref="A4.T10.3.3.3.m1.1.1.cmml">81.3</mn><annotation-xml encoding="MathML-Content" id="A4.T10.3.3.3.m1.1b"><cn type="float" id="A4.T10.3.3.3.m1.1.1.cmml" xref="A4.T10.3.3.3.m1.1.1">81.3</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.3.3.3.m1.1c">81.3</annotation></semantics></math></td>
<td id="A4.T10.4.4.4" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.4.4.4.m1.1" class="ltx_Math" alttext="72.2" display="inline"><semantics id="A4.T10.4.4.4.m1.1a"><mn id="A4.T10.4.4.4.m1.1.1" xref="A4.T10.4.4.4.m1.1.1.cmml">72.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.4.4.4.m1.1b"><cn type="float" id="A4.T10.4.4.4.m1.1.1.cmml" xref="A4.T10.4.4.4.m1.1.1">72.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.4.4.4.m1.1c">72.2</annotation></semantics></math></td>
<td id="A4.T10.5.5.5" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.5.5.5.m1.1" class="ltx_Math" alttext="66.0" display="inline"><semantics id="A4.T10.5.5.5.m1.1a"><mn id="A4.T10.5.5.5.m1.1.1" xref="A4.T10.5.5.5.m1.1.1.cmml">66.0</mn><annotation-xml encoding="MathML-Content" id="A4.T10.5.5.5.m1.1b"><cn type="float" id="A4.T10.5.5.5.m1.1.1.cmml" xref="A4.T10.5.5.5.m1.1.1">66.0</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.5.5.5.m1.1c">66.0</annotation></semantics></math></td>
<td id="A4.T10.6.6.6" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.6.6.6.m1.1" class="ltx_Math" alttext="60.0" display="inline"><semantics id="A4.T10.6.6.6.m1.1a"><mn id="A4.T10.6.6.6.m1.1.1" xref="A4.T10.6.6.6.m1.1.1.cmml">60.0</mn><annotation-xml encoding="MathML-Content" id="A4.T10.6.6.6.m1.1b"><cn type="float" id="A4.T10.6.6.6.m1.1.1.cmml" xref="A4.T10.6.6.6.m1.1.1">60.0</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.6.6.6.m1.1c">60.0</annotation></semantics></math></td>
<td id="A4.T10.7.7.7" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.7.7.7.m1.1" class="ltx_Math" alttext="51.6" display="inline"><semantics id="A4.T10.7.7.7.m1.1a"><mn id="A4.T10.7.7.7.m1.1.1" xref="A4.T10.7.7.7.m1.1.1.cmml">51.6</mn><annotation-xml encoding="MathML-Content" id="A4.T10.7.7.7.m1.1b"><cn type="float" id="A4.T10.7.7.7.m1.1.1.cmml" xref="A4.T10.7.7.7.m1.1.1">51.6</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.7.7.7.m1.1c">51.6</annotation></semantics></math></td>
<td id="A4.T10.8.8.8" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.8.8.8.m1.1" class="ltx_Math" alttext="48.5" display="inline"><semantics id="A4.T10.8.8.8.m1.1a"><mn id="A4.T10.8.8.8.m1.1.1" xref="A4.T10.8.8.8.m1.1.1.cmml">48.5</mn><annotation-xml encoding="MathML-Content" id="A4.T10.8.8.8.m1.1b"><cn type="float" id="A4.T10.8.8.8.m1.1.1.cmml" xref="A4.T10.8.8.8.m1.1.1">48.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.8.8.8.m1.1c">48.5</annotation></semantics></math></td>
<td id="A4.T10.9.9.9" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.9.9.9.m1.1" class="ltx_Math" alttext="33.9" display="inline"><semantics id="A4.T10.9.9.9.m1.1a"><mn id="A4.T10.9.9.9.m1.1.1" xref="A4.T10.9.9.9.m1.1.1.cmml">33.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.9.9.9.m1.1b"><cn type="float" id="A4.T10.9.9.9.m1.1.1.cmml" xref="A4.T10.9.9.9.m1.1.1">33.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.9.9.9.m1.1c">33.9</annotation></semantics></math></td>
<td id="A4.T10.10.10.10" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.10.10.10.m1.1" class="ltx_Math" alttext="68.7" display="inline"><semantics id="A4.T10.10.10.10.m1.1a"><mn id="A4.T10.10.10.10.m1.1.1" xref="A4.T10.10.10.10.m1.1.1.cmml">68.7</mn><annotation-xml encoding="MathML-Content" id="A4.T10.10.10.10.m1.1b"><cn type="float" id="A4.T10.10.10.10.m1.1.1.cmml" xref="A4.T10.10.10.10.m1.1.1">68.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.10.10.10.m1.1c">68.7</annotation></semantics></math></td>
<td id="A4.T10.11.11.11" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.11.11.11.m1.1" class="ltx_Math" alttext="60.9" display="inline"><semantics id="A4.T10.11.11.11.m1.1a"><mn id="A4.T10.11.11.11.m1.1.1" xref="A4.T10.11.11.11.m1.1.1.cmml">60.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.11.11.11.m1.1b"><cn type="float" id="A4.T10.11.11.11.m1.1.1.cmml" xref="A4.T10.11.11.11.m1.1.1">60.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.11.11.11.m1.1c">60.9</annotation></semantics></math></td>
</tr>
<tr id="A4.T10.64.68.4" class="ltx_tr">
<th id="A4.T10.64.68.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="11"><span id="A4.T10.64.68.4.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span></th>
</tr>
<tr id="A4.T10.21.21" class="ltx_tr">
<th id="A4.T10.21.21.11" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ Graphix-T5)</th>
<td id="A4.T10.12.12.1" class="ltx_td ltx_align_center"><math id="A4.T10.12.12.1.m1.1" class="ltx_Math" alttext="89.6" display="inline"><semantics id="A4.T10.12.12.1.m1.1a"><mn id="A4.T10.12.12.1.m1.1.1" xref="A4.T10.12.12.1.m1.1.1.cmml">89.6</mn><annotation-xml encoding="MathML-Content" id="A4.T10.12.12.1.m1.1b"><cn type="float" id="A4.T10.12.12.1.m1.1.1.cmml" xref="A4.T10.12.12.1.m1.1.1">89.6</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.12.12.1.m1.1c">89.6</annotation></semantics></math></td>
<td id="A4.T10.13.13.2" class="ltx_td ltx_align_center"><math id="A4.T10.13.13.2.m1.1" class="ltx_Math" alttext="85.5" display="inline"><semantics id="A4.T10.13.13.2.m1.1a"><mn id="A4.T10.13.13.2.m1.1.1" xref="A4.T10.13.13.2.m1.1.1.cmml">85.5</mn><annotation-xml encoding="MathML-Content" id="A4.T10.13.13.2.m1.1b"><cn type="float" id="A4.T10.13.13.2.m1.1.1.cmml" xref="A4.T10.13.13.2.m1.1.1">85.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.13.13.2.m1.1c">85.5</annotation></semantics></math></td>
<td id="A4.T10.14.14.3" class="ltx_td ltx_align_center"><math id="A4.T10.14.14.3.m1.1" class="ltx_Math" alttext="88.6" display="inline"><semantics id="A4.T10.14.14.3.m1.1a"><mn id="A4.T10.14.14.3.m1.1.1" xref="A4.T10.14.14.3.m1.1.1.cmml">88.6</mn><annotation-xml encoding="MathML-Content" id="A4.T10.14.14.3.m1.1b"><cn type="float" id="A4.T10.14.14.3.m1.1.1.cmml" xref="A4.T10.14.14.3.m1.1.1">88.6</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.14.14.3.m1.1c">88.6</annotation></semantics></math></td>
<td id="A4.T10.15.15.4" class="ltx_td ltx_align_center"><math id="A4.T10.15.15.4.m1.1" class="ltx_Math" alttext="66.4" display="inline"><semantics id="A4.T10.15.15.4.m1.1a"><mn id="A4.T10.15.15.4.m1.1.1" xref="A4.T10.15.15.4.m1.1.1.cmml">66.4</mn><annotation-xml encoding="MathML-Content" id="A4.T10.15.15.4.m1.1b"><cn type="float" id="A4.T10.15.15.4.m1.1.1.cmml" xref="A4.T10.15.15.4.m1.1.1">66.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.15.15.4.m1.1c">66.4</annotation></semantics></math></td>
<td id="A4.T10.16.16.5" class="ltx_td ltx_align_center"><math id="A4.T10.16.16.5.m1.1" class="ltx_Math" alttext="73.9" display="inline"><semantics id="A4.T10.16.16.5.m1.1a"><mn id="A4.T10.16.16.5.m1.1.1" xref="A4.T10.16.16.5.m1.1.1.cmml">73.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.16.16.5.m1.1b"><cn type="float" id="A4.T10.16.16.5.m1.1.1.cmml" xref="A4.T10.16.16.5.m1.1.1">73.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.16.16.5.m1.1c">73.9</annotation></semantics></math></td>
<td id="A4.T10.17.17.6" class="ltx_td ltx_align_center"><math id="A4.T10.17.17.6.m1.1" class="ltx_Math" alttext="50.1" display="inline"><semantics id="A4.T10.17.17.6.m1.1a"><mn id="A4.T10.17.17.6.m1.1.1" xref="A4.T10.17.17.6.m1.1.1.cmml">50.1</mn><annotation-xml encoding="MathML-Content" id="A4.T10.17.17.6.m1.1b"><cn type="float" id="A4.T10.17.17.6.m1.1.1.cmml" xref="A4.T10.17.17.6.m1.1.1">50.1</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.17.17.6.m1.1c">50.1</annotation></semantics></math></td>
<td id="A4.T10.18.18.7" class="ltx_td ltx_align_center"><math id="A4.T10.18.18.7.m1.1" class="ltx_Math" alttext="58.8" display="inline"><semantics id="A4.T10.18.18.7.m1.1a"><mn id="A4.T10.18.18.7.m1.1.1" xref="A4.T10.18.18.7.m1.1.1.cmml">58.8</mn><annotation-xml encoding="MathML-Content" id="A4.T10.18.18.7.m1.1b"><cn type="float" id="A4.T10.18.18.7.m1.1.1.cmml" xref="A4.T10.18.18.7.m1.1.1">58.8</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.18.18.7.m1.1c">58.8</annotation></semantics></math></td>
<td id="A4.T10.19.19.8" class="ltx_td ltx_align_center"><math id="A4.T10.19.19.8.m1.1" class="ltx_Math" alttext="28.0" display="inline"><semantics id="A4.T10.19.19.8.m1.1a"><mn id="A4.T10.19.19.8.m1.1.1" xref="A4.T10.19.19.8.m1.1.1.cmml">28.0</mn><annotation-xml encoding="MathML-Content" id="A4.T10.19.19.8.m1.1b"><cn type="float" id="A4.T10.19.19.8.m1.1.1.cmml" xref="A4.T10.19.19.8.m1.1.1">28.0</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.19.19.8.m1.1c">28.0</annotation></semantics></math></td>
<td id="A4.T10.20.20.9" class="ltx_td ltx_align_center"><math id="A4.T10.20.20.9.m1.1" class="ltx_Math" alttext="80.7" display="inline"><semantics id="A4.T10.20.20.9.m1.1a"><mn id="A4.T10.20.20.9.m1.1.1" xref="A4.T10.20.20.9.m1.1.1.cmml">80.7</mn><annotation-xml encoding="MathML-Content" id="A4.T10.20.20.9.m1.1b"><cn type="float" id="A4.T10.20.20.9.m1.1.1.cmml" xref="A4.T10.20.20.9.m1.1.1">80.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.20.20.9.m1.1c">80.7</annotation></semantics></math></td>
<td id="A4.T10.21.21.10" class="ltx_td ltx_align_center"><math id="A4.T10.21.21.10.m1.1" class="ltx_Math" alttext="60.7" display="inline"><semantics id="A4.T10.21.21.10.m1.1a"><mn id="A4.T10.21.21.10.m1.1.1" xref="A4.T10.21.21.10.m1.1.1.cmml">60.7</mn><annotation-xml encoding="MathML-Content" id="A4.T10.21.21.10.m1.1b"><cn type="float" id="A4.T10.21.21.10.m1.1.1.cmml" xref="A4.T10.21.21.10.m1.1.1">60.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.21.21.10.m1.1c">60.7</annotation></semantics></math></td>
</tr>
<tr id="A4.T10.64.69.5" class="ltx_tr">
<th id="A4.T10.64.69.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="11"><span id="A4.T10.64.69.5.1.1" class="ltx_text ltx_font_typewriter">gpt-4</span></th>
</tr>
<tr id="A4.T10.31.31" class="ltx_tr">
<th id="A4.T10.31.31.11" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ Graphix-T5)</th>
<td id="A4.T10.22.22.1" class="ltx_td ltx_align_center"><math id="A4.T10.22.22.1.m1.1" class="ltx_Math" alttext="91.9" display="inline"><semantics id="A4.T10.22.22.1.m1.1a"><mn id="A4.T10.22.22.1.m1.1.1" xref="A4.T10.22.22.1.m1.1.1.cmml">91.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.22.22.1.m1.1b"><cn type="float" id="A4.T10.22.22.1.m1.1.1.cmml" xref="A4.T10.22.22.1.m1.1.1">91.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.22.22.1.m1.1c">91.9</annotation></semantics></math></td>
<td id="A4.T10.23.23.2" class="ltx_td ltx_align_center"><math id="A4.T10.23.23.2.m1.1" class="ltx_Math" alttext="87.4" display="inline"><semantics id="A4.T10.23.23.2.m1.1a"><mn id="A4.T10.23.23.2.m1.1.1" xref="A4.T10.23.23.2.m1.1.1.cmml">87.4</mn><annotation-xml encoding="MathML-Content" id="A4.T10.23.23.2.m1.1b"><cn type="float" id="A4.T10.23.23.2.m1.1.1.cmml" xref="A4.T10.23.23.2.m1.1.1">87.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.23.23.2.m1.1c">87.4</annotation></semantics></math></td>
<td id="A4.T10.24.24.3" class="ltx_td ltx_align_center"><math id="A4.T10.24.24.3.m1.1" class="ltx_Math" alttext="90.3" display="inline"><semantics id="A4.T10.24.24.3.m1.1a"><mn id="A4.T10.24.24.3.m1.1.1" xref="A4.T10.24.24.3.m1.1.1.cmml">90.3</mn><annotation-xml encoding="MathML-Content" id="A4.T10.24.24.3.m1.1b"><cn type="float" id="A4.T10.24.24.3.m1.1.1.cmml" xref="A4.T10.24.24.3.m1.1.1">90.3</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.24.24.3.m1.1c">90.3</annotation></semantics></math></td>
<td id="A4.T10.25.25.4" class="ltx_td ltx_align_center"><math id="A4.T10.25.25.4.m1.1" class="ltx_Math" alttext="80.4" display="inline"><semantics id="A4.T10.25.25.4.m1.1a"><mn id="A4.T10.25.25.4.m1.1.1" xref="A4.T10.25.25.4.m1.1.1.cmml">80.4</mn><annotation-xml encoding="MathML-Content" id="A4.T10.25.25.4.m1.1b"><cn type="float" id="A4.T10.25.25.4.m1.1.1.cmml" xref="A4.T10.25.25.4.m1.1.1">80.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.25.25.4.m1.1c">80.4</annotation></semantics></math></td>
<td id="A4.T10.26.26.5" class="ltx_td ltx_align_center"><math id="A4.T10.26.26.5.m1.1" class="ltx_Math" alttext="81.2" display="inline"><semantics id="A4.T10.26.26.5.m1.1a"><mn id="A4.T10.26.26.5.m1.1.1" xref="A4.T10.26.26.5.m1.1.1.cmml">81.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.26.26.5.m1.1b"><cn type="float" id="A4.T10.26.26.5.m1.1.1.cmml" xref="A4.T10.26.26.5.m1.1.1">81.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.26.26.5.m1.1c">81.2</annotation></semantics></math></td>
<td id="A4.T10.27.27.6" class="ltx_td ltx_align_center"><math id="A4.T10.27.27.6.m1.1" class="ltx_Math" alttext="66.1" display="inline"><semantics id="A4.T10.27.27.6.m1.1a"><mn id="A4.T10.27.27.6.m1.1.1" xref="A4.T10.27.27.6.m1.1.1.cmml">66.1</mn><annotation-xml encoding="MathML-Content" id="A4.T10.27.27.6.m1.1b"><cn type="float" id="A4.T10.27.27.6.m1.1.1.cmml" xref="A4.T10.27.27.6.m1.1.1">66.1</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.27.27.6.m1.1c">66.1</annotation></semantics></math></td>
<td id="A4.T10.28.28.7" class="ltx_td ltx_align_center"><math id="A4.T10.28.28.7.m1.1" class="ltx_Math" alttext="74.2" display="inline"><semantics id="A4.T10.28.28.7.m1.1a"><mn id="A4.T10.28.28.7.m1.1.1" xref="A4.T10.28.28.7.m1.1.1.cmml">74.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.28.28.7.m1.1b"><cn type="float" id="A4.T10.28.28.7.m1.1.1.cmml" xref="A4.T10.28.28.7.m1.1.1">74.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.28.28.7.m1.1c">74.2</annotation></semantics></math></td>
<td id="A4.T10.29.29.8" class="ltx_td ltx_align_center"><math id="A4.T10.29.29.8.m1.1" class="ltx_Math" alttext="47.6" display="inline"><semantics id="A4.T10.29.29.8.m1.1a"><mn id="A4.T10.29.29.8.m1.1.1" xref="A4.T10.29.29.8.m1.1.1.cmml">47.6</mn><annotation-xml encoding="MathML-Content" id="A4.T10.29.29.8.m1.1b"><cn type="float" id="A4.T10.29.29.8.m1.1.1.cmml" xref="A4.T10.29.29.8.m1.1.1">47.6</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.29.29.8.m1.1c">47.6</annotation></semantics></math></td>
<td id="A4.T10.30.30.9" class="ltx_td ltx_align_center"><math id="A4.T10.30.30.9.m1.1" class="ltx_Math" alttext="86.0" display="inline"><semantics id="A4.T10.30.30.9.m1.1a"><mn id="A4.T10.30.30.9.m1.1.1" xref="A4.T10.30.30.9.m1.1.1.cmml">86.0</mn><annotation-xml encoding="MathML-Content" id="A4.T10.30.30.9.m1.1b"><cn type="float" id="A4.T10.30.30.9.m1.1.1.cmml" xref="A4.T10.30.30.9.m1.1.1">86.0</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.30.30.9.m1.1c">86.0</annotation></semantics></math></td>
<td id="A4.T10.31.31.10" class="ltx_td ltx_align_center"><math id="A4.T10.31.31.10.m1.1" class="ltx_Math" alttext="73.4" display="inline"><semantics id="A4.T10.31.31.10.m1.1a"><mn id="A4.T10.31.31.10.m1.1.1" xref="A4.T10.31.31.10.m1.1.1.cmml">73.4</mn><annotation-xml encoding="MathML-Content" id="A4.T10.31.31.10.m1.1b"><cn type="float" id="A4.T10.31.31.10.m1.1.1.cmml" xref="A4.T10.31.31.10.m1.1.1">73.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.31.31.10.m1.1c">73.4</annotation></semantics></math></td>
</tr>
<tr id="A4.T10.64.70.6" class="ltx_tr">
<th id="A4.T10.64.70.6.1" class="ltx_td ltx_th ltx_th_row ltx_border_t"></th>
<td id="A4.T10.64.70.6.2" class="ltx_td ltx_align_center ltx_border_t" colspan="10"><span id="A4.T10.64.70.6.2.1" class="ltx_text ltx_font_bold">CS<span id="A4.T10.64.70.6.2.1.1" class="ltx_text ltx_font_smallcaps">pider</span></span></td>
</tr>
<tr id="A4.T10.42.42" class="ltx_tr">
<th id="A4.T10.32.32.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><math id="A4.T10.32.32.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A4.T10.32.32.1.m1.1a"><msub id="A4.T10.32.32.1.m1.1.1" xref="A4.T10.32.32.1.m1.1.1.cmml"><mtext id="A4.T10.32.32.1.m1.1.1.2" xref="A4.T10.32.32.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A4.T10.32.32.1.m1.1.1.3" xref="A4.T10.32.32.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A4.T10.32.32.1.m1.1b"><apply id="A4.T10.32.32.1.m1.1.1.cmml" xref="A4.T10.32.32.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T10.32.32.1.m1.1.1.1.cmml" xref="A4.T10.32.32.1.m1.1.1">subscript</csymbol><ci id="A4.T10.32.32.1.m1.1.1.2a.cmml" xref="A4.T10.32.32.1.m1.1.1.2"><mtext id="A4.T10.32.32.1.m1.1.1.2.cmml" xref="A4.T10.32.32.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A4.T10.32.32.1.m1.1.1.3a.cmml" xref="A4.T10.32.32.1.m1.1.1.3"><mtext mathsize="70%" id="A4.T10.32.32.1.m1.1.1.3.cmml" xref="A4.T10.32.32.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.32.32.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math></th>
<td id="A4.T10.33.33.2" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.33.33.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.33.33.2.m1.1a"><mo id="A4.T10.33.33.2.m1.1.1" xref="A4.T10.33.33.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.33.33.2.m1.1b"><minus id="A4.T10.33.33.2.m1.1.1.cmml" xref="A4.T10.33.33.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.33.33.2.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.34.34.3" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.34.34.3.m1.1" class="ltx_Math" alttext="67.2" display="inline"><semantics id="A4.T10.34.34.3.m1.1a"><mn id="A4.T10.34.34.3.m1.1.1" xref="A4.T10.34.34.3.m1.1.1.cmml">67.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.34.34.3.m1.1b"><cn type="float" id="A4.T10.34.34.3.m1.1.1.cmml" xref="A4.T10.34.34.3.m1.1.1">67.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.34.34.3.m1.1c">67.2</annotation></semantics></math></td>
<td id="A4.T10.35.35.4" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.35.35.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.35.35.4.m1.1a"><mo id="A4.T10.35.35.4.m1.1.1" xref="A4.T10.35.35.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.35.35.4.m1.1b"><minus id="A4.T10.35.35.4.m1.1.1.cmml" xref="A4.T10.35.35.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.35.35.4.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.36.36.5" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.36.36.5.m1.1" class="ltx_Math" alttext="49.9" display="inline"><semantics id="A4.T10.36.36.5.m1.1a"><mn id="A4.T10.36.36.5.m1.1.1" xref="A4.T10.36.36.5.m1.1.1.cmml">49.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.36.36.5.m1.1b"><cn type="float" id="A4.T10.36.36.5.m1.1.1.cmml" xref="A4.T10.36.36.5.m1.1.1">49.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.36.36.5.m1.1c">49.9</annotation></semantics></math></td>
<td id="A4.T10.37.37.6" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.37.37.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.37.37.6.m1.1a"><mo id="A4.T10.37.37.6.m1.1.1" xref="A4.T10.37.37.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.37.37.6.m1.1b"><minus id="A4.T10.37.37.6.m1.1.1.cmml" xref="A4.T10.37.37.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.37.37.6.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.38.38.7" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.38.38.7.m1.1" class="ltx_Math" alttext="41.5" display="inline"><semantics id="A4.T10.38.38.7.m1.1a"><mn id="A4.T10.38.38.7.m1.1.1" xref="A4.T10.38.38.7.m1.1.1.cmml">41.5</mn><annotation-xml encoding="MathML-Content" id="A4.T10.38.38.7.m1.1b"><cn type="float" id="A4.T10.38.38.7.m1.1.1.cmml" xref="A4.T10.38.38.7.m1.1.1">41.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.38.38.7.m1.1c">41.5</annotation></semantics></math></td>
<td id="A4.T10.39.39.8" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.39.39.8.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.39.39.8.m1.1a"><mo id="A4.T10.39.39.8.m1.1.1" xref="A4.T10.39.39.8.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.39.39.8.m1.1b"><minus id="A4.T10.39.39.8.m1.1.1.cmml" xref="A4.T10.39.39.8.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.39.39.8.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.40.40.9" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.40.40.9.m1.1" class="ltx_Math" alttext="12.9" display="inline"><semantics id="A4.T10.40.40.9.m1.1a"><mn id="A4.T10.40.40.9.m1.1.1" xref="A4.T10.40.40.9.m1.1.1.cmml">12.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.40.40.9.m1.1b"><cn type="float" id="A4.T10.40.40.9.m1.1.1.cmml" xref="A4.T10.40.40.9.m1.1.1">12.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.40.40.9.m1.1c">12.9</annotation></semantics></math></td>
<td id="A4.T10.41.41.10" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.41.41.10.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.41.41.10.m1.1a"><mo id="A4.T10.41.41.10.m1.1.1" xref="A4.T10.41.41.10.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.41.41.10.m1.1b"><minus id="A4.T10.41.41.10.m1.1.1.cmml" xref="A4.T10.41.41.10.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.41.41.10.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.42.42.11" class="ltx_td ltx_align_center ltx_border_t"><math id="A4.T10.42.42.11.m1.1" class="ltx_Math" alttext="45.5" display="inline"><semantics id="A4.T10.42.42.11.m1.1a"><mn id="A4.T10.42.42.11.m1.1.1" xref="A4.T10.42.42.11.m1.1.1.cmml">45.5</mn><annotation-xml encoding="MathML-Content" id="A4.T10.42.42.11.m1.1b"><cn type="float" id="A4.T10.42.42.11.m1.1.1.cmml" xref="A4.T10.42.42.11.m1.1.1">45.5</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.42.42.11.m1.1c">45.5</annotation></semantics></math></td>
</tr>
<tr id="A4.T10.64.71.7" class="ltx_tr">
<th id="A4.T10.64.71.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="11"><span id="A4.T10.64.71.7.1.1" class="ltx_text ltx_font_typewriter">deepseek-coder-33b-instruct</span></th>
</tr>
<tr id="A4.T10.53.53" class="ltx_tr">
<th id="A4.T10.43.43.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Ours (w/ <math id="A4.T10.43.43.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A4.T10.43.43.1.m1.1a"><msub id="A4.T10.43.43.1.m1.1.1" xref="A4.T10.43.43.1.m1.1.1.cmml"><mtext id="A4.T10.43.43.1.m1.1.1.2" xref="A4.T10.43.43.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A4.T10.43.43.1.m1.1.1.3" xref="A4.T10.43.43.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A4.T10.43.43.1.m1.1b"><apply id="A4.T10.43.43.1.m1.1.1.cmml" xref="A4.T10.43.43.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T10.43.43.1.m1.1.1.1.cmml" xref="A4.T10.43.43.1.m1.1.1">subscript</csymbol><ci id="A4.T10.43.43.1.m1.1.1.2a.cmml" xref="A4.T10.43.43.1.m1.1.1.2"><mtext id="A4.T10.43.43.1.m1.1.1.2.cmml" xref="A4.T10.43.43.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A4.T10.43.43.1.m1.1.1.3a.cmml" xref="A4.T10.43.43.1.m1.1.1.3"><mtext mathsize="70%" id="A4.T10.43.43.1.m1.1.1.3.cmml" xref="A4.T10.43.43.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.43.43.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="A4.T10.44.44.2" class="ltx_td ltx_align_center"><math id="A4.T10.44.44.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.44.44.2.m1.1a"><mo id="A4.T10.44.44.2.m1.1.1" xref="A4.T10.44.44.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.44.44.2.m1.1b"><minus id="A4.T10.44.44.2.m1.1.1.cmml" xref="A4.T10.44.44.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.44.44.2.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.45.45.3" class="ltx_td ltx_align_center"><math id="A4.T10.45.45.3.m1.1" class="ltx_Math" alttext="79.7" display="inline"><semantics id="A4.T10.45.45.3.m1.1a"><mn id="A4.T10.45.45.3.m1.1.1" xref="A4.T10.45.45.3.m1.1.1.cmml">79.7</mn><annotation-xml encoding="MathML-Content" id="A4.T10.45.45.3.m1.1b"><cn type="float" id="A4.T10.45.45.3.m1.1.1.cmml" xref="A4.T10.45.45.3.m1.1.1">79.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.45.45.3.m1.1c">79.7</annotation></semantics></math></td>
<td id="A4.T10.46.46.4" class="ltx_td ltx_align_center"><math id="A4.T10.46.46.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.46.46.4.m1.1a"><mo id="A4.T10.46.46.4.m1.1.1" xref="A4.T10.46.46.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.46.46.4.m1.1b"><minus id="A4.T10.46.46.4.m1.1.1.cmml" xref="A4.T10.46.46.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.46.46.4.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.47.47.5" class="ltx_td ltx_align_center"><math id="A4.T10.47.47.5.m1.1" class="ltx_Math" alttext="58.2" display="inline"><semantics id="A4.T10.47.47.5.m1.1a"><mn id="A4.T10.47.47.5.m1.1.1" xref="A4.T10.47.47.5.m1.1.1.cmml">58.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.47.47.5.m1.1b"><cn type="float" id="A4.T10.47.47.5.m1.1.1.cmml" xref="A4.T10.47.47.5.m1.1.1">58.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.47.47.5.m1.1c">58.2</annotation></semantics></math></td>
<td id="A4.T10.48.48.6" class="ltx_td ltx_align_center"><math id="A4.T10.48.48.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.48.48.6.m1.1a"><mo id="A4.T10.48.48.6.m1.1.1" xref="A4.T10.48.48.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.48.48.6.m1.1b"><minus id="A4.T10.48.48.6.m1.1.1.cmml" xref="A4.T10.48.48.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.48.48.6.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.49.49.7" class="ltx_td ltx_align_center"><math id="A4.T10.49.49.7.m1.1" class="ltx_Math" alttext="40.2" display="inline"><semantics id="A4.T10.49.49.7.m1.1a"><mn id="A4.T10.49.49.7.m1.1.1" xref="A4.T10.49.49.7.m1.1.1.cmml">40.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.49.49.7.m1.1b"><cn type="float" id="A4.T10.49.49.7.m1.1.1.cmml" xref="A4.T10.49.49.7.m1.1.1">40.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.49.49.7.m1.1c">40.2</annotation></semantics></math></td>
<td id="A4.T10.50.50.8" class="ltx_td ltx_align_center"></td>
<td id="A4.T10.51.51.9" class="ltx_td ltx_align_center"><math id="A4.T10.51.51.9.m1.1" class="ltx_Math" alttext="22.4" display="inline"><semantics id="A4.T10.51.51.9.m1.1a"><mn id="A4.T10.51.51.9.m1.1.1" xref="A4.T10.51.51.9.m1.1.1.cmml">22.4</mn><annotation-xml encoding="MathML-Content" id="A4.T10.51.51.9.m1.1b"><cn type="float" id="A4.T10.51.51.9.m1.1.1.cmml" xref="A4.T10.51.51.9.m1.1.1">22.4</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.51.51.9.m1.1c">22.4</annotation></semantics></math></td>
<td id="A4.T10.52.52.10" class="ltx_td ltx_align_center"><math id="A4.T10.52.52.10.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.52.52.10.m1.1a"><mo id="A4.T10.52.52.10.m1.1.1" xref="A4.T10.52.52.10.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.52.52.10.m1.1b"><minus id="A4.T10.52.52.10.m1.1.1.cmml" xref="A4.T10.52.52.10.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.52.52.10.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.53.53.11" class="ltx_td ltx_align_center"><math id="A4.T10.53.53.11.m1.1" class="ltx_Math" alttext="52.9" display="inline"><semantics id="A4.T10.53.53.11.m1.1a"><mn id="A4.T10.53.53.11.m1.1.1" xref="A4.T10.53.53.11.m1.1.1.cmml">52.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.53.53.11.m1.1b"><cn type="float" id="A4.T10.53.53.11.m1.1.1.cmml" xref="A4.T10.53.53.11.m1.1.1">52.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.53.53.11.m1.1c">52.9</annotation></semantics></math></td>
</tr>
<tr id="A4.T10.64.72.8" class="ltx_tr">
<th id="A4.T10.64.72.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="A4.T10.64.72.8.1.1" class="ltx_text ltx_font_typewriter">gpt-4</span></th>
<td id="A4.T10.64.72.8.2" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.3" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.4" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.5" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.6" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.7" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.8" class="ltx_td ltx_border_t"></td>
<td id="A4.T10.64.72.8.9" class="ltx_td ltx_border_t"></td>
</tr>
<tr id="A4.T10.64.64" class="ltx_tr">
<th id="A4.T10.54.54.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb">Ours (w/ <math id="A4.T10.54.54.1.m1.1" class="ltx_Math" alttext="\text{FastRAT}_{\text{ext}}" display="inline"><semantics id="A4.T10.54.54.1.m1.1a"><msub id="A4.T10.54.54.1.m1.1.1" xref="A4.T10.54.54.1.m1.1.1.cmml"><mtext id="A4.T10.54.54.1.m1.1.1.2" xref="A4.T10.54.54.1.m1.1.1.2a.cmml">FastRAT</mtext><mtext id="A4.T10.54.54.1.m1.1.1.3" xref="A4.T10.54.54.1.m1.1.1.3a.cmml">ext</mtext></msub><annotation-xml encoding="MathML-Content" id="A4.T10.54.54.1.m1.1b"><apply id="A4.T10.54.54.1.m1.1.1.cmml" xref="A4.T10.54.54.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T10.54.54.1.m1.1.1.1.cmml" xref="A4.T10.54.54.1.m1.1.1">subscript</csymbol><ci id="A4.T10.54.54.1.m1.1.1.2a.cmml" xref="A4.T10.54.54.1.m1.1.1.2"><mtext id="A4.T10.54.54.1.m1.1.1.2.cmml" xref="A4.T10.54.54.1.m1.1.1.2">FastRAT</mtext></ci><ci id="A4.T10.54.54.1.m1.1.1.3a.cmml" xref="A4.T10.54.54.1.m1.1.1.3"><mtext mathsize="70%" id="A4.T10.54.54.1.m1.1.1.3.cmml" xref="A4.T10.54.54.1.m1.1.1.3">ext</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.54.54.1.m1.1c">\text{FastRAT}_{\text{ext}}</annotation></semantics></math>)</th>
<td id="A4.T10.55.55.2" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.55.55.2.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.55.55.2.m1.1a"><mo id="A4.T10.55.55.2.m1.1.1" xref="A4.T10.55.55.2.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.55.55.2.m1.1b"><minus id="A4.T10.55.55.2.m1.1.1.cmml" xref="A4.T10.55.55.2.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.55.55.2.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.56.56.3" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.56.56.3.m1.1" class="ltx_Math" alttext="81.2" display="inline"><semantics id="A4.T10.56.56.3.m1.1a"><mn id="A4.T10.56.56.3.m1.1.1" xref="A4.T10.56.56.3.m1.1.1.cmml">81.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.56.56.3.m1.1b"><cn type="float" id="A4.T10.56.56.3.m1.1.1.cmml" xref="A4.T10.56.56.3.m1.1.1">81.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.56.56.3.m1.1c">81.2</annotation></semantics></math></td>
<td id="A4.T10.57.57.4" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.57.57.4.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.57.57.4.m1.1a"><mo id="A4.T10.57.57.4.m1.1.1" xref="A4.T10.57.57.4.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.57.57.4.m1.1b"><minus id="A4.T10.57.57.4.m1.1.1.cmml" xref="A4.T10.57.57.4.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.57.57.4.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.58.58.5" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.58.58.5.m1.1" class="ltx_Math" alttext="67.7" display="inline"><semantics id="A4.T10.58.58.5.m1.1a"><mn id="A4.T10.58.58.5.m1.1.1" xref="A4.T10.58.58.5.m1.1.1.cmml">67.7</mn><annotation-xml encoding="MathML-Content" id="A4.T10.58.58.5.m1.1b"><cn type="float" id="A4.T10.58.58.5.m1.1.1.cmml" xref="A4.T10.58.58.5.m1.1.1">67.7</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.58.58.5.m1.1c">67.7</annotation></semantics></math></td>
<td id="A4.T10.59.59.6" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.59.59.6.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.59.59.6.m1.1a"><mo id="A4.T10.59.59.6.m1.1.1" xref="A4.T10.59.59.6.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.59.59.6.m1.1b"><minus id="A4.T10.59.59.6.m1.1.1.cmml" xref="A4.T10.59.59.6.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.59.59.6.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.60.60.7" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.60.60.7.m1.1" class="ltx_Math" alttext="53.2" display="inline"><semantics id="A4.T10.60.60.7.m1.1a"><mn id="A4.T10.60.60.7.m1.1.1" xref="A4.T10.60.60.7.m1.1.1.cmml">53.2</mn><annotation-xml encoding="MathML-Content" id="A4.T10.60.60.7.m1.1b"><cn type="float" id="A4.T10.60.60.7.m1.1.1.cmml" xref="A4.T10.60.60.7.m1.1.1">53.2</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.60.60.7.m1.1c">53.2</annotation></semantics></math></td>
<td id="A4.T10.61.61.8" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.61.61.8.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.61.61.8.m1.1a"><mo id="A4.T10.61.61.8.m1.1.1" xref="A4.T10.61.61.8.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.61.61.8.m1.1b"><minus id="A4.T10.61.61.8.m1.1.1.cmml" xref="A4.T10.61.61.8.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.61.61.8.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.62.62.9" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.62.62.9.m1.1" class="ltx_Math" alttext="29.9" display="inline"><semantics id="A4.T10.62.62.9.m1.1a"><mn id="A4.T10.62.62.9.m1.1.1" xref="A4.T10.62.62.9.m1.1.1.cmml">29.9</mn><annotation-xml encoding="MathML-Content" id="A4.T10.62.62.9.m1.1b"><cn type="float" id="A4.T10.62.62.9.m1.1.1.cmml" xref="A4.T10.62.62.9.m1.1.1">29.9</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.62.62.9.m1.1c">29.9</annotation></semantics></math></td>
<td id="A4.T10.63.63.10" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.63.63.10.m1.1" class="ltx_Math" alttext="-" display="inline"><semantics id="A4.T10.63.63.10.m1.1a"><mo id="A4.T10.63.63.10.m1.1.1" xref="A4.T10.63.63.10.m1.1.1.cmml">−</mo><annotation-xml encoding="MathML-Content" id="A4.T10.63.63.10.m1.1b"><minus id="A4.T10.63.63.10.m1.1.1.cmml" xref="A4.T10.63.63.10.m1.1.1"></minus></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.63.63.10.m1.1c">-</annotation></semantics></math></td>
<td id="A4.T10.64.64.11" class="ltx_td ltx_align_center ltx_border_bb"><math id="A4.T10.64.64.11.m1.1" class="ltx_Math" alttext="61.1" display="inline"><semantics id="A4.T10.64.64.11.m1.1a"><mn id="A4.T10.64.64.11.m1.1.1" xref="A4.T10.64.64.11.m1.1.1.cmml">61.1</mn><annotation-xml encoding="MathML-Content" id="A4.T10.64.64.11.m1.1b"><cn type="float" id="A4.T10.64.64.11.m1.1.1.cmml" xref="A4.T10.64.64.11.m1.1.1">61.1</cn></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.64.64.11.m1.1c">61.1</annotation></semantics></math></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 10: </span>Execution (EX) and exact match (EM) accuracy scores of our framework, on the test splits of S<span id="A4.T10.67.1" class="ltx_text ltx_font_smallcaps">pider</span> and CS<span id="A4.T10.68.2" class="ltx_text ltx_font_smallcaps">pider</span>.</figcaption>
</figure>
</section>
<section id="A5" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix E </span>Baselines</h2>

<div id="A5.p1" class="ltx_para">
<p id="A5.p1.1" class="ltx_p">We compare the performance of our approach against several baselines.
We dichotomize the landscape of baselines in fine-tuning- and prompting-based baselines.</p>
</div>
<section id="A5.SS0.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Fine-tuning-based</h5>

<div id="A5.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="A5.SS0.SSS0.Px1.p1.1" class="ltx_p"><span id="A5.SS0.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_bold">GraPPa</span> uses synthetic data constructed via induced synchronous context-free grammar for pre-training an MLM on the SSP-label classification; <span id="A5.SS0.SSS0.Px1.p1.1.2" class="ltx_text ltx_font_bold">DG-MAML</span> applies meta-learning targeting zero-shot domain generalization. <span id="A5.SS0.SSS0.Px1.p1.1.3" class="ltx_text ltx_font_bold">FastRAT</span> incorporates a decoder-free framework, by directly predicting SQL queries from SSP labels; <span id="A5.SS0.SSS0.Px1.p1.1.4" class="ltx_text ltx_font_bold">Graphix-T5</span> inserts a graph-aware layer into T5 <cite class="ltx_cite ltx_citemacro_cite">Raffel et al. (<a href="#bib.bib19" title="" class="ltx_ref">2020</a>)</cite> to introduce structural inductive bias; <span id="A5.SS0.SSS0.Px1.p1.1.5" class="ltx_text ltx_font_bold">RESDSQL</span> decouples schema linking and SQL skeleton parsing using a framework based on a ranking-enhanced encoder and skeleton-aware decoder; <span id="A5.SS0.SSS0.Px1.p1.1.6" class="ltx_text ltx_font_bold">HG2AST</span> proposes a framework to integrate dedicated structure knowledge by transforming heterogeneous graphs to abstract syntax trees.</p>
</div>
</section>
<section id="A5.SS0.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Prompting-based</h5>

<div id="A5.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="A5.SS0.SSS0.Px2.p1.1" class="ltx_p">Zero-shot prompting with LLMs has been explored by <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib11" title="" class="ltx_ref">Guo et al.</a>; <a href="#bib.bib14" title="" class="ltx_ref">Liu et al.</a>; <a href="#bib.bib18" title="" class="ltx_ref">Pourreza and Rafiei</a></cite>; <span id="A5.SS0.SSS0.Px2.p1.1.2" class="ltx_text ltx_font_bold">C3</span> introduces calibration bias prompting to alleviate LLMs’ biases; <span id="A5.SS0.SSS0.Px2.p1.1.3" class="ltx_text ltx_font_bold">DIN-SQL</span> uses chain-of-thought prompting with pre-defined prompting templates tailored for the assessed question hardness; <span id="A5.SS0.SSS0.Px2.p1.1.4" class="ltx_text ltx_font_bold">DAIL-SQL</span> uses query approximation and SQL skeleton-based similarities for example selection; <span id="A5.SS0.SSS0.Px2.p1.1.5" class="ltx_text ltx_font_bold">SQL-PaLM</span> proposes a framework for <span id="A5.SS0.SSS0.Px2.p1.1.6" class="ltx_text ltx_font_italic">soft</span> column selection and execution-based refinement; <span id="A5.SS0.SSS0.Px2.p1.1.1" class="ltx_text ltx_font_bold">RAG w<math id="A5.SS0.SSS0.Px2.p1.1.1.m1.1" class="ltx_Math" alttext="\mathbf{/}" display="inline"><semantics id="A5.SS0.SSS0.Px2.p1.1.1.m1.1a"><mo id="A5.SS0.SSS0.Px2.p1.1.1.m1.1.1" xref="A5.SS0.SSS0.Px2.p1.1.1.m1.1.1.cmml">/</mo><annotation-xml encoding="MathML-Content" id="A5.SS0.SSS0.Px2.p1.1.1.m1.1b"><divide id="A5.SS0.SSS0.Px2.p1.1.1.m1.1.1.cmml" xref="A5.SS0.SSS0.Px2.p1.1.1.m1.1.1"></divide></annotation-xml><annotation encoding="application/x-tex" id="A5.SS0.SSS0.Px2.p1.1.1.m1.1c">\mathbf{/}</annotation></semantics></math> Rev. Chain</span> augments the input prompt with question skeleton-based example retrieval and an execution-based revision chain.</p>
</div>
</section>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2407.03226" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2407.03227" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2407.03227">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2407.03227" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2407.03228" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Mon Aug  5 16:49:33 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2407.03227
We focus on Text-to-SQL semantic parsing from the perspective of Large Language Models. Motivated by challenges related to the size of commercial database schemata and the deployability of business intelligence solutions, we propose an approach that dynamically retrieves input database information and uses abstract syntax trees to select few-shot examples for in-context learning. Furthermore, we investigate the extent to which an in-parallel semantic parser can be leveraged for generatingapproximatedversions of the expected SQL queries, to support our retrieval. We take this approach to the extreme—we adapt a model consisting of less thanM parameters, to act as an extremely efficient approximator, enhancing it with the ability to process schemata in a parallelised manner. We apply our approach to monolingual and cross-lingual benchmarks for semantic parsing, showing improvements over state-of-the-art baselines. Comprehensive experiments highlight the contribution of modules involved in this retrieval-augmented generation setting, revealing interesting directions for future work.
abstract
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
Abstract
2407.03227
1IntroductionText-to-SQL semantic parsing aims at translating natural language questions into SQL, to facilitate querying relational databases by non-experts.
Given their accessibility benefits, Text-to-SQL applications have become popular recently, with many corporations developing Business Intelligence platforms.The success of Large Language Models (LLMs)  in generalising across diverse Natural Language Processing taskshas fuelled works that looked at how these multi-billion parameter models can be best employed for Text-to-SQL. Recent works in this space have focused on the in-context learning ability of LLMs, demonstrating that significant improvements can be achieved by selecting suitable (question, SQL)  example pairs.
In spite of its underlying benefits, conventional solutions for example selection are usually limited to retrieving examples based solely on the similarity of questions.
Other approaches resort to a preliminary round of parsing whichapproximatesexpected SQL queries, and directly use these approximations in few-shot prompting, or to subsequently select (question, SQL)  pairs by comparing the approximated query to queries within candidate examples.
The approach proposed bytransforms SQL queries into SQL skeletonsand then filters examples by considering overlap token ratio as the similarity between two skeletons. While incorporating SQL skeleton similarity improves over conventional example selection for Text-to-SQL, it can result in structural information loss as exemplified in Table1, where two dissimilar SQL queries are treated as identical. In this paper, we propose a novel approach that selects examples using similarity of normalised SQL Abstract Syntax Trees (ASTs) . We argue that considering the similarity of such hierarchical structures can significantly enhance LLMs’ performance for Text-to-SQL parsing.Apart from example selection, we refine database context input to LLMs by dynamically pruning schemata and selecting values.
From the perspective of LLMs, existing studies achieve improvements by including the full database schema in the prompt and additionallyhintingthe importance of particular schema elements or values.
In this paper, we show that the performance can be boosted with schemata of reduced size.Inspired bythat compute an approximated query for a given input question, we further explore how combinations of a sparse retriever with such an in-parallel semantic parser (we would refer to it asapproximator)  can be used to retrieve relevant database context input to LLMs. For efficiency, we adapt the semantic parser (a decoder-free model withM parameters)  proposed byto process schemata in a parallelised manner.
Using this efficient approximator, our schema pruning strategy selects a relevant sub-schema in order to simplify the task for LLMs and reduce the relevant computational workload. Furthermore, it enables LLM-based Text-to-SQL solutions to handle longer schemata (usually associated with commercial use-cases)  exceeding their context window size.We apply our approach on monolingual (Spider, Spider-DK, Spider-Realand Spider-Syn)  and cross-lingual (CSpider)  benchmarks of different generalisation challenges. We evaluate the applicability of our framework across both closed- and open-source LLMs. Our framework, comprising only a single round of prompting, achieves state-of-the-art performance, outperforming other
baselines which may comprise complex prompting and multiple iterations, when LLMs of equal capacity are involved. Through comprehensive experiments, we highlight strengths and limitations. Our contributions can be summarised as follows:•We propose a novel approach for selecting (question, SQL)  examples using similarity of normalised SQL ASTs.•We take efficient approximation to the extreme, presenting a schema-parallelisable adaptation of the fastest semantic parser to date.•We introduce a framework for dynamically selecting schema elements and database values, offering substantial execution accuracy improvements over prior works while significantly reducing the computational workload of LLMs.•We shed light on the benefits of database value selection and its symbiotic relation to schema pruning for Text-to-SQL LLM prompting.SELECT T2.name, T2.capacity FROM concert AS T1 JOIN stadium AS T2 ON T1.stadium_id = T2.stadium_id WHERE T1.year >= 2014Skeleton:select _ from _ where _SELECT name FROM highschooler WHERE grade = 10Skeleton:select _ from _ where _Table 1:Two SQL queries with identical SQL skeletons.
introduction
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
1Introduction
2407.03227
2PreliminariesLetbe the sequence of tokens of a natural language question for databasewith tablesand columns, whereis the-th column of tableandis the total number of columns in table. Furthermore, letbe the set of all values associated with the databases.t.are the DB value sets associated with respective columns.
The goal of Text-to-SQL semantic parsing is to predict the SQL querygiven thecombination, as follows:(1) For in-context learning, we seek to select pertinent input context including few-shot examples, schema, and database values to simplify the task for LLMs.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
2Preliminaries
2407.03227
3Example Selection using Abstract Syntax TreesOur goal is to identify the most suitable set ofquestion-SQL pairs from an index of examples,, s.t., for maximising the probability of an LLM to predict the correct SQL given:(2) From the perspective of ranking, we consider the relevance score between a candidate exampleand the input.Vanillasemantic search is usually based solely on question embeddings, whereas the structure of SQL queries for similar questions is subject to target databases and can thus differ significantly.To incorporate database context for selecting examples, we propose to re-rank examples retrieved by question embeddings based on normalised SQL ASTs.
Inspired by, our framework utilises a preliminary model to compute an approximated SQL query, structurally similar to the ground truth, givens.t..
Examples are then re-ranked byfor each candidate.AST represents the hierarchical structure of code in a tree form and can be applied to evaluation metrics for code generation. The fact that SQL queries sharing identical abstract meanings may not align with the same syntactic structure poses a challenge for measuring similarity through AST differencing.AST NormalisationAlthough it is infeasible to exhaustively transform a SQL to another equivalent form, we can normalise ASTs to reduce undesired mismatch. Firstly, nodes of identifiers are lowercased and unnecessary references are removed (e.g.<table>.<column>is substituted with<column>if possible) . We then delete nodes that create aliases and map each alias to a copy of the subtree to which it references. For cross-domain settings wherein databases at inference time are unseen in the train set, we mask out nodes of values and identifiers after resolving aliases. Otherwise for in-domain settings we further sort nodes associated with JOIN operations(s)  to ensure the ordering of tables and keys is consistent.AST SimilarityGiven two normalised ASTs, we adopt the Change Distilling algorithmthat computes a list of tree edit operations to transform the source AST to the target AST. Types of tree edit operations include:insert,delete,alignment,moveandupdate. It is essential to note thatmoveoperation relocates a node to a different parent while moving a node within the same parent is analignment. Therefore, we calculate the similarity between ASTs simply as the ratio of alignments to the total number of operations within the list. Examples of our normalisation and AST similarity are provided in AppendixA.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
3Example Selection using Abstract Syntax Trees
2407.03227
4Database Context SelectionApart from relevant question-SQL pairs, prompting for Text-to-SQL parsing requires the context of database schema and values.4.1Schema SelectionWe present a hybrid search strategy that selects a sub-schema given a test question to minimise lengthy and potentially irrelevant schema elements
input to LLMs, while maintaining high recall.Letbe a semantic representation of column. We aggregate the semantic names111Semantic name can refer to simply to the name of a particular or to a concatenation of its name and description.ofand the table it belongs to,, and its corresponding value set in,, as follows:.Given question, we retrieve the most relevant columns usingand. A table is retrieved if any of its columns are retrieved.4.1.1Incorporating for Approximated QueryThe semantic search for schema selection requires a comprehension of the relevance between heterogeneous database information and natural language questions, in addition to interactions across schema elements.
To this end, a trained parser can inherently be a semantic search model for retrieving a sub-schema, where columns and tables are extracted from the approximated query.
We argue that a semantic parser which is performing reasonably on the task, can provide us with an, whose structure would assimilate the structure of the expected final query,. Consequently, we opt to dynamically determine the number of columns to be retrieved byas proportional to the number of unique columns in, returned by the approximator. A sub-schema is then obtained by merging schema elements selected bywith elements from the approximated query.4.1.2Approximating for Longer SchemataTo further reduce the computational workload, we opt for using asmallermodel for computing the approximated query,. However, smaller models usually have shorter context windows (i.e.k tokens) , and, as such, they cannot be easily scaled to the requirements of larger schemata. To this end, we propose an approach that enables transformer-based encoders to process longer schemata, in a parallelised manner.We start with FastRAT, which exploits a decoder-free architecture for efficient text-to-SQL parsing. Given a concatenation of the input natural language questionwith the column and table names of a database schema, FastRAT computes the SQL operation in which each element of the input schema would participate in the expected SQL query. We refer to these SQL operations as SQL Semantic Prediction (SSP)  labels. SQL queries are then deterministically constructed from the predicted SSP labels. We introduce a schema splitting strategy to scale the model up to the requirements of schemata comprising several columns.We augment the input embedding matrix of the model, with two special schema-completion tokens,[full_schema]and[part_schema], which are used for signalling cases in which a full and a partial schema are provided as input respectively. Our goal is to split a schema consisting ofcolumns intosplits s.t. each split includes a maximum, pre-defined number of columns. Each split consists of the question tokens, a single schema-completion token, the table names of the inputand up to a maximumnumber of columns allocated to this particular split (see Algorithm1for further details) .Input:# col. name tokens concat.# flatten concat. of tab. name tokens# q tokens: int1splits;2ifthen3prefix;45else6prefix;78end if9;#one sp per split10fortodo11;1213ifthen14;15splits.append(sp) ;1617;1819end if2021end forReturnsplitsAlgorithm 1Algorithm for splitting a schema into smaller splits.The returned schema splits along with the SSP labels corresponding to the schema elements of each split are treated as independent instances during training. At test time, an input schema is split according to Algorithm1, and the model is input with a batch of the resulting splits. After aggregating the results from all splits, we obtain the SSP label for each column. Inconsistencies across the SSP labels of tables are resolved using majority voting. We refer to this model as.4.2Value SelectionThe inference of LLMs for text-to-SQL parsing can be augmented with column values. We select values for columns in a schema (or a sub-schema)  by simply matching keywords in questions and values. This is based on the assumption that LLMs can generalise to unseen values given a set of representative values; thus, the recall and precision of value selection are less critical. We consider value selection providing additional information for LLMs to discern covert differences among columns. An example of our resulting prompt is shown in AppendixB.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
4Database Context Selection
2407.03227
5ExperimentsWe run experiments using two approximators:and Graphix-T5.
Graphix-T5 is is the approximator used by DAIL-SQL, and is included to facilitate a fair comparison against the closest work to ours.is trained and tested using, unless otherwise stated (cf. Section5.4) . For all experiments, we use 5 (question, SQL)  examples.We test our approach against both closed- and open-source LLMs:(i) gpt-3.5-turbo(gpt-3.5-turbo-0613) ,(ii) gpt-4(gpt-4-0613)  and(iii) deepseek-coder-33b-instruct.Results using additional models from the DeepSeek family are provided in AppendixC.4.5.1DatasetsWe experiment with several SQL datasets, seeking to explore the effectiveness of our approach on both monolingual and cross-lingual setups. Specifically, we report experiments on CSpiderand Spider.
Since CSpideris a translated version of Spiderin Chinese, when it comes to the natural language questions, the characteristics of the two with respect to structure and number of examples are identical.
We focus our evaluation on the development sets222AppendixDincludes results on the test sets., which are used as test sets in our experiments. These splits consists ofexamples of questions onunique databases that are not met at training time.We rely on the training splits to maintain an index of (question, SQL)  examples, one for each dataset. Using these splits, we train a monolingual and a cross-lingual version of.Furthermore, we use popular Spidervariants:(i) Spider-DK,(ii) Spider-Realand(iii) Spider-Synto evaluate zero-shot domain generalisation in English (leveraging the Spider(question, SQL)  examples index) .Consistently with the relevant leaderboards333https://taolusi.github.io/CSpider-explorer/andhttps://yale-lily.github.io/spider, we report results using execution (EX)  and exact match (EM)  accuracy.444EX and EM scores are computed using:https://github.com/taoyds/test-suite-sql-eval.Since CSpidercomes without relevant DB content, we follow previous works, and we focus our evaluation on EM scores.ModelEXEMGraPPaFastRATGraphix-T5RESDSQLdeepseek-coder-33b-instructOurs (w/) Ours (w/ Graphix-T5) PaLM2Few-shotSQL-PaLMtext-davinci-003Zero-shotRAG wRev. Chaingpt-3.5-turboZero-shotC3DAIL-SQLOurs (w/) Ours (w/ Graphix-T5) gpt-4Zero-shotDIN-SQLDAIL-SQLOurs (w/) Ours (w/ Graphix-T5) Table 2:EX and EM accuracies on the development split of Spider. Fine-tuning-based baselines are listed at the top part of the table. Results of our approach are shown with bothand Graphix-T5 as approximators. Thebestmodel is in bold, thesecond bestis underlined, and thebest prompt-based setupis in blue.5.2BaselinesWe dichotomize the landscape of baselines in fine-tuning- and prompting-based baselines. Further details are provided in AppendixE.Fine-tuning-based(i) GraPPa,(ii) DG-MAML,(iii) FastRAT,(iv) Graphix-T5,(v) RESDSQLand(vi) HG2AST.Prompting-basedZero-shot LLM prompting has been explored by;(i) C3introduces calibration bias for LLM prompting;(ii) DIN-SQLuses chain-of-thought prompting with pre-defined prompting templates tailored for the assessed question hardness;(iii) DAIL-SQLuses query approximation and SQL skeleton-based similarities for example selection;(iv) SQL-PaLMproposes a framework forsoftcolumn selection and execution-based refinement;(v) RAG wRev. Chainaugments the input prompt with question skeleton-based example retrieval and an execution-based revision chain.ModelSpider-DKSpider-RealSpider-SynCSpiderEXEMEXEMEXEMEMRAT-SQL + BERTDG-MAMLFastRAT48.5HG2AST555Without using question translation;EM when question translation is used.RESDSQLdeepseek-coder-33b-instructOurs (w/) gpt-3.5-turboZero-shotDAIL-SQLOurs (w/) PaLM2Few-shotSQL-PaLMgpt-4DAIL-SQLOurs (w/) Table 3:Results on Spider-DK, Spider-Real, Spider-Synand CSpider. Fine-tuning-based baselines are listed at the top. Thebestmodel is in bold,second bestis underlined, and thebest prompt-based setupis in blue.5.3Text-to-SQL EvaluationTable2and3summarise the results of our approach withdeepseek-coder-33b-instruct,gpt-3.5-turboandgpt-4against the baselines. Our approach, comprising a single-prompting round, surpasses other LLM-based solutions, that incorporate several prompting iterations, for LLMs of the same capacity. We note consistent improvements over DAIL-SQL, the closest work to ours, even whenis used as approximator (i.e. a model consisting ofM vs theB parameters that DAIL-SQL’s approximator is using) . For the same approximator, our framework is able to meet, performance standards of DAIL-SQL (equipped withgpt-4and an additional self-consistency prompting step)  using an open-source model as backbone LLM, by achieving shorter prompts in a single prompting step.Spiderresults are consistent with the results across the various Spider variants and CSpider666In CSpider, questions are fully translated in Chinese while the DB content remains in English. Due to this limitation, DB schema and content selection are disabled.(Table3) .
Our approach leveringand AST-based re-ranking for example selection out-performs other prompting-based solution, and is in-line with the scores of state-of-the-art fine-tuning-based baselines. Whilegpt-4is the most capable model within our framework (with this being more noticeable in the case of Spider-Syn) , we observe surprising findings with DeepSeek with which in many cases our approach can surpass much more computationally expensive alternatives based on larger closed-source LLMs. Our findings remain consistent withsince the EM scores of prompting-based methods fall behind those of their fine-tuning based counterparts.5.3.1Schema Selection EvaluationApproximatorSchema Selection SetupRecallSchema Shorten.EXEMOracleGold QueryNABM25 (top-) BM25 (top-) Approx. QueryApprox. Query + BM25 (top-) Approx. Query + BM25 (top-) Approx. Query + BM25 (dynamic top-) Graphix-T5NAGraphix-T5Approx. QueryGraphix-T5Approx. Query + BM25 (dynamic top-) Table 4:Recall, Schema Shortening, EX and EM scores (usinggpt-3.5-turbo)  across different schema selection setups, on the development split of Spider. Value selection is enabled for the selected columns across all setups. For the oracle setup, we report performance upper-bounds usingonlythe schema elements from the gold query.We evaluate our proposed schema selection strategy in a two-fold manner, given that value selection is applied for selected columns. Firstly, we use recall and schema shortening (rate)  to compute averaged metrics across all samples showcasing the extent to which(i) the most relevant schema elements are successfully retrieved, and(ii) the size of the resulting schema, after selection, with respect to its original size. Secondly, we explore how the performance of the end-system changes across different schema pruning settings by reporting EX and EM scores. Recall is the percentage of samples for which all ground-truth schema elements are selected. Schema shortening is the number of schema elements that are excluded divided by the total number of schema elements. Results are summarised in Table4.The benefits of schema selection are apparent in the oracle setup, in which only schema elements from the gold query are included in the prompt (cf. Table8) . In this setup, the highest execution accuracy is achieved while filtering outof the original schema on average. We note that our approach of coupling the schema elements returned in the approximated query with the ones returned by BM25 navigates a healthy trade-off between maximising recall and reducing processing of unnecessary schema elements. We also notice that our strategy of dynamically determining the number of retained schema elements per input (cf. Section4.1.1)  results in improvements compared to static top-determination. For roughly the same extent of schema shortening (i.e. by comparing scores with dynamic top-against top-) , the results with the former are higher across all metrics.5.3.2Ablation StudyTable5shows a comprehensive ablation study for the efficacy of our database context selection, and example selection methods including DAILand AST.
We consistently notice improvement when selecting examples using AST, for the same approximator. Interestingly, the performance gap is increasing the better the approximator becomes, leading to an improvementin the case of an oracle approximator. This finding is in agreement with our hypothesis that AST re-ranking can preserve structural information for more precise example selection when. The inclusion of combined schema and value selection (SVS)  leads to further improvements when coupled with example selection based on AST or DAIL.ApproximatorSelectionEXEMNAQuestion Similarity74.752.3DAIL78.661.4DAIL + SVS81.362.3AST79.363.6AST + VS80.463.8AST + SS78.963.8AST + SVS82.065.7Graphix-T5DAIL77.861.9DAIL + SVS81.063.7AST79.865.6AST + VS81.466.4AST + SS80.266.2AST + SVS83.068.8OracleDAIL79.163.2DAIL + SVS82.566.0AST81.067.6AST + VS82.668.1AST + SS82.569.6AST + SVS84.671.3Table 5:EX and EM scores on the development set of Spider, withgpt-3.5-turbo, across different approximators, and selection setups: example selection (with DAIL or AST) , schema selection (SS) , value selection (VS) , and schema & value selection (SVS) . We report results from oracle approximator using gold queries.5.4Schema SplittingWe evaluate the effect of splitting a schema intosplits, usingfor schema selection. Figure1shows EX scores across different maximum number of columns per schema split () , on the development set of Spider. We see that the EX scores of our approach remain consistent across different. The performance in the case where particular schemata from the development set are split intoorsplits (i.e. fororrespectively)  is identical to the scores where schemata are split using the defaultwith whichhas been trained.Figure 1:Execution accuracy scores on on the development set of Spideracross different maximum numbers of columns per schema split,. The results of our approach, usinggpt-3.5-turbo, are presented across different Spider-query difficulty levels.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
5Experiments
2407.03227
6DiscussionAre there any theoretical performance upper limits for example selection using AST?For each data instance in the development set of Spider, we compute the average AST similarity between the approximated query and each SQL query that is included (after example selection)  in the corresponding prompt. In Table6, we measure EX scores on the development set of Spider, across different AST-similarity intervals. We see an obvious correlation between execution accuracy and AST scores–execution accuracy is higher for higher AST scores. Besides highlighting an empirical, execution accuracy upper-bound, in the case of test questions whose SQL structure is well-covered (i.e. with high AST score)  in the examples space, our approach can hint data instances that might be challenging for the current configuration, without even requiring to prompt the target LLM or executing the resulting SQL against a database instance.
Challenging data instances can be taken into consideration with respect to the existence of insufficient examples into support the expected SQL structure or a potentially harmful approximator.AST IntervalApproximatorOracleGraphixTable 6:EX scores on the Spiderdevelopment set usinggpt-3.5-turbo, across different average AST-similarity intervals.Is the choice of approximator critical?Although our AST re-ranking and schema selection benefit from more accurate SQL predicted by a stronger approximator, the choice of approximator depends on the desired trade-off between effectiveness and efficiency in practice.is overtimes faster than Graphix-T5on an A100G GPU, while the resulting difference, within our framework, in EX on the Spiderdevelopment set is(Table5) .Does schema selection improve the performance?In Table5, we noted that performing schema selection (i.e. SS)  without DB value selection does not necessarily lead to performance improvements. This is in partial agreement withthat hard column selection can be harmful for the end-to-end performance, and can be attributed to the drop of recall, when less capable approximators are involved. Nonetheless, as we note in Section5.3.2, the combination of schema and value selection (SVS)  can consistently improve EX and EM, while significantly reducing the LLM token-processing cost due to shortened schema.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
6Discussion
2407.03227
7Related WorkSignificant number of recent works have looked at how LLMs can be employed in Text-to-SQL scenarios.
More recent works have looked at how incorporating examples in the prompt could benefit the performance of LLMs in the end task.
In spite of its underlying benefits, conventional solutions for example selection have focused on retrieving pairs using question similarity. Other approaches have sought toapproximateexpected SQL queries, and either directly use these approximations in the prompt, in a few-shot settingor to filter candidate (question, SQL)  pairs by taking into consideration the similarity of their corresponding SQL query skeletonsagainst the skeleton of the approximated SQL. We argue that such example selection strategies can result in information loss, and we propose an approach for re-ranking examples using similarity of normalised SQL ASTs.The benefits of schema selection for Text-to-SQL have been highlighted across the relevant bibliography. From the LLMs perspective, pruning schema elements from the prompts has been usually leading to performance degradation.
Inspired by, we compute a preliminary query for a givenby we adapting FastRAT, to the requirements of processing longer schemata, in a parallelised manner. We couple the resultingapproximatorwith a sparse retriever, and we propose a dynamic strategy for reducing the computational cost of the task while achieving performance improvements.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
7Related Work
2407.03227
8ConclusionIn this paper, we augment LLMs for Text-to-SQL semantic parsing by selecting suitable examples and database information. We present a novel AST-based metric to rank examples by similarity of SQL queries. Our hybrid search strategy for schema selection reuses a preliminary query to reduce irrelevant schema elements while maintaining high recall. Extensive experiments demonstrate that our AST-based ranking outperforms previous approaches of example selection and that a symbiotic combination of schema and value selection can further enhance the end-to-end performance of both closed- and open-source LLM solutions.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
8Conclusion
2407.03227
LimitationsThere are limitations with regards to both of our example selection and schema selection. Our AST-based ranking can be biased when an approximated SQL deviates significantly from structurally correct answers. To address the failure of approximators, a future direction is to sensibly diversify selected examples such that LLMs can generalise compositionally. As for schema selection, our semantic search relies on an approximator which is essentially a parser with high precision in schema linking but lack mechanisms to control recall as a standalone model. Therefore, it is worth extending cross-encoder architecture such asFastRATto support ranking schema elements while being a SQL approximator in the meantime.We demonstrate that schema splitting strategies within our framework can be applied across various numbers of splits without noticeable performance degradation. Nonetheless, given the lack of available datasets that incorporate longer commercial schemata, we focus our experiments on the cross-database setting provided by CSpiderand Spidervariants.
body
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
Limitations
2407.03227
Ethics StatementWe do not make use of any private, proprietary, or sensitive data.is trained on publicly available Text-to-SQL datasets, using publicly available encoder-models as base. Our framework for retrieval-augmented generation builds on-top of large, pre-trained language models, which may have been trained using proprietary data (e.g. in the case of the OpenAI models) . Given the nature of pre-training schemes, it is possible that our system could carry forward biases present in the datasets and/or the involved LLMs.
conclusion
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
Ethics Statement