<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2406.04604] Learning Task Decomposition to Assist Humans in Competitive Programming</title><meta property="og:description" content="When using language models (LMs) to solve complex problems, humans might struggle to understand the LM-generated solutions and repair the flawed ones.
To assist humans in repairing them, we propose to automatically dec…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Learning Task Decomposition to Assist Humans in Competitive Programming">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Learning Task Decomposition to Assist Humans in Competitive Programming">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2406.04604">

<!--Generated on Fri Jul  5 23:06:16 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document">
<h1 class="ltx_title ltx_title_document">Learning Task Decomposition to Assist Humans
<br class="ltx_break">in Competitive Programming</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">
Jiaxin Wen<sup id="id10.10.id1" class="ltx_sup"><span id="id10.10.id1.1" class="ltx_text ltx_font_italic">1,2</span></sup>,
Ruiqi Zhong<sup id="id11.11.id2" class="ltx_sup">3</sup>,
Pei Ke<sup id="id12.12.id3" class="ltx_sup"><span id="id12.12.id3.1" class="ltx_text ltx_font_italic">1,2,†</span></sup>,
Zhihong Shao<sup id="id13.13.id4" class="ltx_sup"><span id="id13.13.id4.1" class="ltx_text ltx_font_italic">1,2</span></sup>,
<br class="ltx_break"><span id="id5.5.1" class="ltx_text ltx_font_bold">Hongning Wang<sup id="id5.5.1.1" class="ltx_sup"><span id="id5.5.1.1.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2</span></sup></span>
<span id="id6.6.2" class="ltx_text ltx_font_bold">, Minlie Huang<sup id="id6.6.2.1" class="ltx_sup"><span id="id6.6.2.1.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2,†</span></sup></span> 
<br class="ltx_break"><sup id="id14.14.id5" class="ltx_sup">1</sup>The CoAI group, Tsinghua University, Beijing, China 
<br class="ltx_break"><sup id="id15.15.id6" class="ltx_sup">2</sup>Department of Computer Science and Technology, Tsinghua University, Beijing, China
<br class="ltx_break"><sup id="id16.16.id7" class="ltx_sup">3</sup> University of California, Berkeley 
<br class="ltx_break"><span id="id17.17.id8" class="ltx_text ltx_font_typewriter">wenjx22@mails.tsinghua.edu.cn, aihuang@tsinghua.edu.cn</span> 
<br class="ltx_break">
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id18.id1" class="ltx_p">When using language models (LMs) to solve complex problems, humans might struggle to understand the LM-generated solutions and repair the flawed ones.
To assist humans in repairing them, we propose to automatically decompose complex solutions into multiple simpler pieces that correspond to specific subtasks. We introduce a novel objective for learning task decomposition, termed <em id="id18.id1.1" class="ltx_emph ltx_font_italic">assistive value</em> (<span id="id18.id1.2" class="ltx_text ltx_font_typewriter">AssistV</span>), which measures the feasibility and speed for humans to repair the decomposed solution. We collect a dataset of human repair experiences on different decomposed solutions. Utilizing the collected data as in-context examples, we then learn to critique, refine, and rank decomposed solutions to improve <span id="id18.id1.3" class="ltx_text ltx_font_typewriter">AssistV</span>. We validate our method under competitive programming problems: under 177 hours of human study, our method enables non-experts to solve 33.3% more problems, speeds them up by 3.3x, and empowers them to match unassisted experts.</p>
</div>
<div id="p1" class="ltx_para">
<span id="p1.1" class="ltx_ERROR undefined">\pdfcolInitStack</span>
<p id="p1.2" class="ltx_p">tcb@breakable




































































</p>
</div>
<div id="p2" class="ltx_para ltx_noindent">
<div id="p2.9" class="ltx_block ltx_align_bottom">
<p id="p2.9.10" class="ltx_p"><span id="p2.9.10.1" class="ltx_text ltx_font_bold">Learning Task Decomposition to Assist Humans
<br class="ltx_break">in Competitive Programming</span></p>
<br class="ltx_break ltx_centering">
<p id="p2.9.9" class="ltx_p ltx_align_center" style="width:433.6pt;"><span id="p2.9.9.9" class="ltx_text ltx_inline-block" style="width:0.0pt;">
<span id="p2.9.9.9.9" class="ltx_tabular ltx_align_top">
<span id="p2.4.4.4.4.4" class="ltx_tr">
<span id="p2.4.4.4.4.4.4" class="ltx_td ltx_align_center"><span id="p2.4.4.4.4.4.4.4" class="ltx_text ltx_font_bold">
Jiaxin Wen<sup id="p2.4.4.4.4.4.4.4.1" class="ltx_sup"><span id="p2.4.4.4.4.4.4.4.1.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2</span></sup>,
Ruiqi Zhong<sup id="p2.4.4.4.4.4.4.4.2" class="ltx_sup"><span id="p2.4.4.4.4.4.4.4.2.1" class="ltx_text ltx_font_medium">3</span></sup>,
Pei Ke<sup id="p2.4.4.4.4.4.4.4.3" class="ltx_sup"><span id="p2.4.4.4.4.4.4.4.3.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2,†</span></sup>,
Zhihong Shao<sup id="p2.4.4.4.4.4.4.4.4" class="ltx_sup"><span id="p2.4.4.4.4.4.4.4.4.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2</span></sup>,</span></span></span>
<span id="p2.6.6.6.6.6" class="ltx_tr">
<span id="p2.6.6.6.6.6.2" class="ltx_td ltx_align_center"><span id="p2.5.5.5.5.5.1.1" class="ltx_text ltx_font_bold">Hongning Wang<sup id="p2.5.5.5.5.5.1.1.1" class="ltx_sup"><span id="p2.5.5.5.5.5.1.1.1.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2</span></sup></span>
<span id="p2.6.6.6.6.6.2.2" class="ltx_text ltx_font_bold">, Minlie Huang<sup id="p2.6.6.6.6.6.2.2.1" class="ltx_sup"><span id="p2.6.6.6.6.6.2.2.1.1" class="ltx_text ltx_font_medium ltx_font_italic">1,2,†</span></sup></span></span></span>
<span id="p2.7.7.7.7.7" class="ltx_tr">
<span id="p2.7.7.7.7.7.1" class="ltx_td ltx_align_center"><sup id="p2.7.7.7.7.7.1.1" class="ltx_sup">1</sup>The CoAI group, Tsinghua University, Beijing, China</span></span>
<span id="p2.8.8.8.8.8" class="ltx_tr">
<span id="p2.8.8.8.8.8.1" class="ltx_td ltx_align_center"><sup id="p2.8.8.8.8.8.1.1" class="ltx_sup">2</sup>Department of Computer Science and Technology, Tsinghua University, Beijing, China</span></span>
<span id="p2.9.9.9.9.9" class="ltx_tr">
<span id="p2.9.9.9.9.9.1" class="ltx_td ltx_align_center"><sup id="p2.9.9.9.9.9.1.1" class="ltx_sup">3</sup> University of California, Berkeley</span></span>
<span id="p2.9.9.9.9.10" class="ltx_tr">
<span id="p2.9.9.9.9.10.1" class="ltx_td ltx_align_center"><span id="p2.9.9.9.9.10.1.1" class="ltx_text ltx_font_typewriter">wenjx22@mails.tsinghua.edu.cn, aihuang@tsinghua.edu.cn</span></span></span>
</span></span></p>
<br class="ltx_break ltx_centering">
</div>
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>
<span id="footnotex1" class="ltx_note ltx_role_footnotetext"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotetext: </span><sup id="footnotex1.1" class="ltx_sup">†</sup> Corresponding author</span></span></span>
<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">With their increased capabilities, language models (LMs) are used to perform increasingly complex and high-impact problems <cite class="ltx_cite ltx_citemacro_citep">(Trinh et al., <a href="#bib.bib31" title="" class="ltx_ref">2024</a>; Li et al., <a href="#bib.bib20" title="" class="ltx_ref">2022</a>; Huang et al., <a href="#bib.bib15" title="" class="ltx_ref">2023b</a>)</cite>.
The scalable oversight challenge emerges <cite class="ltx_cite ltx_citemacro_citep">(Amodei et al., <a href="#bib.bib1" title="" class="ltx_ref">2016</a>)</cite>:
LMs might fail to provide reliable solutions for these problems, but it is also difficult for humans to evaluate and improve LMs’ solutions due to therequired significant time and expertise.</p>
</div>
<figure id="S1.F1" class="ltx_figure"><img src="/html/2406.04604/assets/x1.png" id="S1.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="172" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>Decompositions can assist humans in supervising models to solve complex problems. <span id="S1.F1.6.1" class="ltx_text ltx_font_bold">Left</span>: To solve a problem, an LM would first propose an initial solution; our goal is to decompose the initial solution into multiple simpler pieces such that humans can repair it more easily. (Sub)Task descriptions are truncated for brevity. <span id="S1.F1.7.2" class="ltx_text ltx_font_bold">Right</span>: The <em id="S1.F1.8.3" class="ltx_emph ltx_font_italic">assistive value</em> (<span id="S1.F1.9.4" class="ltx_text ltx_font_typewriter">AssistV</span>) of a decomposition measures the feasibility and speed of humans to repair the decomposed solution in the actual problem-solving process. For example, Decomposition B has a higher <span id="S1.F1.10.5" class="ltx_text ltx_font_typewriter">AssistV</span> value than A in practice, as it further decomposes the complex if-statement into two simpler subtasks, which effectively assists humans in identifying a missing condition.</figcaption>
</figure>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">One strategy to assist humans is task decomposition: as shown in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, humans can more easily understand and repair complex solutions after they are decomposed into simpler pieces that correspond to specific subtasks <cite class="ltx_cite ltx_citemacro_citep">(Lee and Anderson, <a href="#bib.bib18" title="" class="ltx_ref">2001</a>)</cite>.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.1" class="ltx_p">However, not all decompositions are helpful, and it is challenging for humans to design an effective one<cite class="ltx_cite ltx_citemacro_cite">Connolly and Dean (<a href="#bib.bib9" title="" class="ltx_ref">1997</a>); Selby (<a href="#bib.bib29" title="" class="ltx_ref">2015</a>); Correa et al. (<a href="#bib.bib10" title="" class="ltx_ref">2023</a>)</cite>.
For example, <cite class="ltx_cite ltx_citemacro_citet">Charitsis et al. (<a href="#bib.bib5" title="" class="ltx_ref">2023</a>)</cite> shows that improper decomposition designed by novice programmers can impede human debugging performance.
To decompose better, we need methods beyond using fixed heuristic rules <cite class="ltx_cite ltx_citemacro_cite">Wu et al. (<a href="#bib.bib33" title="" class="ltx_ref">2021</a>)</cite> or learning from author-crafted demonstrations <cite class="ltx_cite ltx_citemacro_cite">Yao et al. (<a href="#bib.bib35" title="" class="ltx_ref">2022</a>); Zelikman et al. (<a href="#bib.bib36" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
<div id="S1.p4" class="ltx_para">
<p id="S1.p4.3" class="ltx_p">In this paper, we introduce a novel objective for learning task decomposition: <em id="S1.p4.3.1" class="ltx_emph ltx_font_italic">assistive value</em> (<span id="S1.p4.3.2" class="ltx_text ltx_font_typewriter">AssistV</span>, Equation <a href="#S2.E1" title="In 2.1 Task Definition ‣ 2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>),
which measures the feasibility and speed of humans to repair a decomposed solution in the actual annotation process (Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> right).
To improve <span id="S1.p4.3.3" class="ltx_text ltx_font_typewriter">AssistV</span>, we first collect a dataset of decompositions, measure their <span id="S1.p4.3.4" class="ltx_text ltx_font_typewriter">AssistV</span>, and ask human annotators to provide a natural language critique on what makes a decomposition (not) helpful.
Then we design a three-stage process to generate high-<span id="S1.p4.3.5" class="ltx_text ltx_font_typewriter">AssistV</span> decompositions, where each stage is implemented by an LLM that learns from our dataset in context: 1) learn a critique model <math id="S1.p4.1.m1.1" class="ltx_Math" alttext="\pi_{\text{critique}}" display="inline"><semantics id="S1.p4.1.m1.1a"><msub id="S1.p4.1.m1.1.1" xref="S1.p4.1.m1.1.1.cmml"><mi id="S1.p4.1.m1.1.1.2" xref="S1.p4.1.m1.1.1.2.cmml">π</mi><mtext id="S1.p4.1.m1.1.1.3" xref="S1.p4.1.m1.1.1.3a.cmml">critique</mtext></msub><annotation-xml encoding="MathML-Content" id="S1.p4.1.m1.1b"><apply id="S1.p4.1.m1.1.1.cmml" xref="S1.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S1.p4.1.m1.1.1.1.cmml" xref="S1.p4.1.m1.1.1">subscript</csymbol><ci id="S1.p4.1.m1.1.1.2.cmml" xref="S1.p4.1.m1.1.1.2">𝜋</ci><ci id="S1.p4.1.m1.1.1.3a.cmml" xref="S1.p4.1.m1.1.1.3"><mtext mathsize="70%" id="S1.p4.1.m1.1.1.3.cmml" xref="S1.p4.1.m1.1.1.3">critique</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p4.1.m1.1c">\pi_{\text{critique}}</annotation></semantics></math> for predicting human critique on how to improve the initial decomposition for higher <span id="S1.p4.3.6" class="ltx_text ltx_font_typewriter">AssistV</span>, 2) learn a refine model <math id="S1.p4.2.m2.1" class="ltx_Math" alttext="\pi_{\text{refine}}" display="inline"><semantics id="S1.p4.2.m2.1a"><msub id="S1.p4.2.m2.1.1" xref="S1.p4.2.m2.1.1.cmml"><mi id="S1.p4.2.m2.1.1.2" xref="S1.p4.2.m2.1.1.2.cmml">π</mi><mtext id="S1.p4.2.m2.1.1.3" xref="S1.p4.2.m2.1.1.3a.cmml">refine</mtext></msub><annotation-xml encoding="MathML-Content" id="S1.p4.2.m2.1b"><apply id="S1.p4.2.m2.1.1.cmml" xref="S1.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S1.p4.2.m2.1.1.1.cmml" xref="S1.p4.2.m2.1.1">subscript</csymbol><ci id="S1.p4.2.m2.1.1.2.cmml" xref="S1.p4.2.m2.1.1.2">𝜋</ci><ci id="S1.p4.2.m2.1.1.3a.cmml" xref="S1.p4.2.m2.1.1.3"><mtext mathsize="70%" id="S1.p4.2.m2.1.1.3.cmml" xref="S1.p4.2.m2.1.1.3">refine</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p4.2.m2.1c">\pi_{\text{refine}}</annotation></semantics></math> to incorporate the critique to refine the initial decomposition, and 3) learn a rank model <math id="S1.p4.3.m3.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S1.p4.3.m3.1a"><msub id="S1.p4.3.m3.1.1" xref="S1.p4.3.m3.1.1.cmml"><mi id="S1.p4.3.m3.1.1.2" xref="S1.p4.3.m3.1.1.2.cmml">π</mi><mtext id="S1.p4.3.m3.1.1.3" xref="S1.p4.3.m3.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S1.p4.3.m3.1b"><apply id="S1.p4.3.m3.1.1.cmml" xref="S1.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S1.p4.3.m3.1.1.1.cmml" xref="S1.p4.3.m3.1.1">subscript</csymbol><ci id="S1.p4.3.m3.1.1.2.cmml" xref="S1.p4.3.m3.1.1.2">𝜋</ci><ci id="S1.p4.3.m3.1.1.3a.cmml" xref="S1.p4.3.m3.1.1.3"><mtext mathsize="70%" id="S1.p4.3.m3.1.1.3.cmml" xref="S1.p4.3.m3.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p4.3.m3.1c">\pi_{\text{rank}}</annotation></semantics></math> to select a decomposition with high <span id="S1.p4.3.7" class="ltx_text ltx_font_typewriter">AssistV</span>.</p>
</div>
<div id="S1.p5" class="ltx_para">
<p id="S1.p5.1" class="ltx_p">We chose competitive programming as a testbed to validate our method for scalable oversight, since it is a challenging task that both LMs and humans alone struggle to solve.
We recruit 30 Python programmers, including 11 experts <span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>This group includes medalists in National or International
Olympiad in Informatics.</span></span></span> and 19 non-experts, to repair model-generated program solutions for competitive coding problems, resulting in a total of 177 worker hours.
Experiment results show that our method enables humans to solve 33.3% more problems, speeds up non-experts and experts by 3.3 and 2.4 times, and assists non-experts to match the performance of non-assisted experts.
We then analyze LMs’ ability to select decompositions with higher <span id="S1.p5.1.1" class="ltx_text ltx_font_typewriter">AssistV</span>:
while humans’ intuitive judgment is not better than random (49.5%), GPT-3.5-Turbo achieves 62.5% accuracy by learning from human repair experiences, and GPT-4 is 15.6% better. This result indicates that LMs could learn to perform better than humans at predicting what is more helpful for humans.</p>
</div>
<div id="S1.p6" class="ltx_para">
<p id="S1.p6.1" class="ltx_p">Our core contributions are:</p>
<ul id="S1.I1" class="ltx_itemize">
<li id="S1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i1.p1" class="ltx_para">
<p id="S1.I1.i1.p1.1" class="ltx_p">We assist humans with scalable oversight via automated task decomposition.</p>
</div>
</li>
<li id="S1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i2.p1" class="ltx_para">
<p id="S1.I1.i2.p1.1" class="ltx_p">We introduce a novel objective for learning task decomposition: <span id="S1.I1.i2.p1.1.1" class="ltx_text ltx_font_typewriter">AssistV</span>, and we propose a three-stage method to produce a high-<span id="S1.I1.i2.p1.1.2" class="ltx_text ltx_font_typewriter">AssistV</span> decomposition by learning to critique, refine, and rank decompositions.</p>
</div>
</li>
<li id="S1.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i3.p1" class="ltx_para">
<p id="S1.I1.i3.p1.1" class="ltx_p">We show that our method is effective in competitive programming, improving human supervision performance and bridging the expertise gap.</p>
</div>
</li>
</ul>
</div>
<div id="S1.p7" class="ltx_para">
<p id="S1.p7.1" class="ltx_p">Overall, even when LMs cannot solve the problem themselves, they can still learn to assist humans.
By learning from human experiences to repair solutions, more capable models can predict assistive values more accurately and sometimes more accurately than humans, highlighting the potential of learning-based methods to assist humans.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Methodology</h2>

<figure id="S2.F2" class="ltx_figure"><img src="/html/2406.04604/assets/x2.png" id="S2.F2.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="152" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 2: </span>Method overview. <span id="S2.F2.8.1" class="ltx_text ltx_font_bold">Left</span>: we sample multiple decompositions from LMs and evaluate them based on assistive value <math id="S2.F2.3.m1.1" class="ltx_Math" alttext="\eta" display="inline"><semantics id="S2.F2.3.m1.1b"><mi id="S2.F2.3.m1.1.1" xref="S2.F2.3.m1.1.1.cmml">η</mi><annotation-xml encoding="MathML-Content" id="S2.F2.3.m1.1c"><ci id="S2.F2.3.m1.1.1.cmml" xref="S2.F2.3.m1.1.1">𝜂</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.F2.3.m1.1d">\eta</annotation></semantics></math> and critique <math id="S2.F2.4.m2.1" class="ltx_Math" alttext="C" display="inline"><semantics id="S2.F2.4.m2.1b"><mi id="S2.F2.4.m2.1.1" xref="S2.F2.4.m2.1.1.cmml">C</mi><annotation-xml encoding="MathML-Content" id="S2.F2.4.m2.1c"><ci id="S2.F2.4.m2.1.1.cmml" xref="S2.F2.4.m2.1.1">𝐶</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.F2.4.m2.1d">C</annotation></semantics></math>. We then construct pair-wise decompositions to demonstrate the difference between low- and high-<span id="S2.F2.9.2" class="ltx_text ltx_font_typewriter">AssistV</span> decompositions. <span id="S2.F2.10.3" class="ltx_text ltx_font_bold">Right</span>: Starting from a vanilla decomposition generated by naively prompting LMs, we use the collected pair-wise data as in-context demonstrations to learn three models to critique, refine, and rank decompositions to better assist humans.</figcaption>
</figure>
<section id="S2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.1 </span>Task Definition</h3>

<div id="S2.SS1.p1" class="ltx_para">
<p id="S2.SS1.p1.1" class="ltx_p">Our objective is to assist human labelers in repairing model-generated solutions to complex tasks by learning a decomposition model. This model decomposes input solutions into multiple easier-to-repair pieces corresponding to specific subtasks.</p>
</div>
<div id="S2.SS1.p2" class="ltx_para">
<p id="S2.SS1.p2.5" class="ltx_p">Formally, given a problem <math id="S2.SS1.p2.1.m1.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S2.SS1.p2.1.m1.1a"><mi id="S2.SS1.p2.1.m1.1.1" xref="S2.SS1.p2.1.m1.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.1.m1.1b"><ci id="S2.SS1.p2.1.m1.1.1.cmml" xref="S2.SS1.p2.1.m1.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.1.m1.1c">P</annotation></semantics></math> and an initial model solution <math id="S2.SS1.p2.2.m2.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S2.SS1.p2.2.m2.1a"><mi id="S2.SS1.p2.2.m2.1.1" xref="S2.SS1.p2.2.m2.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.2.m2.1b"><ci id="S2.SS1.p2.2.m2.1.1.cmml" xref="S2.SS1.p2.2.m2.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.2.m2.1c">A</annotation></semantics></math>, we aim to transform <math id="S2.SS1.p2.3.m3.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S2.SS1.p2.3.m3.1a"><mi id="S2.SS1.p2.3.m3.1.1" xref="S2.SS1.p2.3.m3.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.3.m3.1b"><ci id="S2.SS1.p2.3.m3.1.1.cmml" xref="S2.SS1.p2.3.m3.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.3.m3.1c">A</annotation></semantics></math> into a decomposed solution <math id="S2.SS1.p2.4.m4.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.SS1.p2.4.m4.1a"><msub id="S2.SS1.p2.4.m4.1.1" xref="S2.SS1.p2.4.m4.1.1.cmml"><mi id="S2.SS1.p2.4.m4.1.1.2" xref="S2.SS1.p2.4.m4.1.1.2.cmml">A</mi><mi id="S2.SS1.p2.4.m4.1.1.3" xref="S2.SS1.p2.4.m4.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.4.m4.1b"><apply id="S2.SS1.p2.4.m4.1.1.cmml" xref="S2.SS1.p2.4.m4.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.4.m4.1.1.1.cmml" xref="S2.SS1.p2.4.m4.1.1">subscript</csymbol><ci id="S2.SS1.p2.4.m4.1.1.2.cmml" xref="S2.SS1.p2.4.m4.1.1.2">𝐴</ci><ci id="S2.SS1.p2.4.m4.1.1.3.cmml" xref="S2.SS1.p2.4.m4.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.4.m4.1c">A_{d}</annotation></semantics></math> to improve its assistive value <math id="S2.SS1.p2.5.m5.1" class="ltx_Math" alttext="\eta" display="inline"><semantics id="S2.SS1.p2.5.m5.1a"><mi id="S2.SS1.p2.5.m5.1.1" xref="S2.SS1.p2.5.m5.1.1.cmml">η</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.5.m5.1b"><ci id="S2.SS1.p2.5.m5.1.1.cmml" xref="S2.SS1.p2.5.m5.1.1">𝜂</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.5.m5.1c">\eta</annotation></semantics></math>, as defined by:</p>
<table id="S2.E1" class="ltx_equationgroup ltx_eqn_table">
<tbody>
<tr id="S2.E1X" class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S2.E1X.2.1.1.m1.2" class="ltx_Math" alttext="\displaystyle\eta(A_{d})=\int_{0}^{T}\texttt{eval}(A_{d}^{t})\mathrm{d}t" display="inline"><semantics id="S2.E1X.2.1.1.m1.2a"><mrow id="S2.E1X.2.1.1.m1.2.2" xref="S2.E1X.2.1.1.m1.2.2.cmml"><mrow id="S2.E1X.2.1.1.m1.1.1.1" xref="S2.E1X.2.1.1.m1.1.1.1.cmml"><mi id="S2.E1X.2.1.1.m1.1.1.1.3" xref="S2.E1X.2.1.1.m1.1.1.1.3.cmml">η</mi><mo lspace="0em" rspace="0em" id="S2.E1X.2.1.1.m1.1.1.1.2" xref="S2.E1X.2.1.1.m1.1.1.1.2.cmml">​</mo><mrow id="S2.E1X.2.1.1.m1.1.1.1.1.1" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.E1X.2.1.1.m1.1.1.1.1.1.2" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.cmml">(</mo><msub id="S2.E1X.2.1.1.m1.1.1.1.1.1.1" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.cmml"><mi id="S2.E1X.2.1.1.m1.1.1.1.1.1.1.2" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.2.cmml">A</mi><mi id="S2.E1X.2.1.1.m1.1.1.1.1.1.1.3" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.3.cmml">d</mi></msub><mo stretchy="false" id="S2.E1X.2.1.1.m1.1.1.1.1.1.3" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S2.E1X.2.1.1.m1.2.2.3" xref="S2.E1X.2.1.1.m1.2.2.3.cmml">=</mo><mrow id="S2.E1X.2.1.1.m1.2.2.2" xref="S2.E1X.2.1.1.m1.2.2.2.cmml"><mstyle displaystyle="true" id="S2.E1X.2.1.1.m1.2.2.2.2" xref="S2.E1X.2.1.1.m1.2.2.2.2.cmml"><msubsup id="S2.E1X.2.1.1.m1.2.2.2.2a" xref="S2.E1X.2.1.1.m1.2.2.2.2.cmml"><mo id="S2.E1X.2.1.1.m1.2.2.2.2.2.2" xref="S2.E1X.2.1.1.m1.2.2.2.2.2.2.cmml">∫</mo><mn id="S2.E1X.2.1.1.m1.2.2.2.2.2.3" xref="S2.E1X.2.1.1.m1.2.2.2.2.2.3.cmml">0</mn><mi id="S2.E1X.2.1.1.m1.2.2.2.2.3" xref="S2.E1X.2.1.1.m1.2.2.2.2.3.cmml">T</mi></msubsup></mstyle><mrow id="S2.E1X.2.1.1.m1.2.2.2.1" xref="S2.E1X.2.1.1.m1.2.2.2.1.cmml"><mtext class="ltx_mathvariant_monospace" id="S2.E1X.2.1.1.m1.2.2.2.1.3" xref="S2.E1X.2.1.1.m1.2.2.2.1.3a.cmml">eval</mtext><mo lspace="0em" rspace="0em" id="S2.E1X.2.1.1.m1.2.2.2.1.2" xref="S2.E1X.2.1.1.m1.2.2.2.1.2.cmml">​</mo><mrow id="S2.E1X.2.1.1.m1.2.2.2.1.1.1" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.cmml"><mo stretchy="false" id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.2" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.cmml">(</mo><msubsup id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.cmml"><mi id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.2" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.2.cmml">A</mi><mi id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.3" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.3.cmml">d</mi><mi id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.3" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.3.cmml">t</mi></msubsup><mo stretchy="false" id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.3" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.cmml">)</mo></mrow><mo lspace="0em" rspace="0em" id="S2.E1X.2.1.1.m1.2.2.2.1.2a" xref="S2.E1X.2.1.1.m1.2.2.2.1.2.cmml">​</mo><mrow id="S2.E1X.2.1.1.m1.2.2.2.1.4" xref="S2.E1X.2.1.1.m1.2.2.2.1.4.cmml"><mo rspace="0em" id="S2.E1X.2.1.1.m1.2.2.2.1.4.1" xref="S2.E1X.2.1.1.m1.2.2.2.1.4.1.cmml">d</mo><mi id="S2.E1X.2.1.1.m1.2.2.2.1.4.2" xref="S2.E1X.2.1.1.m1.2.2.2.1.4.2.cmml">t</mi></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.E1X.2.1.1.m1.2b"><apply id="S2.E1X.2.1.1.m1.2.2.cmml" xref="S2.E1X.2.1.1.m1.2.2"><eq id="S2.E1X.2.1.1.m1.2.2.3.cmml" xref="S2.E1X.2.1.1.m1.2.2.3"></eq><apply id="S2.E1X.2.1.1.m1.1.1.1.cmml" xref="S2.E1X.2.1.1.m1.1.1.1"><times id="S2.E1X.2.1.1.m1.1.1.1.2.cmml" xref="S2.E1X.2.1.1.m1.1.1.1.2"></times><ci id="S2.E1X.2.1.1.m1.1.1.1.3.cmml" xref="S2.E1X.2.1.1.m1.1.1.1.3">𝜂</ci><apply id="S2.E1X.2.1.1.m1.1.1.1.1.1.1.cmml" xref="S2.E1X.2.1.1.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.E1X.2.1.1.m1.1.1.1.1.1.1.1.cmml" xref="S2.E1X.2.1.1.m1.1.1.1.1.1">subscript</csymbol><ci id="S2.E1X.2.1.1.m1.1.1.1.1.1.1.2.cmml" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.2">𝐴</ci><ci id="S2.E1X.2.1.1.m1.1.1.1.1.1.1.3.cmml" xref="S2.E1X.2.1.1.m1.1.1.1.1.1.1.3">𝑑</ci></apply></apply><apply id="S2.E1X.2.1.1.m1.2.2.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2"><apply id="S2.E1X.2.1.1.m1.2.2.2.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2"><csymbol cd="ambiguous" id="S2.E1X.2.1.1.m1.2.2.2.2.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2">superscript</csymbol><apply id="S2.E1X.2.1.1.m1.2.2.2.2.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2"><csymbol cd="ambiguous" id="S2.E1X.2.1.1.m1.2.2.2.2.2.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2">subscript</csymbol><int id="S2.E1X.2.1.1.m1.2.2.2.2.2.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2.2.2"></int><cn type="integer" id="S2.E1X.2.1.1.m1.2.2.2.2.2.3.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2.2.3">0</cn></apply><ci id="S2.E1X.2.1.1.m1.2.2.2.2.3.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.2.3">𝑇</ci></apply><apply id="S2.E1X.2.1.1.m1.2.2.2.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1"><times id="S2.E1X.2.1.1.m1.2.2.2.1.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.2"></times><ci id="S2.E1X.2.1.1.m1.2.2.2.1.3a.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.3"><mtext class="ltx_mathvariant_monospace" id="S2.E1X.2.1.1.m1.2.2.2.1.3.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.3">eval</mtext></ci><apply id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1">superscript</csymbol><apply id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1">subscript</csymbol><ci id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.2">𝐴</ci><ci id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.3.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.2.3">𝑑</ci></apply><ci id="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.3.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.1.1.1.3">𝑡</ci></apply><apply id="S2.E1X.2.1.1.m1.2.2.2.1.4.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.4"><csymbol cd="latexml" id="S2.E1X.2.1.1.m1.2.2.2.1.4.1.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.4.1">differential-d</csymbol><ci id="S2.E1X.2.1.1.m1.2.2.2.1.4.2.cmml" xref="S2.E1X.2.1.1.m1.2.2.2.1.4.2">𝑡</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E1X.2.1.1.m1.2c">\displaystyle\eta(A_{d})=\int_{0}^{T}\texttt{eval}(A_{d}^{t})\mathrm{d}t</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equationgroup ltx_align_right">(1)</span></td>
</tr>
</tbody>
</table>
<p id="S2.SS1.p2.15" class="ltx_p">where <math id="S2.SS1.p2.6.m1.1" class="ltx_Math" alttext="A_{d}^{t}" display="inline"><semantics id="S2.SS1.p2.6.m1.1a"><msubsup id="S2.SS1.p2.6.m1.1.1" xref="S2.SS1.p2.6.m1.1.1.cmml"><mi id="S2.SS1.p2.6.m1.1.1.2.2" xref="S2.SS1.p2.6.m1.1.1.2.2.cmml">A</mi><mi id="S2.SS1.p2.6.m1.1.1.2.3" xref="S2.SS1.p2.6.m1.1.1.2.3.cmml">d</mi><mi id="S2.SS1.p2.6.m1.1.1.3" xref="S2.SS1.p2.6.m1.1.1.3.cmml">t</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.6.m1.1b"><apply id="S2.SS1.p2.6.m1.1.1.cmml" xref="S2.SS1.p2.6.m1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.6.m1.1.1.1.cmml" xref="S2.SS1.p2.6.m1.1.1">superscript</csymbol><apply id="S2.SS1.p2.6.m1.1.1.2.cmml" xref="S2.SS1.p2.6.m1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.6.m1.1.1.2.1.cmml" xref="S2.SS1.p2.6.m1.1.1">subscript</csymbol><ci id="S2.SS1.p2.6.m1.1.1.2.2.cmml" xref="S2.SS1.p2.6.m1.1.1.2.2">𝐴</ci><ci id="S2.SS1.p2.6.m1.1.1.2.3.cmml" xref="S2.SS1.p2.6.m1.1.1.2.3">𝑑</ci></apply><ci id="S2.SS1.p2.6.m1.1.1.3.cmml" xref="S2.SS1.p2.6.m1.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.6.m1.1c">A_{d}^{t}</annotation></semantics></math> denotes the solution repaired by humans after spending time <math id="S2.SS1.p2.7.m2.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS1.p2.7.m2.1a"><mi id="S2.SS1.p2.7.m2.1.1" xref="S2.SS1.p2.7.m2.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.7.m2.1b"><ci id="S2.SS1.p2.7.m2.1.1.cmml" xref="S2.SS1.p2.7.m2.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.7.m2.1c">t</annotation></semantics></math> from its initial solution <math id="S2.SS1.p2.8.m3.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.SS1.p2.8.m3.1a"><msub id="S2.SS1.p2.8.m3.1.1" xref="S2.SS1.p2.8.m3.1.1.cmml"><mi id="S2.SS1.p2.8.m3.1.1.2" xref="S2.SS1.p2.8.m3.1.1.2.cmml">A</mi><mi id="S2.SS1.p2.8.m3.1.1.3" xref="S2.SS1.p2.8.m3.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.8.m3.1b"><apply id="S2.SS1.p2.8.m3.1.1.cmml" xref="S2.SS1.p2.8.m3.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.8.m3.1.1.1.cmml" xref="S2.SS1.p2.8.m3.1.1">subscript</csymbol><ci id="S2.SS1.p2.8.m3.1.1.2.cmml" xref="S2.SS1.p2.8.m3.1.1.2">𝐴</ci><ci id="S2.SS1.p2.8.m3.1.1.3.cmml" xref="S2.SS1.p2.8.m3.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.8.m3.1c">A_{d}</annotation></semantics></math>, and <math id="S2.SS1.p2.9.m4.1" class="ltx_Math" alttext="\texttt{eval}(\cdot)" display="inline"><semantics id="S2.SS1.p2.9.m4.1a"><mrow id="S2.SS1.p2.9.m4.1.2" xref="S2.SS1.p2.9.m4.1.2.cmml"><mtext class="ltx_mathvariant_monospace" id="S2.SS1.p2.9.m4.1.2.2" xref="S2.SS1.p2.9.m4.1.2.2a.cmml">eval</mtext><mo lspace="0em" rspace="0em" id="S2.SS1.p2.9.m4.1.2.1" xref="S2.SS1.p2.9.m4.1.2.1.cmml">​</mo><mrow id="S2.SS1.p2.9.m4.1.2.3.2" xref="S2.SS1.p2.9.m4.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.9.m4.1.2.3.2.1" xref="S2.SS1.p2.9.m4.1.2.cmml">(</mo><mo lspace="0em" rspace="0em" id="S2.SS1.p2.9.m4.1.1" xref="S2.SS1.p2.9.m4.1.1.cmml">⋅</mo><mo stretchy="false" id="S2.SS1.p2.9.m4.1.2.3.2.2" xref="S2.SS1.p2.9.m4.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.9.m4.1b"><apply id="S2.SS1.p2.9.m4.1.2.cmml" xref="S2.SS1.p2.9.m4.1.2"><times id="S2.SS1.p2.9.m4.1.2.1.cmml" xref="S2.SS1.p2.9.m4.1.2.1"></times><ci id="S2.SS1.p2.9.m4.1.2.2a.cmml" xref="S2.SS1.p2.9.m4.1.2.2"><mtext class="ltx_mathvariant_monospace" id="S2.SS1.p2.9.m4.1.2.2.cmml" xref="S2.SS1.p2.9.m4.1.2.2">eval</mtext></ci><ci id="S2.SS1.p2.9.m4.1.1.cmml" xref="S2.SS1.p2.9.m4.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.9.m4.1c">\texttt{eval}(\cdot)</annotation></semantics></math> is a metric of solution quality. In competitive programming, we set <math id="S2.SS1.p2.10.m5.1" class="ltx_Math" alttext="\texttt{eval}(\cdot)" display="inline"><semantics id="S2.SS1.p2.10.m5.1a"><mrow id="S2.SS1.p2.10.m5.1.2" xref="S2.SS1.p2.10.m5.1.2.cmml"><mtext class="ltx_mathvariant_monospace" id="S2.SS1.p2.10.m5.1.2.2" xref="S2.SS1.p2.10.m5.1.2.2a.cmml">eval</mtext><mo lspace="0em" rspace="0em" id="S2.SS1.p2.10.m5.1.2.1" xref="S2.SS1.p2.10.m5.1.2.1.cmml">​</mo><mrow id="S2.SS1.p2.10.m5.1.2.3.2" xref="S2.SS1.p2.10.m5.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.10.m5.1.2.3.2.1" xref="S2.SS1.p2.10.m5.1.2.cmml">(</mo><mo lspace="0em" rspace="0em" id="S2.SS1.p2.10.m5.1.1" xref="S2.SS1.p2.10.m5.1.1.cmml">⋅</mo><mo stretchy="false" id="S2.SS1.p2.10.m5.1.2.3.2.2" xref="S2.SS1.p2.10.m5.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.10.m5.1b"><apply id="S2.SS1.p2.10.m5.1.2.cmml" xref="S2.SS1.p2.10.m5.1.2"><times id="S2.SS1.p2.10.m5.1.2.1.cmml" xref="S2.SS1.p2.10.m5.1.2.1"></times><ci id="S2.SS1.p2.10.m5.1.2.2a.cmml" xref="S2.SS1.p2.10.m5.1.2.2"><mtext class="ltx_mathvariant_monospace" id="S2.SS1.p2.10.m5.1.2.2.cmml" xref="S2.SS1.p2.10.m5.1.2.2">eval</mtext></ci><ci id="S2.SS1.p2.10.m5.1.1.cmml" xref="S2.SS1.p2.10.m5.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.10.m5.1c">\texttt{eval}(\cdot)</annotation></semantics></math> as the passing rates on unit test cases. Assistive value <math id="S2.SS1.p2.11.m6.1" class="ltx_Math" alttext="\eta(A_{d})" display="inline"><semantics id="S2.SS1.p2.11.m6.1a"><mrow id="S2.SS1.p2.11.m6.1.1" xref="S2.SS1.p2.11.m6.1.1.cmml"><mi id="S2.SS1.p2.11.m6.1.1.3" xref="S2.SS1.p2.11.m6.1.1.3.cmml">η</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.11.m6.1.1.2" xref="S2.SS1.p2.11.m6.1.1.2.cmml">​</mo><mrow id="S2.SS1.p2.11.m6.1.1.1.1" xref="S2.SS1.p2.11.m6.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.p2.11.m6.1.1.1.1.2" xref="S2.SS1.p2.11.m6.1.1.1.1.1.cmml">(</mo><msub id="S2.SS1.p2.11.m6.1.1.1.1.1" xref="S2.SS1.p2.11.m6.1.1.1.1.1.cmml"><mi id="S2.SS1.p2.11.m6.1.1.1.1.1.2" xref="S2.SS1.p2.11.m6.1.1.1.1.1.2.cmml">A</mi><mi id="S2.SS1.p2.11.m6.1.1.1.1.1.3" xref="S2.SS1.p2.11.m6.1.1.1.1.1.3.cmml">d</mi></msub><mo stretchy="false" id="S2.SS1.p2.11.m6.1.1.1.1.3" xref="S2.SS1.p2.11.m6.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.11.m6.1b"><apply id="S2.SS1.p2.11.m6.1.1.cmml" xref="S2.SS1.p2.11.m6.1.1"><times id="S2.SS1.p2.11.m6.1.1.2.cmml" xref="S2.SS1.p2.11.m6.1.1.2"></times><ci id="S2.SS1.p2.11.m6.1.1.3.cmml" xref="S2.SS1.p2.11.m6.1.1.3">𝜂</ci><apply id="S2.SS1.p2.11.m6.1.1.1.1.1.cmml" xref="S2.SS1.p2.11.m6.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.11.m6.1.1.1.1.1.1.cmml" xref="S2.SS1.p2.11.m6.1.1.1.1">subscript</csymbol><ci id="S2.SS1.p2.11.m6.1.1.1.1.1.2.cmml" xref="S2.SS1.p2.11.m6.1.1.1.1.1.2">𝐴</ci><ci id="S2.SS1.p2.11.m6.1.1.1.1.1.3.cmml" xref="S2.SS1.p2.11.m6.1.1.1.1.1.3">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.11.m6.1c">\eta(A_{d})</annotation></semantics></math> summarizes the solution quality over the human repairing process starting from <math id="S2.SS1.p2.12.m7.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.SS1.p2.12.m7.1a"><msub id="S2.SS1.p2.12.m7.1.1" xref="S2.SS1.p2.12.m7.1.1.cmml"><mi id="S2.SS1.p2.12.m7.1.1.2" xref="S2.SS1.p2.12.m7.1.1.2.cmml">A</mi><mi id="S2.SS1.p2.12.m7.1.1.3" xref="S2.SS1.p2.12.m7.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.12.m7.1b"><apply id="S2.SS1.p2.12.m7.1.1.cmml" xref="S2.SS1.p2.12.m7.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.12.m7.1.1.1.cmml" xref="S2.SS1.p2.12.m7.1.1">subscript</csymbol><ci id="S2.SS1.p2.12.m7.1.1.2.cmml" xref="S2.SS1.p2.12.m7.1.1.2">𝐴</ci><ci id="S2.SS1.p2.12.m7.1.1.3.cmml" xref="S2.SS1.p2.12.m7.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.12.m7.1c">A_{d}</annotation></semantics></math> <cite class="ltx_cite ltx_citemacro_cite">Bradley (<a href="#bib.bib4" title="" class="ltx_ref">1997</a>)</cite>, with example trajectories shown in Figure <a href="#S3.F4" title="Figure 4 ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>. A higher value of <math id="S2.SS1.p2.13.m8.1" class="ltx_Math" alttext="\eta(A_{d})" display="inline"><semantics id="S2.SS1.p2.13.m8.1a"><mrow id="S2.SS1.p2.13.m8.1.1" xref="S2.SS1.p2.13.m8.1.1.cmml"><mi id="S2.SS1.p2.13.m8.1.1.3" xref="S2.SS1.p2.13.m8.1.1.3.cmml">η</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.13.m8.1.1.2" xref="S2.SS1.p2.13.m8.1.1.2.cmml">​</mo><mrow id="S2.SS1.p2.13.m8.1.1.1.1" xref="S2.SS1.p2.13.m8.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.p2.13.m8.1.1.1.1.2" xref="S2.SS1.p2.13.m8.1.1.1.1.1.cmml">(</mo><msub id="S2.SS1.p2.13.m8.1.1.1.1.1" xref="S2.SS1.p2.13.m8.1.1.1.1.1.cmml"><mi id="S2.SS1.p2.13.m8.1.1.1.1.1.2" xref="S2.SS1.p2.13.m8.1.1.1.1.1.2.cmml">A</mi><mi id="S2.SS1.p2.13.m8.1.1.1.1.1.3" xref="S2.SS1.p2.13.m8.1.1.1.1.1.3.cmml">d</mi></msub><mo stretchy="false" id="S2.SS1.p2.13.m8.1.1.1.1.3" xref="S2.SS1.p2.13.m8.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.13.m8.1b"><apply id="S2.SS1.p2.13.m8.1.1.cmml" xref="S2.SS1.p2.13.m8.1.1"><times id="S2.SS1.p2.13.m8.1.1.2.cmml" xref="S2.SS1.p2.13.m8.1.1.2"></times><ci id="S2.SS1.p2.13.m8.1.1.3.cmml" xref="S2.SS1.p2.13.m8.1.1.3">𝜂</ci><apply id="S2.SS1.p2.13.m8.1.1.1.1.1.cmml" xref="S2.SS1.p2.13.m8.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.13.m8.1.1.1.1.1.1.cmml" xref="S2.SS1.p2.13.m8.1.1.1.1">subscript</csymbol><ci id="S2.SS1.p2.13.m8.1.1.1.1.1.2.cmml" xref="S2.SS1.p2.13.m8.1.1.1.1.1.2">𝐴</ci><ci id="S2.SS1.p2.13.m8.1.1.1.1.1.3.cmml" xref="S2.SS1.p2.13.m8.1.1.1.1.1.3">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.13.m8.1c">\eta(A_{d})</annotation></semantics></math> indicates that assisted humans can more easily repair <math id="S2.SS1.p2.14.m9.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.SS1.p2.14.m9.1a"><msub id="S2.SS1.p2.14.m9.1.1" xref="S2.SS1.p2.14.m9.1.1.cmml"><mi id="S2.SS1.p2.14.m9.1.1.2" xref="S2.SS1.p2.14.m9.1.1.2.cmml">A</mi><mi id="S2.SS1.p2.14.m9.1.1.3" xref="S2.SS1.p2.14.m9.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.14.m9.1b"><apply id="S2.SS1.p2.14.m9.1.1.cmml" xref="S2.SS1.p2.14.m9.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.14.m9.1.1.1.cmml" xref="S2.SS1.p2.14.m9.1.1">subscript</csymbol><ci id="S2.SS1.p2.14.m9.1.1.2.cmml" xref="S2.SS1.p2.14.m9.1.1.2">𝐴</ci><ci id="S2.SS1.p2.14.m9.1.1.3.cmml" xref="S2.SS1.p2.14.m9.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.14.m9.1c">A_{d}</annotation></semantics></math>, thus efficiently providing a high-quality label to the problem <math id="S2.SS1.p2.15.m10.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S2.SS1.p2.15.m10.1a"><mi id="S2.SS1.p2.15.m10.1.1" xref="S2.SS1.p2.15.m10.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.15.m10.1b"><ci id="S2.SS1.p2.15.m10.1.1.cmml" xref="S2.SS1.p2.15.m10.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.15.m10.1c">P</annotation></semantics></math>.</p>
</div>
<div id="S2.SS1.p3" class="ltx_para">
<p id="S2.SS1.p3.1" class="ltx_p">While there are diverse ways to decompose one single solution as shown in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, effective decomposition that improves human problem-solving performance is non-trivial and can even be challenging for humans to devise <cite class="ltx_cite ltx_citemacro_cite">Charitsis et al. (<a href="#bib.bib5" title="" class="ltx_ref">2023</a>)</cite>. To decompose better, we need methods beyond prior methods that rely on fixed heuristic rules or learn from author-crafted demonstrations. In this paper, we propose to improve the assistive value of task decomposition by learning from human problem-solving experiences.</p>
</div>
<div id="S2.SS1.p4" class="ltx_para">
<p id="S2.SS1.p4.5" class="ltx_p">Specifically, we first construct a training set of human repair experiences between different decomposed solutions <math id="S2.SS1.p4.1.m1.3" class="ltx_Math" alttext="D_{train}=\{(\hat{A}_{d},A_{d},C,P)\}" display="inline"><semantics id="S2.SS1.p4.1.m1.3a"><mrow id="S2.SS1.p4.1.m1.3.3" xref="S2.SS1.p4.1.m1.3.3.cmml"><msub id="S2.SS1.p4.1.m1.3.3.3" xref="S2.SS1.p4.1.m1.3.3.3.cmml"><mi id="S2.SS1.p4.1.m1.3.3.3.2" xref="S2.SS1.p4.1.m1.3.3.3.2.cmml">D</mi><mrow id="S2.SS1.p4.1.m1.3.3.3.3" xref="S2.SS1.p4.1.m1.3.3.3.3.cmml"><mi id="S2.SS1.p4.1.m1.3.3.3.3.2" xref="S2.SS1.p4.1.m1.3.3.3.3.2.cmml">t</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p4.1.m1.3.3.3.3.1" xref="S2.SS1.p4.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS1.p4.1.m1.3.3.3.3.3" xref="S2.SS1.p4.1.m1.3.3.3.3.3.cmml">r</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p4.1.m1.3.3.3.3.1a" xref="S2.SS1.p4.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS1.p4.1.m1.3.3.3.3.4" xref="S2.SS1.p4.1.m1.3.3.3.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p4.1.m1.3.3.3.3.1b" xref="S2.SS1.p4.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS1.p4.1.m1.3.3.3.3.5" xref="S2.SS1.p4.1.m1.3.3.3.3.5.cmml">i</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p4.1.m1.3.3.3.3.1c" xref="S2.SS1.p4.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS1.p4.1.m1.3.3.3.3.6" xref="S2.SS1.p4.1.m1.3.3.3.3.6.cmml">n</mi></mrow></msub><mo id="S2.SS1.p4.1.m1.3.3.2" xref="S2.SS1.p4.1.m1.3.3.2.cmml">=</mo><mrow id="S2.SS1.p4.1.m1.3.3.1.1" xref="S2.SS1.p4.1.m1.3.3.1.2.cmml"><mo stretchy="false" id="S2.SS1.p4.1.m1.3.3.1.1.2" xref="S2.SS1.p4.1.m1.3.3.1.2.cmml">{</mo><mrow id="S2.SS1.p4.1.m1.3.3.1.1.1.2" xref="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml"><mo stretchy="false" id="S2.SS1.p4.1.m1.3.3.1.1.1.2.3" xref="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml">(</mo><msub id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.cmml"><mover accent="true" id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.cmml"><mi id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.2" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.2.cmml">A</mi><mo id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.1" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.3" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.3.cmml">d</mi></msub><mo id="S2.SS1.p4.1.m1.3.3.1.1.1.2.4" xref="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml">,</mo><msub id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.cmml"><mi id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.2" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.2.cmml">A</mi><mi id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.3" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.3.cmml">d</mi></msub><mo id="S2.SS1.p4.1.m1.3.3.1.1.1.2.5" xref="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml">,</mo><mi id="S2.SS1.p4.1.m1.1.1" xref="S2.SS1.p4.1.m1.1.1.cmml">C</mi><mo id="S2.SS1.p4.1.m1.3.3.1.1.1.2.6" xref="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml">,</mo><mi id="S2.SS1.p4.1.m1.2.2" xref="S2.SS1.p4.1.m1.2.2.cmml">P</mi><mo stretchy="false" id="S2.SS1.p4.1.m1.3.3.1.1.1.2.7" xref="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml">)</mo></mrow><mo stretchy="false" id="S2.SS1.p4.1.m1.3.3.1.1.3" xref="S2.SS1.p4.1.m1.3.3.1.2.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p4.1.m1.3b"><apply id="S2.SS1.p4.1.m1.3.3.cmml" xref="S2.SS1.p4.1.m1.3.3"><eq id="S2.SS1.p4.1.m1.3.3.2.cmml" xref="S2.SS1.p4.1.m1.3.3.2"></eq><apply id="S2.SS1.p4.1.m1.3.3.3.cmml" xref="S2.SS1.p4.1.m1.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.p4.1.m1.3.3.3.1.cmml" xref="S2.SS1.p4.1.m1.3.3.3">subscript</csymbol><ci id="S2.SS1.p4.1.m1.3.3.3.2.cmml" xref="S2.SS1.p4.1.m1.3.3.3.2">𝐷</ci><apply id="S2.SS1.p4.1.m1.3.3.3.3.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3"><times id="S2.SS1.p4.1.m1.3.3.3.3.1.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3.1"></times><ci id="S2.SS1.p4.1.m1.3.3.3.3.2.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3.2">𝑡</ci><ci id="S2.SS1.p4.1.m1.3.3.3.3.3.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3.3">𝑟</ci><ci id="S2.SS1.p4.1.m1.3.3.3.3.4.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3.4">𝑎</ci><ci id="S2.SS1.p4.1.m1.3.3.3.3.5.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3.5">𝑖</ci><ci id="S2.SS1.p4.1.m1.3.3.3.3.6.cmml" xref="S2.SS1.p4.1.m1.3.3.3.3.6">𝑛</ci></apply></apply><set id="S2.SS1.p4.1.m1.3.3.1.2.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1"><vector id="S2.SS1.p4.1.m1.3.3.1.1.1.3.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2"><apply id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.1.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1">subscript</csymbol><apply id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2"><ci id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.1.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.1">^</ci><ci id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.2.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.2.2">𝐴</ci></apply><ci id="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.3.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.1.1.3">𝑑</ci></apply><apply id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2"><csymbol cd="ambiguous" id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.1.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2">subscript</csymbol><ci id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.2.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.2">𝐴</ci><ci id="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.3.cmml" xref="S2.SS1.p4.1.m1.3.3.1.1.1.2.2.3">𝑑</ci></apply><ci id="S2.SS1.p4.1.m1.1.1.cmml" xref="S2.SS1.p4.1.m1.1.1">𝐶</ci><ci id="S2.SS1.p4.1.m1.2.2.cmml" xref="S2.SS1.p4.1.m1.2.2">𝑃</ci></vector></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p4.1.m1.3c">D_{train}=\{(\hat{A}_{d},A_{d},C,P)\}</annotation></semantics></math>, where <math id="S2.SS1.p4.2.m2.2" class="ltx_Math" alttext="\eta(\hat{A}_{d})&gt;\eta(A_{d})" display="inline"><semantics id="S2.SS1.p4.2.m2.2a"><mrow id="S2.SS1.p4.2.m2.2.2" xref="S2.SS1.p4.2.m2.2.2.cmml"><mrow id="S2.SS1.p4.2.m2.1.1.1" xref="S2.SS1.p4.2.m2.1.1.1.cmml"><mi id="S2.SS1.p4.2.m2.1.1.1.3" xref="S2.SS1.p4.2.m2.1.1.1.3.cmml">η</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p4.2.m2.1.1.1.2" xref="S2.SS1.p4.2.m2.1.1.1.2.cmml">​</mo><mrow id="S2.SS1.p4.2.m2.1.1.1.1.1" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.p4.2.m2.1.1.1.1.1.2" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.cmml">(</mo><msub id="S2.SS1.p4.2.m2.1.1.1.1.1.1" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.cmml"><mover accent="true" id="S2.SS1.p4.2.m2.1.1.1.1.1.1.2" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.cmml"><mi id="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.2" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.2.cmml">A</mi><mo id="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.1" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS1.p4.2.m2.1.1.1.1.1.1.3" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.3.cmml">d</mi></msub><mo stretchy="false" id="S2.SS1.p4.2.m2.1.1.1.1.1.3" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S2.SS1.p4.2.m2.2.2.3" xref="S2.SS1.p4.2.m2.2.2.3.cmml">&gt;</mo><mrow id="S2.SS1.p4.2.m2.2.2.2" xref="S2.SS1.p4.2.m2.2.2.2.cmml"><mi id="S2.SS1.p4.2.m2.2.2.2.3" xref="S2.SS1.p4.2.m2.2.2.2.3.cmml">η</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p4.2.m2.2.2.2.2" xref="S2.SS1.p4.2.m2.2.2.2.2.cmml">​</mo><mrow id="S2.SS1.p4.2.m2.2.2.2.1.1" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.p4.2.m2.2.2.2.1.1.2" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.cmml">(</mo><msub id="S2.SS1.p4.2.m2.2.2.2.1.1.1" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.cmml"><mi id="S2.SS1.p4.2.m2.2.2.2.1.1.1.2" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.2.cmml">A</mi><mi id="S2.SS1.p4.2.m2.2.2.2.1.1.1.3" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.3.cmml">d</mi></msub><mo stretchy="false" id="S2.SS1.p4.2.m2.2.2.2.1.1.3" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p4.2.m2.2b"><apply id="S2.SS1.p4.2.m2.2.2.cmml" xref="S2.SS1.p4.2.m2.2.2"><gt id="S2.SS1.p4.2.m2.2.2.3.cmml" xref="S2.SS1.p4.2.m2.2.2.3"></gt><apply id="S2.SS1.p4.2.m2.1.1.1.cmml" xref="S2.SS1.p4.2.m2.1.1.1"><times id="S2.SS1.p4.2.m2.1.1.1.2.cmml" xref="S2.SS1.p4.2.m2.1.1.1.2"></times><ci id="S2.SS1.p4.2.m2.1.1.1.3.cmml" xref="S2.SS1.p4.2.m2.1.1.1.3">𝜂</ci><apply id="S2.SS1.p4.2.m2.1.1.1.1.1.1.cmml" xref="S2.SS1.p4.2.m2.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p4.2.m2.1.1.1.1.1.1.1.cmml" xref="S2.SS1.p4.2.m2.1.1.1.1.1">subscript</csymbol><apply id="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.cmml" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.2"><ci id="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.1.cmml" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.1">^</ci><ci id="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.2.cmml" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.2.2">𝐴</ci></apply><ci id="S2.SS1.p4.2.m2.1.1.1.1.1.1.3.cmml" xref="S2.SS1.p4.2.m2.1.1.1.1.1.1.3">𝑑</ci></apply></apply><apply id="S2.SS1.p4.2.m2.2.2.2.cmml" xref="S2.SS1.p4.2.m2.2.2.2"><times id="S2.SS1.p4.2.m2.2.2.2.2.cmml" xref="S2.SS1.p4.2.m2.2.2.2.2"></times><ci id="S2.SS1.p4.2.m2.2.2.2.3.cmml" xref="S2.SS1.p4.2.m2.2.2.2.3">𝜂</ci><apply id="S2.SS1.p4.2.m2.2.2.2.1.1.1.cmml" xref="S2.SS1.p4.2.m2.2.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS1.p4.2.m2.2.2.2.1.1.1.1.cmml" xref="S2.SS1.p4.2.m2.2.2.2.1.1">subscript</csymbol><ci id="S2.SS1.p4.2.m2.2.2.2.1.1.1.2.cmml" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.2">𝐴</ci><ci id="S2.SS1.p4.2.m2.2.2.2.1.1.1.3.cmml" xref="S2.SS1.p4.2.m2.2.2.2.1.1.1.3">𝑑</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p4.2.m2.2c">\eta(\hat{A}_{d})&gt;\eta(A_{d})</annotation></semantics></math> and <math id="S2.SS1.p4.3.m3.1" class="ltx_Math" alttext="C" display="inline"><semantics id="S2.SS1.p4.3.m3.1a"><mi id="S2.SS1.p4.3.m3.1.1" xref="S2.SS1.p4.3.m3.1.1.cmml">C</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p4.3.m3.1b"><ci id="S2.SS1.p4.3.m3.1.1.cmml" xref="S2.SS1.p4.3.m3.1.1">𝐶</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p4.3.m3.1c">C</annotation></semantics></math> is a natural language critique that explains why the decomposition <math id="S2.SS1.p4.4.m4.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.SS1.p4.4.m4.1a"><msub id="S2.SS1.p4.4.m4.1.1" xref="S2.SS1.p4.4.m4.1.1.cmml"><mi id="S2.SS1.p4.4.m4.1.1.2" xref="S2.SS1.p4.4.m4.1.1.2.cmml">A</mi><mi id="S2.SS1.p4.4.m4.1.1.3" xref="S2.SS1.p4.4.m4.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p4.4.m4.1b"><apply id="S2.SS1.p4.4.m4.1.1.cmml" xref="S2.SS1.p4.4.m4.1.1"><csymbol cd="ambiguous" id="S2.SS1.p4.4.m4.1.1.1.cmml" xref="S2.SS1.p4.4.m4.1.1">subscript</csymbol><ci id="S2.SS1.p4.4.m4.1.1.2.cmml" xref="S2.SS1.p4.4.m4.1.1.2">𝐴</ci><ci id="S2.SS1.p4.4.m4.1.1.3.cmml" xref="S2.SS1.p4.4.m4.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p4.4.m4.1c">A_{d}</annotation></semantics></math> leads to a lower assistive value than <math id="S2.SS1.p4.5.m5.1" class="ltx_Math" alttext="\hat{A}_{d}" display="inline"><semantics id="S2.SS1.p4.5.m5.1a"><msub id="S2.SS1.p4.5.m5.1.1" xref="S2.SS1.p4.5.m5.1.1.cmml"><mover accent="true" id="S2.SS1.p4.5.m5.1.1.2" xref="S2.SS1.p4.5.m5.1.1.2.cmml"><mi id="S2.SS1.p4.5.m5.1.1.2.2" xref="S2.SS1.p4.5.m5.1.1.2.2.cmml">A</mi><mo id="S2.SS1.p4.5.m5.1.1.2.1" xref="S2.SS1.p4.5.m5.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS1.p4.5.m5.1.1.3" xref="S2.SS1.p4.5.m5.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p4.5.m5.1b"><apply id="S2.SS1.p4.5.m5.1.1.cmml" xref="S2.SS1.p4.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS1.p4.5.m5.1.1.1.cmml" xref="S2.SS1.p4.5.m5.1.1">subscript</csymbol><apply id="S2.SS1.p4.5.m5.1.1.2.cmml" xref="S2.SS1.p4.5.m5.1.1.2"><ci id="S2.SS1.p4.5.m5.1.1.2.1.cmml" xref="S2.SS1.p4.5.m5.1.1.2.1">^</ci><ci id="S2.SS1.p4.5.m5.1.1.2.2.cmml" xref="S2.SS1.p4.5.m5.1.1.2.2">𝐴</ci></apply><ci id="S2.SS1.p4.5.m5.1.1.3.cmml" xref="S2.SS1.p4.5.m5.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p4.5.m5.1c">\hat{A}_{d}</annotation></semantics></math>. Utilizing the collected training set, we learn to critique, refine, and rank decompositions to improve assistive value. Figure <a href="#S2.F2" title="Figure 2 ‣ 2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> presents the overview of our framework.</p>
</div>
</section>
<section id="S2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.2 </span>Data Collection</h3>

<section id="S2.SS2.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Data Preparation</h5>

<div id="S2.SS2.SSS0.Px1.p1" class="ltx_para">
<p id="S2.SS2.SSS0.Px1.p1.4" class="ltx_p">For each problem <math id="S2.SS2.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S2.SS2.SSS0.Px1.p1.1.m1.1a"><mi id="S2.SS2.SSS0.Px1.p1.1.m1.1.1" xref="S2.SS2.SSS0.Px1.p1.1.m1.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px1.p1.1.m1.1b"><ci id="S2.SS2.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S2.SS2.SSS0.Px1.p1.1.m1.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px1.p1.1.m1.1c">P</annotation></semantics></math> and an initial solution <math id="S2.SS2.SSS0.Px1.p1.2.m2.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S2.SS2.SSS0.Px1.p1.2.m2.1a"><mi id="S2.SS2.SSS0.Px1.p1.2.m2.1.1" xref="S2.SS2.SSS0.Px1.p1.2.m2.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px1.p1.2.m2.1b"><ci id="S2.SS2.SSS0.Px1.p1.2.m2.1.1.cmml" xref="S2.SS2.SSS0.Px1.p1.2.m2.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px1.p1.2.m2.1c">A</annotation></semantics></math>, we obtain <math id="S2.SS2.SSS0.Px1.p1.3.m3.1" class="ltx_Math" alttext="K" display="inline"><semantics id="S2.SS2.SSS0.Px1.p1.3.m3.1a"><mi id="S2.SS2.SSS0.Px1.p1.3.m3.1.1" xref="S2.SS2.SSS0.Px1.p1.3.m3.1.1.cmml">K</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px1.p1.3.m3.1b"><ci id="S2.SS2.SSS0.Px1.p1.3.m3.1.1.cmml" xref="S2.SS2.SSS0.Px1.p1.3.m3.1.1">𝐾</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px1.p1.3.m3.1c">K</annotation></semantics></math> different decomposed solutions <math id="S2.SS2.SSS0.Px1.p1.4.m4.3" class="ltx_Math" alttext="\{A_{d}^{1},\cdots,A_{d}^{K}\}" display="inline"><semantics id="S2.SS2.SSS0.Px1.p1.4.m4.3a"><mrow id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.3.cmml"><mo stretchy="false" id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.3" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.3.cmml">{</mo><msubsup id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.cmml"><mi id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.2" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.3" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.3.cmml">d</mi><mn id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.3" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.3.cmml">1</mn></msubsup><mo id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.4" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.3.cmml">,</mo><mi mathvariant="normal" id="S2.SS2.SSS0.Px1.p1.4.m4.1.1" xref="S2.SS2.SSS0.Px1.p1.4.m4.1.1.cmml">⋯</mi><mo id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.5" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.3.cmml">,</mo><msubsup id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.cmml"><mi id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.2" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.3" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.3.cmml">d</mi><mi id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.3" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.3.cmml">K</mi></msubsup><mo stretchy="false" id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.6" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.3.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px1.p1.4.m4.3b"><set id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.3.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2"><apply id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.1.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1">superscript</csymbol><apply id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.1.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1">subscript</csymbol><ci id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.2.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.3.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.2.3">𝑑</ci></apply><cn type="integer" id="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.3.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.2.2.1.1.3">1</cn></apply><ci id="S2.SS2.SSS0.Px1.p1.4.m4.1.1.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.1.1">⋯</ci><apply id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.1.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2">superscript</csymbol><apply id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.1.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2">subscript</csymbol><ci id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.2.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.3.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.2.3">𝑑</ci></apply><ci id="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.3.cmml" xref="S2.SS2.SSS0.Px1.p1.4.m4.3.3.2.2.3">𝐾</ci></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px1.p1.4.m4.3c">\{A_{d}^{1},\cdots,A_{d}^{K}\}</annotation></semantics></math> by sampling from various LMs with few-shot prompting. See more details about data preparation in Appendix <a href="#A1.SS2" title="A.2 Data Collection ‣ Appendix A Implementation Details ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A.2</span></a>.</p>
</div>
</section>
<section id="S2.SS2.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Recruiting Human Annotators</h5>

<div id="S2.SS2.SSS0.Px2.p1" class="ltx_para">
<p id="S2.SS2.SSS0.Px2.p1.1" class="ltx_p">We recruit annotators from college students. We conduct a pre-survey about their background and divide them into two groups based on their programming skills. We recruit 11 expert annotators who can solve Leetcode hard-level problems. Especially, 6 of them are medalists in the National or International Olympiad in Informatics. We recruit 19 non-expert annotators who can solve Leetcode medium-level problems but hardly solve hard-level problems. We conduct a warm-up test to train annotators to use our experiment environment. We further verify annotators’ programming skills based on their performance in the warm-up test.</p>
</div>
</section>
<section id="S2.SS2.SSS0.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Collecting Assistive Value</h5>

<div id="S2.SS2.SSS0.Px3.p1" class="ltx_para">
<p id="S2.SS2.SSS0.Px3.p1.1" class="ltx_p">Following our definition of assistive value in Equation <a href="#S2.E1" title="In 2.1 Task Definition ‣ 2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, we collect assistive value labels of different decompositions in the actual human annotation process. Specifically, for each sampled decomposed solution <math id="S2.SS2.SSS0.Px3.p1.1.m1.1" class="ltx_Math" alttext="A^{i}_{d}" display="inline"><semantics id="S2.SS2.SSS0.Px3.p1.1.m1.1a"><msubsup id="S2.SS2.SSS0.Px3.p1.1.m1.1.1" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.cmml"><mi id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.2" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.3" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.3.cmml">d</mi><mi id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.3" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px3.p1.1.m1.1b"><apply id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.1.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1">subscript</csymbol><apply id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.1.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.2.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.3.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.2.3">𝑖</ci></apply><ci id="S2.SS2.SSS0.Px3.p1.1.m1.1.1.3.cmml" xref="S2.SS2.SSS0.Px3.p1.1.m1.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px3.p1.1.m1.1c">A^{i}_{d}</annotation></semantics></math>, we collect the following labels:</p>
<ul id="S2.I1" class="ltx_itemize">
<li id="S2.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S2.I1.i1.p1" class="ltx_para">
<p id="S2.I1.i1.p1.2" class="ltx_p"><span id="S2.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">Assistive Value <math id="S2.I1.i1.p1.1.1.m1.1" class="ltx_Math" alttext="\eta^{i}" display="inline"><semantics id="S2.I1.i1.p1.1.1.m1.1a"><msup id="S2.I1.i1.p1.1.1.m1.1.1" xref="S2.I1.i1.p1.1.1.m1.1.1.cmml"><mi id="S2.I1.i1.p1.1.1.m1.1.1.2" xref="S2.I1.i1.p1.1.1.m1.1.1.2.cmml">η</mi><mi id="S2.I1.i1.p1.1.1.m1.1.1.3" xref="S2.I1.i1.p1.1.1.m1.1.1.3.cmml">i</mi></msup><annotation-xml encoding="MathML-Content" id="S2.I1.i1.p1.1.1.m1.1b"><apply id="S2.I1.i1.p1.1.1.m1.1.1.cmml" xref="S2.I1.i1.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.I1.i1.p1.1.1.m1.1.1.1.cmml" xref="S2.I1.i1.p1.1.1.m1.1.1">superscript</csymbol><ci id="S2.I1.i1.p1.1.1.m1.1.1.2.cmml" xref="S2.I1.i1.p1.1.1.m1.1.1.2">𝜂</ci><ci id="S2.I1.i1.p1.1.1.m1.1.1.3.cmml" xref="S2.I1.i1.p1.1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i1.p1.1.1.m1.1c">\eta^{i}</annotation></semantics></math>:</span> We calculate the assistive value of <math id="S2.I1.i1.p1.2.m1.1" class="ltx_Math" alttext="A_{d}^{i}" display="inline"><semantics id="S2.I1.i1.p1.2.m1.1a"><msubsup id="S2.I1.i1.p1.2.m1.1.1" xref="S2.I1.i1.p1.2.m1.1.1.cmml"><mi id="S2.I1.i1.p1.2.m1.1.1.2.2" xref="S2.I1.i1.p1.2.m1.1.1.2.2.cmml">A</mi><mi id="S2.I1.i1.p1.2.m1.1.1.2.3" xref="S2.I1.i1.p1.2.m1.1.1.2.3.cmml">d</mi><mi id="S2.I1.i1.p1.2.m1.1.1.3" xref="S2.I1.i1.p1.2.m1.1.1.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.I1.i1.p1.2.m1.1b"><apply id="S2.I1.i1.p1.2.m1.1.1.cmml" xref="S2.I1.i1.p1.2.m1.1.1"><csymbol cd="ambiguous" id="S2.I1.i1.p1.2.m1.1.1.1.cmml" xref="S2.I1.i1.p1.2.m1.1.1">superscript</csymbol><apply id="S2.I1.i1.p1.2.m1.1.1.2.cmml" xref="S2.I1.i1.p1.2.m1.1.1"><csymbol cd="ambiguous" id="S2.I1.i1.p1.2.m1.1.1.2.1.cmml" xref="S2.I1.i1.p1.2.m1.1.1">subscript</csymbol><ci id="S2.I1.i1.p1.2.m1.1.1.2.2.cmml" xref="S2.I1.i1.p1.2.m1.1.1.2.2">𝐴</ci><ci id="S2.I1.i1.p1.2.m1.1.1.2.3.cmml" xref="S2.I1.i1.p1.2.m1.1.1.2.3">𝑑</ci></apply><ci id="S2.I1.i1.p1.2.m1.1.1.3.cmml" xref="S2.I1.i1.p1.2.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i1.p1.2.m1.1c">A_{d}^{i}</annotation></semantics></math> following the definition described in Equation <a href="#S2.E1" title="In 2.1 Task Definition ‣ 2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>.</p>
</div>
</li>
<li id="S2.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S2.I1.i2.p1" class="ltx_para">
<p id="S2.I1.i2.p1.2" class="ltx_p"><span id="S2.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Critique on decomposition <math id="S2.I1.i2.p1.1.1.m1.1" class="ltx_Math" alttext="C^{i}" display="inline"><semantics id="S2.I1.i2.p1.1.1.m1.1a"><msup id="S2.I1.i2.p1.1.1.m1.1.1" xref="S2.I1.i2.p1.1.1.m1.1.1.cmml"><mi id="S2.I1.i2.p1.1.1.m1.1.1.2" xref="S2.I1.i2.p1.1.1.m1.1.1.2.cmml">C</mi><mi id="S2.I1.i2.p1.1.1.m1.1.1.3" xref="S2.I1.i2.p1.1.1.m1.1.1.3.cmml">i</mi></msup><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.1.1.m1.1b"><apply id="S2.I1.i2.p1.1.1.m1.1.1.cmml" xref="S2.I1.i2.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.I1.i2.p1.1.1.m1.1.1.1.cmml" xref="S2.I1.i2.p1.1.1.m1.1.1">superscript</csymbol><ci id="S2.I1.i2.p1.1.1.m1.1.1.2.cmml" xref="S2.I1.i2.p1.1.1.m1.1.1.2">𝐶</ci><ci id="S2.I1.i2.p1.1.1.m1.1.1.3.cmml" xref="S2.I1.i2.p1.1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.1.1.m1.1c">C^{i}</annotation></semantics></math>:</span> After repairing, we ask annotators to provide natural language critique <math id="S2.I1.i2.p1.2.m1.1" class="ltx_Math" alttext="C^{i}" display="inline"><semantics id="S2.I1.i2.p1.2.m1.1a"><msup id="S2.I1.i2.p1.2.m1.1.1" xref="S2.I1.i2.p1.2.m1.1.1.cmml"><mi id="S2.I1.i2.p1.2.m1.1.1.2" xref="S2.I1.i2.p1.2.m1.1.1.2.cmml">C</mi><mi id="S2.I1.i2.p1.2.m1.1.1.3" xref="S2.I1.i2.p1.2.m1.1.1.3.cmml">i</mi></msup><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.2.m1.1b"><apply id="S2.I1.i2.p1.2.m1.1.1.cmml" xref="S2.I1.i2.p1.2.m1.1.1"><csymbol cd="ambiguous" id="S2.I1.i2.p1.2.m1.1.1.1.cmml" xref="S2.I1.i2.p1.2.m1.1.1">superscript</csymbol><ci id="S2.I1.i2.p1.2.m1.1.1.2.cmml" xref="S2.I1.i2.p1.2.m1.1.1.2">𝐶</ci><ci id="S2.I1.i2.p1.2.m1.1.1.3.cmml" xref="S2.I1.i2.p1.2.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.2.m1.1c">C^{i}</annotation></semantics></math> on how decomposition assists or hinders their debugging.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="S2.SS2.SSS0.Px4" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Constructing Pair-wise Demonstrations</h5>

<div id="S2.SS2.SSS0.Px4.p1" class="ltx_para">
<p id="S2.SS2.SSS0.Px4.p1.9" class="ltx_p">We make <math id="S2.SS2.SSS0.Px4.p1.1.m1.2" class="ltx_Math" alttext="(A_{d}^{j},A_{d}^{i})" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.1.m1.2a"><mrow id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.3.cmml"><mo stretchy="false" id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.3" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.3.cmml">(</mo><msubsup id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.2" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.3" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.3.cmml">d</mi><mi id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.3" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.3.cmml">j</mi></msubsup><mo id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.4" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.3.cmml">,</mo><msubsup id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.cmml"><mi id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.2" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.3" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.3.cmml">d</mi><mi id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.3" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.3.cmml">i</mi></msubsup><mo stretchy="false" id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.5" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.3.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.1.m1.2b"><interval closure="open" id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.3.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2"><apply id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1">superscript</csymbol><apply id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.1.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1">subscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.2.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.3.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.2.3">𝑑</ci></apply><ci id="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.1.1.1.1.3">𝑗</ci></apply><apply id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.1.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2">superscript</csymbol><apply id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.1.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2">subscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.2.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.3.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.2.3">𝑑</ci></apply><ci id="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.3.cmml" xref="S2.SS2.SSS0.Px4.p1.1.m1.2.2.2.2.3">𝑖</ci></apply></interval></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.1.m1.2c">(A_{d}^{j},A_{d}^{i})</annotation></semantics></math> a comparison pair if they meet two requirements: (1) There is a substantial difference in assistive value between <math id="S2.SS2.SSS0.Px4.p1.2.m2.1" class="ltx_Math" alttext="\eta^{j}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.2.m2.1a"><msup id="S2.SS2.SSS0.Px4.p1.2.m2.1.1" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.2.m2.1.1.2" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1.2.cmml">η</mi><mi id="S2.SS2.SSS0.Px4.p1.2.m2.1.1.3" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1.3.cmml">j</mi></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.2.m2.1b"><apply id="S2.SS2.SSS0.Px4.p1.2.m2.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.2.m2.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.2.m2.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1.2">𝜂</ci><ci id="S2.SS2.SSS0.Px4.p1.2.m2.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.2.m2.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.2.m2.1c">\eta^{j}</annotation></semantics></math> and <math id="S2.SS2.SSS0.Px4.p1.3.m3.1" class="ltx_Math" alttext="\eta^{i}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.3.m3.1a"><msup id="S2.SS2.SSS0.Px4.p1.3.m3.1.1" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.3.m3.1.1.2" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1.2.cmml">η</mi><mi id="S2.SS2.SSS0.Px4.p1.3.m3.1.1.3" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1.3.cmml">i</mi></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.3.m3.1b"><apply id="S2.SS2.SSS0.Px4.p1.3.m3.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.3.m3.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.3.m3.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1.2">𝜂</ci><ci id="S2.SS2.SSS0.Px4.p1.3.m3.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.3.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.3.m3.1c">\eta^{i}</annotation></semantics></math>. (2) The advantages in critique <math id="S2.SS2.SSS0.Px4.p1.4.m4.1" class="ltx_Math" alttext="C^{i}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.4.m4.1a"><msup id="S2.SS2.SSS0.Px4.p1.4.m4.1.1" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.4.m4.1.1.2" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1.2.cmml">C</mi><mi id="S2.SS2.SSS0.Px4.p1.4.m4.1.1.3" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1.3.cmml">i</mi></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.4.m4.1b"><apply id="S2.SS2.SSS0.Px4.p1.4.m4.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.4.m4.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.4.m4.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1.2">𝐶</ci><ci id="S2.SS2.SSS0.Px4.p1.4.m4.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.4.m4.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.4.m4.1c">C^{i}</annotation></semantics></math> match the disadvantages in <math id="S2.SS2.SSS0.Px4.p1.5.m5.1" class="ltx_Math" alttext="C^{j}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.5.m5.1a"><msup id="S2.SS2.SSS0.Px4.p1.5.m5.1.1" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.5.m5.1.1.2" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1.2.cmml">C</mi><mi id="S2.SS2.SSS0.Px4.p1.5.m5.1.1.3" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1.3.cmml">j</mi></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.5.m5.1b"><apply id="S2.SS2.SSS0.Px4.p1.5.m5.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.5.m5.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.5.m5.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1.2">𝐶</ci><ci id="S2.SS2.SSS0.Px4.p1.5.m5.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.5.m5.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.5.m5.1c">C^{j}</annotation></semantics></math><span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>In our experiments, we manually inspect human critiques and perform matching, as we only need a few examples for in-context learning. See Appendix <a href="#A1.SS2" title="A.2 Data Collection ‣ Appendix A Implementation Details ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A.2</span></a> for more discussions.</span></span></span>, and hence <math id="S2.SS2.SSS0.Px4.p1.6.m6.1" class="ltx_Math" alttext="C^{i}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.6.m6.1a"><msup id="S2.SS2.SSS0.Px4.p1.6.m6.1.1" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.6.m6.1.1.2" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1.2.cmml">C</mi><mi id="S2.SS2.SSS0.Px4.p1.6.m6.1.1.3" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1.3.cmml">i</mi></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.6.m6.1b"><apply id="S2.SS2.SSS0.Px4.p1.6.m6.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.6.m6.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.6.m6.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1.2">𝐶</ci><ci id="S2.SS2.SSS0.Px4.p1.6.m6.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.6.m6.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.6.m6.1c">C^{i}</annotation></semantics></math> and <math id="S2.SS2.SSS0.Px4.p1.7.m7.1" class="ltx_Math" alttext="C^{j}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.7.m7.1a"><msup id="S2.SS2.SSS0.Px4.p1.7.m7.1.1" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.7.m7.1.1.2" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1.2.cmml">C</mi><mi id="S2.SS2.SSS0.Px4.p1.7.m7.1.1.3" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1.3.cmml">j</mi></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.7.m7.1b"><apply id="S2.SS2.SSS0.Px4.p1.7.m7.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.7.m7.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1">superscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.7.m7.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1.2">𝐶</ci><ci id="S2.SS2.SSS0.Px4.p1.7.m7.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.7.m7.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.7.m7.1c">C^{j}</annotation></semantics></math> explain why <math id="S2.SS2.SSS0.Px4.p1.8.m8.1" class="ltx_Math" alttext="A_{d}^{j}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.8.m8.1a"><msubsup id="S2.SS2.SSS0.Px4.p1.8.m8.1.1" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.2" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.3" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.3.cmml">d</mi><mi id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.3" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.3.cmml">j</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.8.m8.1b"><apply id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1">superscript</csymbol><apply id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.1.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1">subscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.2.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.3.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.2.3">𝑑</ci></apply><ci id="S2.SS2.SSS0.Px4.p1.8.m8.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.8.m8.1.1.3">𝑗</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.8.m8.1c">A_{d}^{j}</annotation></semantics></math> leads to more efficient human repair than <math id="S2.SS2.SSS0.Px4.p1.9.m9.1" class="ltx_Math" alttext="A_{d}^{i}" display="inline"><semantics id="S2.SS2.SSS0.Px4.p1.9.m9.1a"><msubsup id="S2.SS2.SSS0.Px4.p1.9.m9.1.1" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.cmml"><mi id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.2" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.2.cmml">A</mi><mi id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.3" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.3.cmml">d</mi><mi id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.3" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.3.cmml">i</mi></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS2.SSS0.Px4.p1.9.m9.1b"><apply id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.1.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1">superscript</csymbol><apply id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1"><csymbol cd="ambiguous" id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.1.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1">subscript</csymbol><ci id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.2.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.2">𝐴</ci><ci id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.3.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.2.3">𝑑</ci></apply><ci id="S2.SS2.SSS0.Px4.p1.9.m9.1.1.3.cmml" xref="S2.SS2.SSS0.Px4.p1.9.m9.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.SSS0.Px4.p1.9.m9.1c">A_{d}^{i}</annotation></semantics></math>.</p>
</div>
</section>
</section>
<section id="S2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.3 </span>Models</h3>

<div id="S2.SS3.p1" class="ltx_para">
<p id="S2.SS3.p1.3" class="ltx_p">Starting with a vanilla decomposition generated by naively prompting LMs, we predict human critiques on it with <math id="S2.SS3.p1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{critic}}" display="inline"><semantics id="S2.SS3.p1.1.m1.1a"><msub id="S2.SS3.p1.1.m1.1.1" xref="S2.SS3.p1.1.m1.1.1.cmml"><mi id="S2.SS3.p1.1.m1.1.1.2" xref="S2.SS3.p1.1.m1.1.1.2.cmml">π</mi><mtext id="S2.SS3.p1.1.m1.1.1.3" xref="S2.SS3.p1.1.m1.1.1.3a.cmml">critic</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.1.m1.1b"><apply id="S2.SS3.p1.1.m1.1.1.cmml" xref="S2.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p1.1.m1.1.1.1.cmml" xref="S2.SS3.p1.1.m1.1.1">subscript</csymbol><ci id="S2.SS3.p1.1.m1.1.1.2.cmml" xref="S2.SS3.p1.1.m1.1.1.2">𝜋</ci><ci id="S2.SS3.p1.1.m1.1.1.3a.cmml" xref="S2.SS3.p1.1.m1.1.1.3"><mtext mathsize="70%" id="S2.SS3.p1.1.m1.1.1.3.cmml" xref="S2.SS3.p1.1.m1.1.1.3">critic</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.1.m1.1c">\pi_{\text{critic}}</annotation></semantics></math>, refine the decomposition according to the critique with <math id="S2.SS3.p1.2.m2.1" class="ltx_Math" alttext="\pi_{\text{refine}}" display="inline"><semantics id="S2.SS3.p1.2.m2.1a"><msub id="S2.SS3.p1.2.m2.1.1" xref="S2.SS3.p1.2.m2.1.1.cmml"><mi id="S2.SS3.p1.2.m2.1.1.2" xref="S2.SS3.p1.2.m2.1.1.2.cmml">π</mi><mtext id="S2.SS3.p1.2.m2.1.1.3" xref="S2.SS3.p1.2.m2.1.1.3a.cmml">refine</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.2.m2.1b"><apply id="S2.SS3.p1.2.m2.1.1.cmml" xref="S2.SS3.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S2.SS3.p1.2.m2.1.1.1.cmml" xref="S2.SS3.p1.2.m2.1.1">subscript</csymbol><ci id="S2.SS3.p1.2.m2.1.1.2.cmml" xref="S2.SS3.p1.2.m2.1.1.2">𝜋</ci><ci id="S2.SS3.p1.2.m2.1.1.3a.cmml" xref="S2.SS3.p1.2.m2.1.1.3"><mtext mathsize="70%" id="S2.SS3.p1.2.m2.1.1.3.cmml" xref="S2.SS3.p1.2.m2.1.1.3">refine</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.2.m2.1c">\pi_{\text{refine}}</annotation></semantics></math>, and rank candidate decompositions to select the final high-<span id="S2.SS3.p1.3.1" class="ltx_text ltx_font_typewriter">AssistV</span> output with <math id="S2.SS3.p1.3.m3.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S2.SS3.p1.3.m3.1a"><msub id="S2.SS3.p1.3.m3.1.1" xref="S2.SS3.p1.3.m3.1.1.cmml"><mi id="S2.SS3.p1.3.m3.1.1.2" xref="S2.SS3.p1.3.m3.1.1.2.cmml">π</mi><mtext id="S2.SS3.p1.3.m3.1.1.3" xref="S2.SS3.p1.3.m3.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.3.m3.1b"><apply id="S2.SS3.p1.3.m3.1.1.cmml" xref="S2.SS3.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S2.SS3.p1.3.m3.1.1.1.cmml" xref="S2.SS3.p1.3.m3.1.1">subscript</csymbol><ci id="S2.SS3.p1.3.m3.1.1.2.cmml" xref="S2.SS3.p1.3.m3.1.1.2">𝜋</ci><ci id="S2.SS3.p1.3.m3.1.1.3a.cmml" xref="S2.SS3.p1.3.m3.1.1.3"><mtext mathsize="70%" id="S2.SS3.p1.3.m3.1.1.3.cmml" xref="S2.SS3.p1.3.m3.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.3.m3.1c">\pi_{\text{rank}}</annotation></semantics></math>. Figure <a href="#S2.F2" title="Figure 2 ‣ 2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> presents the overview of our framework. Notably, we introduce critique as an integral step instead of directly generating refined decomposition since it provides enriched information for models to learn how decomposition can achieve improved assistive value. Additionally, it also enables controllable decomposition as humans can manually edit critiques (e.g., requiring specific decomposition on certain complex subtasks).</p>
</div>
<div id="S2.SS3.p2" class="ltx_para">
<p id="S2.SS3.p2.1" class="ltx_p">We next describe the detailed inputs and outputs of each model:</p>
<ul id="S2.I2" class="ltx_itemize">
<li id="S2.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S2.I2.i1.p1" class="ltx_para">
<p id="S2.I2.i1.p1.4" class="ltx_p"><span id="S2.I2.i1.p1.1.1" class="ltx_text ltx_font_bold">Critic Model <math id="S2.I2.i1.p1.1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{critic}}" display="inline"><semantics id="S2.I2.i1.p1.1.1.m1.1a"><msub id="S2.I2.i1.p1.1.1.m1.1.1" xref="S2.I2.i1.p1.1.1.m1.1.1.cmml"><mi id="S2.I2.i1.p1.1.1.m1.1.1.2" xref="S2.I2.i1.p1.1.1.m1.1.1.2.cmml">π</mi><mtext class="ltx_mathvariant_bold" id="S2.I2.i1.p1.1.1.m1.1.1.3" xref="S2.I2.i1.p1.1.1.m1.1.1.3a.cmml">critic</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i1.p1.1.1.m1.1b"><apply id="S2.I2.i1.p1.1.1.m1.1.1.cmml" xref="S2.I2.i1.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.I2.i1.p1.1.1.m1.1.1.1.cmml" xref="S2.I2.i1.p1.1.1.m1.1.1">subscript</csymbol><ci id="S2.I2.i1.p1.1.1.m1.1.1.2.cmml" xref="S2.I2.i1.p1.1.1.m1.1.1.2">𝜋</ci><ci id="S2.I2.i1.p1.1.1.m1.1.1.3a.cmml" xref="S2.I2.i1.p1.1.1.m1.1.1.3"><mtext class="ltx_mathvariant_bold" mathsize="70%" id="S2.I2.i1.p1.1.1.m1.1.1.3.cmml" xref="S2.I2.i1.p1.1.1.m1.1.1.3">critic</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i1.p1.1.1.m1.1c">\pi_{\text{critic}}</annotation></semantics></math></span>: It takes a problem <math id="S2.I2.i1.p1.2.m1.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S2.I2.i1.p1.2.m1.1a"><mi id="S2.I2.i1.p1.2.m1.1.1" xref="S2.I2.i1.p1.2.m1.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S2.I2.i1.p1.2.m1.1b"><ci id="S2.I2.i1.p1.2.m1.1.1.cmml" xref="S2.I2.i1.p1.2.m1.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i1.p1.2.m1.1c">P</annotation></semantics></math> and a decomposed solution <math id="S2.I2.i1.p1.3.m2.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.I2.i1.p1.3.m2.1a"><msub id="S2.I2.i1.p1.3.m2.1.1" xref="S2.I2.i1.p1.3.m2.1.1.cmml"><mi id="S2.I2.i1.p1.3.m2.1.1.2" xref="S2.I2.i1.p1.3.m2.1.1.2.cmml">A</mi><mi id="S2.I2.i1.p1.3.m2.1.1.3" xref="S2.I2.i1.p1.3.m2.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i1.p1.3.m2.1b"><apply id="S2.I2.i1.p1.3.m2.1.1.cmml" xref="S2.I2.i1.p1.3.m2.1.1"><csymbol cd="ambiguous" id="S2.I2.i1.p1.3.m2.1.1.1.cmml" xref="S2.I2.i1.p1.3.m2.1.1">subscript</csymbol><ci id="S2.I2.i1.p1.3.m2.1.1.2.cmml" xref="S2.I2.i1.p1.3.m2.1.1.2">𝐴</ci><ci id="S2.I2.i1.p1.3.m2.1.1.3.cmml" xref="S2.I2.i1.p1.3.m2.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i1.p1.3.m2.1c">A_{d}</annotation></semantics></math> as inputs, and outputs critique on how to improve <math id="S2.I2.i1.p1.4.m3.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.I2.i1.p1.4.m3.1a"><msub id="S2.I2.i1.p1.4.m3.1.1" xref="S2.I2.i1.p1.4.m3.1.1.cmml"><mi id="S2.I2.i1.p1.4.m3.1.1.2" xref="S2.I2.i1.p1.4.m3.1.1.2.cmml">A</mi><mi id="S2.I2.i1.p1.4.m3.1.1.3" xref="S2.I2.i1.p1.4.m3.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i1.p1.4.m3.1b"><apply id="S2.I2.i1.p1.4.m3.1.1.cmml" xref="S2.I2.i1.p1.4.m3.1.1"><csymbol cd="ambiguous" id="S2.I2.i1.p1.4.m3.1.1.1.cmml" xref="S2.I2.i1.p1.4.m3.1.1">subscript</csymbol><ci id="S2.I2.i1.p1.4.m3.1.1.2.cmml" xref="S2.I2.i1.p1.4.m3.1.1.2">𝐴</ci><ci id="S2.I2.i1.p1.4.m3.1.1.3.cmml" xref="S2.I2.i1.p1.4.m3.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i1.p1.4.m3.1c">A_{d}</annotation></semantics></math> for higher assistive value.</p>
</div>
</li>
<li id="S2.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S2.I2.i2.p1" class="ltx_para">
<p id="S2.I2.i2.p1.5" class="ltx_p"><span id="S2.I2.i2.p1.1.1" class="ltx_text ltx_font_bold">Refine Model <math id="S2.I2.i2.p1.1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{refine}}" display="inline"><semantics id="S2.I2.i2.p1.1.1.m1.1a"><msub id="S2.I2.i2.p1.1.1.m1.1.1" xref="S2.I2.i2.p1.1.1.m1.1.1.cmml"><mi id="S2.I2.i2.p1.1.1.m1.1.1.2" xref="S2.I2.i2.p1.1.1.m1.1.1.2.cmml">π</mi><mtext class="ltx_mathvariant_bold" id="S2.I2.i2.p1.1.1.m1.1.1.3" xref="S2.I2.i2.p1.1.1.m1.1.1.3a.cmml">refine</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i2.p1.1.1.m1.1b"><apply id="S2.I2.i2.p1.1.1.m1.1.1.cmml" xref="S2.I2.i2.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.I2.i2.p1.1.1.m1.1.1.1.cmml" xref="S2.I2.i2.p1.1.1.m1.1.1">subscript</csymbol><ci id="S2.I2.i2.p1.1.1.m1.1.1.2.cmml" xref="S2.I2.i2.p1.1.1.m1.1.1.2">𝜋</ci><ci id="S2.I2.i2.p1.1.1.m1.1.1.3a.cmml" xref="S2.I2.i2.p1.1.1.m1.1.1.3"><mtext class="ltx_mathvariant_bold" mathsize="70%" id="S2.I2.i2.p1.1.1.m1.1.1.3.cmml" xref="S2.I2.i2.p1.1.1.m1.1.1.3">refine</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i2.p1.1.1.m1.1c">\pi_{\text{refine}}</annotation></semantics></math></span>: It takes a problem <math id="S2.I2.i2.p1.2.m1.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S2.I2.i2.p1.2.m1.1a"><mi id="S2.I2.i2.p1.2.m1.1.1" xref="S2.I2.i2.p1.2.m1.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S2.I2.i2.p1.2.m1.1b"><ci id="S2.I2.i2.p1.2.m1.1.1.cmml" xref="S2.I2.i2.p1.2.m1.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i2.p1.2.m1.1c">P</annotation></semantics></math>, a decomposed solution <math id="S2.I2.i2.p1.3.m2.1" class="ltx_Math" alttext="A_{d}" display="inline"><semantics id="S2.I2.i2.p1.3.m2.1a"><msub id="S2.I2.i2.p1.3.m2.1.1" xref="S2.I2.i2.p1.3.m2.1.1.cmml"><mi id="S2.I2.i2.p1.3.m2.1.1.2" xref="S2.I2.i2.p1.3.m2.1.1.2.cmml">A</mi><mi id="S2.I2.i2.p1.3.m2.1.1.3" xref="S2.I2.i2.p1.3.m2.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i2.p1.3.m2.1b"><apply id="S2.I2.i2.p1.3.m2.1.1.cmml" xref="S2.I2.i2.p1.3.m2.1.1"><csymbol cd="ambiguous" id="S2.I2.i2.p1.3.m2.1.1.1.cmml" xref="S2.I2.i2.p1.3.m2.1.1">subscript</csymbol><ci id="S2.I2.i2.p1.3.m2.1.1.2.cmml" xref="S2.I2.i2.p1.3.m2.1.1.2">𝐴</ci><ci id="S2.I2.i2.p1.3.m2.1.1.3.cmml" xref="S2.I2.i2.p1.3.m2.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i2.p1.3.m2.1c">A_{d}</annotation></semantics></math>, and a critique <math id="S2.I2.i2.p1.4.m3.1" class="ltx_Math" alttext="C" display="inline"><semantics id="S2.I2.i2.p1.4.m3.1a"><mi id="S2.I2.i2.p1.4.m3.1.1" xref="S2.I2.i2.p1.4.m3.1.1.cmml">C</mi><annotation-xml encoding="MathML-Content" id="S2.I2.i2.p1.4.m3.1b"><ci id="S2.I2.i2.p1.4.m3.1.1.cmml" xref="S2.I2.i2.p1.4.m3.1.1">𝐶</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i2.p1.4.m3.1c">C</annotation></semantics></math> as inputs, and outputs a refined decomposed solution <math id="S2.I2.i2.p1.5.m4.1" class="ltx_Math" alttext="\hat{A}_{d}" display="inline"><semantics id="S2.I2.i2.p1.5.m4.1a"><msub id="S2.I2.i2.p1.5.m4.1.1" xref="S2.I2.i2.p1.5.m4.1.1.cmml"><mover accent="true" id="S2.I2.i2.p1.5.m4.1.1.2" xref="S2.I2.i2.p1.5.m4.1.1.2.cmml"><mi id="S2.I2.i2.p1.5.m4.1.1.2.2" xref="S2.I2.i2.p1.5.m4.1.1.2.2.cmml">A</mi><mo id="S2.I2.i2.p1.5.m4.1.1.2.1" xref="S2.I2.i2.p1.5.m4.1.1.2.1.cmml">^</mo></mover><mi id="S2.I2.i2.p1.5.m4.1.1.3" xref="S2.I2.i2.p1.5.m4.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i2.p1.5.m4.1b"><apply id="S2.I2.i2.p1.5.m4.1.1.cmml" xref="S2.I2.i2.p1.5.m4.1.1"><csymbol cd="ambiguous" id="S2.I2.i2.p1.5.m4.1.1.1.cmml" xref="S2.I2.i2.p1.5.m4.1.1">subscript</csymbol><apply id="S2.I2.i2.p1.5.m4.1.1.2.cmml" xref="S2.I2.i2.p1.5.m4.1.1.2"><ci id="S2.I2.i2.p1.5.m4.1.1.2.1.cmml" xref="S2.I2.i2.p1.5.m4.1.1.2.1">^</ci><ci id="S2.I2.i2.p1.5.m4.1.1.2.2.cmml" xref="S2.I2.i2.p1.5.m4.1.1.2.2">𝐴</ci></apply><ci id="S2.I2.i2.p1.5.m4.1.1.3.cmml" xref="S2.I2.i2.p1.5.m4.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i2.p1.5.m4.1c">\hat{A}_{d}</annotation></semantics></math>.</p>
</div>
</li>
<li id="S2.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S2.I2.i3.p1" class="ltx_para">
<p id="S2.I2.i3.p1.4" class="ltx_p"><span id="S2.I2.i3.p1.1.1" class="ltx_text ltx_font_bold">Ranking Model <math id="S2.I2.i3.p1.1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S2.I2.i3.p1.1.1.m1.1a"><msub id="S2.I2.i3.p1.1.1.m1.1.1" xref="S2.I2.i3.p1.1.1.m1.1.1.cmml"><mi id="S2.I2.i3.p1.1.1.m1.1.1.2" xref="S2.I2.i3.p1.1.1.m1.1.1.2.cmml">π</mi><mtext class="ltx_mathvariant_bold" id="S2.I2.i3.p1.1.1.m1.1.1.3" xref="S2.I2.i3.p1.1.1.m1.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.I2.i3.p1.1.1.m1.1b"><apply id="S2.I2.i3.p1.1.1.m1.1.1.cmml" xref="S2.I2.i3.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.I2.i3.p1.1.1.m1.1.1.1.cmml" xref="S2.I2.i3.p1.1.1.m1.1.1">subscript</csymbol><ci id="S2.I2.i3.p1.1.1.m1.1.1.2.cmml" xref="S2.I2.i3.p1.1.1.m1.1.1.2">𝜋</ci><ci id="S2.I2.i3.p1.1.1.m1.1.1.3a.cmml" xref="S2.I2.i3.p1.1.1.m1.1.1.3"><mtext class="ltx_mathvariant_bold" mathsize="70%" id="S2.I2.i3.p1.1.1.m1.1.1.3.cmml" xref="S2.I2.i3.p1.1.1.m1.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i3.p1.1.1.m1.1c">\pi_{\text{rank}}</annotation></semantics></math></span>: It takes a problem <math id="S2.I2.i3.p1.2.m1.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S2.I2.i3.p1.2.m1.1a"><mi id="S2.I2.i3.p1.2.m1.1.1" xref="S2.I2.i3.p1.2.m1.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S2.I2.i3.p1.2.m1.1b"><ci id="S2.I2.i3.p1.2.m1.1.1.cmml" xref="S2.I2.i3.p1.2.m1.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i3.p1.2.m1.1c">P</annotation></semantics></math> and two decomposed solutions <math id="S2.I2.i3.p1.3.m2.1" class="ltx_Math" alttext="A_{d}^{1}" display="inline"><semantics id="S2.I2.i3.p1.3.m2.1a"><msubsup id="S2.I2.i3.p1.3.m2.1.1" xref="S2.I2.i3.p1.3.m2.1.1.cmml"><mi id="S2.I2.i3.p1.3.m2.1.1.2.2" xref="S2.I2.i3.p1.3.m2.1.1.2.2.cmml">A</mi><mi id="S2.I2.i3.p1.3.m2.1.1.2.3" xref="S2.I2.i3.p1.3.m2.1.1.2.3.cmml">d</mi><mn id="S2.I2.i3.p1.3.m2.1.1.3" xref="S2.I2.i3.p1.3.m2.1.1.3.cmml">1</mn></msubsup><annotation-xml encoding="MathML-Content" id="S2.I2.i3.p1.3.m2.1b"><apply id="S2.I2.i3.p1.3.m2.1.1.cmml" xref="S2.I2.i3.p1.3.m2.1.1"><csymbol cd="ambiguous" id="S2.I2.i3.p1.3.m2.1.1.1.cmml" xref="S2.I2.i3.p1.3.m2.1.1">superscript</csymbol><apply id="S2.I2.i3.p1.3.m2.1.1.2.cmml" xref="S2.I2.i3.p1.3.m2.1.1"><csymbol cd="ambiguous" id="S2.I2.i3.p1.3.m2.1.1.2.1.cmml" xref="S2.I2.i3.p1.3.m2.1.1">subscript</csymbol><ci id="S2.I2.i3.p1.3.m2.1.1.2.2.cmml" xref="S2.I2.i3.p1.3.m2.1.1.2.2">𝐴</ci><ci id="S2.I2.i3.p1.3.m2.1.1.2.3.cmml" xref="S2.I2.i3.p1.3.m2.1.1.2.3">𝑑</ci></apply><cn type="integer" id="S2.I2.i3.p1.3.m2.1.1.3.cmml" xref="S2.I2.i3.p1.3.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i3.p1.3.m2.1c">A_{d}^{1}</annotation></semantics></math>, <math id="S2.I2.i3.p1.4.m3.1" class="ltx_Math" alttext="A_{d}^{2}" display="inline"><semantics id="S2.I2.i3.p1.4.m3.1a"><msubsup id="S2.I2.i3.p1.4.m3.1.1" xref="S2.I2.i3.p1.4.m3.1.1.cmml"><mi id="S2.I2.i3.p1.4.m3.1.1.2.2" xref="S2.I2.i3.p1.4.m3.1.1.2.2.cmml">A</mi><mi id="S2.I2.i3.p1.4.m3.1.1.2.3" xref="S2.I2.i3.p1.4.m3.1.1.2.3.cmml">d</mi><mn id="S2.I2.i3.p1.4.m3.1.1.3" xref="S2.I2.i3.p1.4.m3.1.1.3.cmml">2</mn></msubsup><annotation-xml encoding="MathML-Content" id="S2.I2.i3.p1.4.m3.1b"><apply id="S2.I2.i3.p1.4.m3.1.1.cmml" xref="S2.I2.i3.p1.4.m3.1.1"><csymbol cd="ambiguous" id="S2.I2.i3.p1.4.m3.1.1.1.cmml" xref="S2.I2.i3.p1.4.m3.1.1">superscript</csymbol><apply id="S2.I2.i3.p1.4.m3.1.1.2.cmml" xref="S2.I2.i3.p1.4.m3.1.1"><csymbol cd="ambiguous" id="S2.I2.i3.p1.4.m3.1.1.2.1.cmml" xref="S2.I2.i3.p1.4.m3.1.1">subscript</csymbol><ci id="S2.I2.i3.p1.4.m3.1.1.2.2.cmml" xref="S2.I2.i3.p1.4.m3.1.1.2.2">𝐴</ci><ci id="S2.I2.i3.p1.4.m3.1.1.2.3.cmml" xref="S2.I2.i3.p1.4.m3.1.1.2.3">𝑑</ci></apply><cn type="integer" id="S2.I2.i3.p1.4.m3.1.1.3.cmml" xref="S2.I2.i3.p1.4.m3.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I2.i3.p1.4.m3.1c">A_{d}^{2}</annotation></semantics></math> as inputs, and outputs a ranking that predicts which decomposition leads to higher assistive value.</p>
</div>
</li>
</ul>
</div>
<div id="S2.SS3.p3" class="ltx_para">
<p id="S2.SS3.p3.1" class="ltx_p">For model training, inspired by recent works showing that modern LMs can learn to critique and refine model outputs via in-context learning <cite class="ltx_cite ltx_citemacro_cite">Bai et al. (<a href="#bib.bib3" title="" class="ltx_ref">2022</a>); Sun et al. (<a href="#bib.bib30" title="" class="ltx_ref">2023</a>)</cite>, we use this approach to learn our three models, where the collected training data <math id="S2.SS3.p3.1.m1.3" class="ltx_Math" alttext="D_{train}=\{(\hat{A}_{d},A_{d},C,P)\}" display="inline"><semantics id="S2.SS3.p3.1.m1.3a"><mrow id="S2.SS3.p3.1.m1.3.3" xref="S2.SS3.p3.1.m1.3.3.cmml"><msub id="S2.SS3.p3.1.m1.3.3.3" xref="S2.SS3.p3.1.m1.3.3.3.cmml"><mi id="S2.SS3.p3.1.m1.3.3.3.2" xref="S2.SS3.p3.1.m1.3.3.3.2.cmml">D</mi><mrow id="S2.SS3.p3.1.m1.3.3.3.3" xref="S2.SS3.p3.1.m1.3.3.3.3.cmml"><mi id="S2.SS3.p3.1.m1.3.3.3.3.2" xref="S2.SS3.p3.1.m1.3.3.3.3.2.cmml">t</mi><mo lspace="0em" rspace="0em" id="S2.SS3.p3.1.m1.3.3.3.3.1" xref="S2.SS3.p3.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS3.p3.1.m1.3.3.3.3.3" xref="S2.SS3.p3.1.m1.3.3.3.3.3.cmml">r</mi><mo lspace="0em" rspace="0em" id="S2.SS3.p3.1.m1.3.3.3.3.1a" xref="S2.SS3.p3.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS3.p3.1.m1.3.3.3.3.4" xref="S2.SS3.p3.1.m1.3.3.3.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S2.SS3.p3.1.m1.3.3.3.3.1b" xref="S2.SS3.p3.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS3.p3.1.m1.3.3.3.3.5" xref="S2.SS3.p3.1.m1.3.3.3.3.5.cmml">i</mi><mo lspace="0em" rspace="0em" id="S2.SS3.p3.1.m1.3.3.3.3.1c" xref="S2.SS3.p3.1.m1.3.3.3.3.1.cmml">​</mo><mi id="S2.SS3.p3.1.m1.3.3.3.3.6" xref="S2.SS3.p3.1.m1.3.3.3.3.6.cmml">n</mi></mrow></msub><mo id="S2.SS3.p3.1.m1.3.3.2" xref="S2.SS3.p3.1.m1.3.3.2.cmml">=</mo><mrow id="S2.SS3.p3.1.m1.3.3.1.1" xref="S2.SS3.p3.1.m1.3.3.1.2.cmml"><mo stretchy="false" id="S2.SS3.p3.1.m1.3.3.1.1.2" xref="S2.SS3.p3.1.m1.3.3.1.2.cmml">{</mo><mrow id="S2.SS3.p3.1.m1.3.3.1.1.1.2" xref="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml"><mo stretchy="false" id="S2.SS3.p3.1.m1.3.3.1.1.1.2.3" xref="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml">(</mo><msub id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.cmml"><mover accent="true" id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.cmml"><mi id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.2" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.2.cmml">A</mi><mo id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.1" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.3" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.3.cmml">d</mi></msub><mo id="S2.SS3.p3.1.m1.3.3.1.1.1.2.4" xref="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml">,</mo><msub id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.cmml"><mi id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.2" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.2.cmml">A</mi><mi id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.3" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.3.cmml">d</mi></msub><mo id="S2.SS3.p3.1.m1.3.3.1.1.1.2.5" xref="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml">,</mo><mi id="S2.SS3.p3.1.m1.1.1" xref="S2.SS3.p3.1.m1.1.1.cmml">C</mi><mo id="S2.SS3.p3.1.m1.3.3.1.1.1.2.6" xref="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml">,</mo><mi id="S2.SS3.p3.1.m1.2.2" xref="S2.SS3.p3.1.m1.2.2.cmml">P</mi><mo stretchy="false" id="S2.SS3.p3.1.m1.3.3.1.1.1.2.7" xref="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml">)</mo></mrow><mo stretchy="false" id="S2.SS3.p3.1.m1.3.3.1.1.3" xref="S2.SS3.p3.1.m1.3.3.1.2.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p3.1.m1.3b"><apply id="S2.SS3.p3.1.m1.3.3.cmml" xref="S2.SS3.p3.1.m1.3.3"><eq id="S2.SS3.p3.1.m1.3.3.2.cmml" xref="S2.SS3.p3.1.m1.3.3.2"></eq><apply id="S2.SS3.p3.1.m1.3.3.3.cmml" xref="S2.SS3.p3.1.m1.3.3.3"><csymbol cd="ambiguous" id="S2.SS3.p3.1.m1.3.3.3.1.cmml" xref="S2.SS3.p3.1.m1.3.3.3">subscript</csymbol><ci id="S2.SS3.p3.1.m1.3.3.3.2.cmml" xref="S2.SS3.p3.1.m1.3.3.3.2">𝐷</ci><apply id="S2.SS3.p3.1.m1.3.3.3.3.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3"><times id="S2.SS3.p3.1.m1.3.3.3.3.1.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3.1"></times><ci id="S2.SS3.p3.1.m1.3.3.3.3.2.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3.2">𝑡</ci><ci id="S2.SS3.p3.1.m1.3.3.3.3.3.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3.3">𝑟</ci><ci id="S2.SS3.p3.1.m1.3.3.3.3.4.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3.4">𝑎</ci><ci id="S2.SS3.p3.1.m1.3.3.3.3.5.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3.5">𝑖</ci><ci id="S2.SS3.p3.1.m1.3.3.3.3.6.cmml" xref="S2.SS3.p3.1.m1.3.3.3.3.6">𝑛</ci></apply></apply><set id="S2.SS3.p3.1.m1.3.3.1.2.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1"><vector id="S2.SS3.p3.1.m1.3.3.1.1.1.3.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2"><apply id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.1.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1">subscript</csymbol><apply id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2"><ci id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.1.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.1">^</ci><ci id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.2.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.2.2">𝐴</ci></apply><ci id="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.3.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.1.1.3">𝑑</ci></apply><apply id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2"><csymbol cd="ambiguous" id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.1.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2">subscript</csymbol><ci id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.2.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.2">𝐴</ci><ci id="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.3.cmml" xref="S2.SS3.p3.1.m1.3.3.1.1.1.2.2.3">𝑑</ci></apply><ci id="S2.SS3.p3.1.m1.1.1.cmml" xref="S2.SS3.p3.1.m1.1.1">𝐶</ci><ci id="S2.SS3.p3.1.m1.2.2.cmml" xref="S2.SS3.p3.1.m1.2.2">𝑃</ci></vector></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p3.1.m1.3c">D_{train}=\{(\hat{A}_{d},A_{d},C,P)\}</annotation></semantics></math> are formatted into in-context examples for each model. Example prompts are shown in Appendix <a href="#A4" title="Appendix D Example Prompts ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">D</span></a>.</p>
</div>
<div id="S2.SS3.p4" class="ltx_para">
<p id="S2.SS3.p4.5" class="ltx_p">At inference time, we apply <math id="S2.SS3.p4.1.m1.1" class="ltx_Math" alttext="\pi_{\text{critic}}" display="inline"><semantics id="S2.SS3.p4.1.m1.1a"><msub id="S2.SS3.p4.1.m1.1.1" xref="S2.SS3.p4.1.m1.1.1.cmml"><mi id="S2.SS3.p4.1.m1.1.1.2" xref="S2.SS3.p4.1.m1.1.1.2.cmml">π</mi><mtext id="S2.SS3.p4.1.m1.1.1.3" xref="S2.SS3.p4.1.m1.1.1.3a.cmml">critic</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p4.1.m1.1b"><apply id="S2.SS3.p4.1.m1.1.1.cmml" xref="S2.SS3.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p4.1.m1.1.1.1.cmml" xref="S2.SS3.p4.1.m1.1.1">subscript</csymbol><ci id="S2.SS3.p4.1.m1.1.1.2.cmml" xref="S2.SS3.p4.1.m1.1.1.2">𝜋</ci><ci id="S2.SS3.p4.1.m1.1.1.3a.cmml" xref="S2.SS3.p4.1.m1.1.1.3"><mtext mathsize="70%" id="S2.SS3.p4.1.m1.1.1.3.cmml" xref="S2.SS3.p4.1.m1.1.1.3">critic</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p4.1.m1.1c">\pi_{\text{critic}}</annotation></semantics></math>, followed by <math id="S2.SS3.p4.2.m2.1" class="ltx_Math" alttext="\pi_{\text{refine}}" display="inline"><semantics id="S2.SS3.p4.2.m2.1a"><msub id="S2.SS3.p4.2.m2.1.1" xref="S2.SS3.p4.2.m2.1.1.cmml"><mi id="S2.SS3.p4.2.m2.1.1.2" xref="S2.SS3.p4.2.m2.1.1.2.cmml">π</mi><mtext id="S2.SS3.p4.2.m2.1.1.3" xref="S2.SS3.p4.2.m2.1.1.3a.cmml">refine</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p4.2.m2.1b"><apply id="S2.SS3.p4.2.m2.1.1.cmml" xref="S2.SS3.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S2.SS3.p4.2.m2.1.1.1.cmml" xref="S2.SS3.p4.2.m2.1.1">subscript</csymbol><ci id="S2.SS3.p4.2.m2.1.1.2.cmml" xref="S2.SS3.p4.2.m2.1.1.2">𝜋</ci><ci id="S2.SS3.p4.2.m2.1.1.3a.cmml" xref="S2.SS3.p4.2.m2.1.1.3"><mtext mathsize="70%" id="S2.SS3.p4.2.m2.1.1.3.cmml" xref="S2.SS3.p4.2.m2.1.1.3">refine</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p4.2.m2.1c">\pi_{\text{refine}}</annotation></semantics></math>, to produce a refined decomposition set <math id="S2.SS3.p4.3.m3.1" class="ltx_Math" alttext="\{\hat{A}_{d}\}" display="inline"><semantics id="S2.SS3.p4.3.m3.1a"><mrow id="S2.SS3.p4.3.m3.1.1.1" xref="S2.SS3.p4.3.m3.1.1.2.cmml"><mo stretchy="false" id="S2.SS3.p4.3.m3.1.1.1.2" xref="S2.SS3.p4.3.m3.1.1.2.cmml">{</mo><msub id="S2.SS3.p4.3.m3.1.1.1.1" xref="S2.SS3.p4.3.m3.1.1.1.1.cmml"><mover accent="true" id="S2.SS3.p4.3.m3.1.1.1.1.2" xref="S2.SS3.p4.3.m3.1.1.1.1.2.cmml"><mi id="S2.SS3.p4.3.m3.1.1.1.1.2.2" xref="S2.SS3.p4.3.m3.1.1.1.1.2.2.cmml">A</mi><mo id="S2.SS3.p4.3.m3.1.1.1.1.2.1" xref="S2.SS3.p4.3.m3.1.1.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS3.p4.3.m3.1.1.1.1.3" xref="S2.SS3.p4.3.m3.1.1.1.1.3.cmml">d</mi></msub><mo stretchy="false" id="S2.SS3.p4.3.m3.1.1.1.3" xref="S2.SS3.p4.3.m3.1.1.2.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p4.3.m3.1b"><set id="S2.SS3.p4.3.m3.1.1.2.cmml" xref="S2.SS3.p4.3.m3.1.1.1"><apply id="S2.SS3.p4.3.m3.1.1.1.1.cmml" xref="S2.SS3.p4.3.m3.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p4.3.m3.1.1.1.1.1.cmml" xref="S2.SS3.p4.3.m3.1.1.1.1">subscript</csymbol><apply id="S2.SS3.p4.3.m3.1.1.1.1.2.cmml" xref="S2.SS3.p4.3.m3.1.1.1.1.2"><ci id="S2.SS3.p4.3.m3.1.1.1.1.2.1.cmml" xref="S2.SS3.p4.3.m3.1.1.1.1.2.1">^</ci><ci id="S2.SS3.p4.3.m3.1.1.1.1.2.2.cmml" xref="S2.SS3.p4.3.m3.1.1.1.1.2.2">𝐴</ci></apply><ci id="S2.SS3.p4.3.m3.1.1.1.1.3.cmml" xref="S2.SS3.p4.3.m3.1.1.1.1.3">𝑑</ci></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p4.3.m3.1c">\{\hat{A}_{d}\}</annotation></semantics></math>, and then use <math id="S2.SS3.p4.4.m4.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S2.SS3.p4.4.m4.1a"><msub id="S2.SS3.p4.4.m4.1.1" xref="S2.SS3.p4.4.m4.1.1.cmml"><mi id="S2.SS3.p4.4.m4.1.1.2" xref="S2.SS3.p4.4.m4.1.1.2.cmml">π</mi><mtext id="S2.SS3.p4.4.m4.1.1.3" xref="S2.SS3.p4.4.m4.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p4.4.m4.1b"><apply id="S2.SS3.p4.4.m4.1.1.cmml" xref="S2.SS3.p4.4.m4.1.1"><csymbol cd="ambiguous" id="S2.SS3.p4.4.m4.1.1.1.cmml" xref="S2.SS3.p4.4.m4.1.1">subscript</csymbol><ci id="S2.SS3.p4.4.m4.1.1.2.cmml" xref="S2.SS3.p4.4.m4.1.1.2">𝜋</ci><ci id="S2.SS3.p4.4.m4.1.1.3a.cmml" xref="S2.SS3.p4.4.m4.1.1.3"><mtext mathsize="70%" id="S2.SS3.p4.4.m4.1.1.3.cmml" xref="S2.SS3.p4.4.m4.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p4.4.m4.1c">\pi_{\text{rank}}</annotation></semantics></math> to select a decomposition <math id="S2.SS3.p4.5.m5.1" class="ltx_Math" alttext="A_{d}^{*}" display="inline"><semantics id="S2.SS3.p4.5.m5.1a"><msubsup id="S2.SS3.p4.5.m5.1.1" xref="S2.SS3.p4.5.m5.1.1.cmml"><mi id="S2.SS3.p4.5.m5.1.1.2.2" xref="S2.SS3.p4.5.m5.1.1.2.2.cmml">A</mi><mi id="S2.SS3.p4.5.m5.1.1.2.3" xref="S2.SS3.p4.5.m5.1.1.2.3.cmml">d</mi><mo id="S2.SS3.p4.5.m5.1.1.3" xref="S2.SS3.p4.5.m5.1.1.3.cmml">∗</mo></msubsup><annotation-xml encoding="MathML-Content" id="S2.SS3.p4.5.m5.1b"><apply id="S2.SS3.p4.5.m5.1.1.cmml" xref="S2.SS3.p4.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS3.p4.5.m5.1.1.1.cmml" xref="S2.SS3.p4.5.m5.1.1">superscript</csymbol><apply id="S2.SS3.p4.5.m5.1.1.2.cmml" xref="S2.SS3.p4.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS3.p4.5.m5.1.1.2.1.cmml" xref="S2.SS3.p4.5.m5.1.1">subscript</csymbol><ci id="S2.SS3.p4.5.m5.1.1.2.2.cmml" xref="S2.SS3.p4.5.m5.1.1.2.2">𝐴</ci><ci id="S2.SS3.p4.5.m5.1.1.2.3.cmml" xref="S2.SS3.p4.5.m5.1.1.2.3">𝑑</ci></apply><times id="S2.SS3.p4.5.m5.1.1.3.cmml" xref="S2.SS3.p4.5.m5.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p4.5.m5.1c">A_{d}^{*}</annotation></semantics></math> among them as the final decomposition.</p>
</div>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Experiments</h2>

<figure id="S3.F4" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="S3.F4.1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_center ltx_align_top" style="width:173.4pt;"><img src="/html/2406.04604/assets/pics/acc.png" id="S3.F4.1.g1" class="ltx_graphics ltx_img_square" width="359" height="370" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 3: </span>Humans provide higher-quality labels with decomposition. Dark color denotes strict accuracy of human-repaired programs; light color denotes test case average accuracy.</figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="S3.F4.2" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_center ltx_align_top" style="width:238.5pt;"><img src="/html/2406.04604/assets/x3.png" id="S3.F4.2.g1" class="ltx_graphics ltx_img_landscape" width="461" height="185" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 4: </span>Decomposition improves human efficiency. We plot the relationship between the human-repaired program’s test case average accuracy and human time cost.</figcaption>
</figure>
</div>
</div>
</figure>
<section id="S3.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.1 </span>Setup</h3>

<section id="S3.SS1.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Benchmark</h5>

<div id="S3.SS1.SSS0.Px1.p1" class="ltx_para">
<p id="S3.SS1.SSS0.Px1.p1.1" class="ltx_p">We conduct experiments with the problems from two widely adopted competition-level code generation benchmarks, namely APPS <cite class="ltx_cite ltx_citemacro_cite">Hendrycks et al. (<a href="#bib.bib13" title="" class="ltx_ref">2021</a>)</cite> and Code-Contests <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib20" title="" class="ltx_ref">2022</a>)</cite>. For human evaluation, we filter those problems where the model-generated program directly passes all test cases and randomly sample 30 problems as the test data.</p>
</div>
</section>
<section id="S3.SS1.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Metric</h5>

<div id="S3.SS1.SSS0.Px2.p1" class="ltx_para">
<p id="S3.SS1.SSS0.Px2.p1.1" class="ltx_p">We measure the supervision quality in competitive programming by programs’ passing rates on unit test cases. Specifically, we aggregate the program’s performance on test cases with two metrics. <span id="S3.SS1.SSS0.Px2.p1.1.1" class="ltx_text ltx_font_bold">Test Case Average Accuracy</span> computes the average fraction of test cases passed among all the test cases. <span id="S3.SS1.SSS0.Px2.p1.1.2" class="ltx_text ltx_font_bold">Strict Accuracy</span> computes the average fraction of programs that pass all test cases.</p>
</div>
</section>
<section id="S3.SS1.SSS0.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Baselines</h5>

<div id="S3.SS1.SSS0.Px3.p1" class="ltx_para">
<p id="S3.SS1.SSS0.Px3.p1.1" class="ltx_p">We consider three baselines:</p>
<ul id="S3.I1" class="ltx_itemize">
<li id="S3.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i1.p1" class="ltx_para">
<p id="S3.I1.i1.p1.3" class="ltx_p"><span id="S3.I1.i1.p1.3.1" class="ltx_text ltx_font_bold">Initial</span>: It prompts a code language model <math id="S3.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="S3.I1.i1.p1.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.I1.i1.p1.1.m1.1.1" xref="S3.I1.i1.p1.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="S3.I1.i1.p1.1.m1.1b"><ci id="S3.I1.i1.p1.1.m1.1.1.cmml" xref="S3.I1.i1.p1.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i1.p1.1.m1.1c">\mathcal{M}</annotation></semantics></math> with the problem <math id="S3.I1.i1.p1.2.m2.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S3.I1.i1.p1.2.m2.1a"><mi id="S3.I1.i1.p1.2.m2.1.1" xref="S3.I1.i1.p1.2.m2.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S3.I1.i1.p1.2.m2.1b"><ci id="S3.I1.i1.p1.2.m2.1.1.cmml" xref="S3.I1.i1.p1.2.m2.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i1.p1.2.m2.1c">P</annotation></semantics></math> to generate a solution <math id="S3.I1.i1.p1.3.m3.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S3.I1.i1.p1.3.m3.1a"><mi id="S3.I1.i1.p1.3.m3.1.1" xref="S3.I1.i1.p1.3.m3.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S3.I1.i1.p1.3.m3.1b"><ci id="S3.I1.i1.p1.3.m3.1.1.cmml" xref="S3.I1.i1.p1.3.m3.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i1.p1.3.m3.1c">A</annotation></semantics></math> without explicit instructions for decomposition.</p>
</div>
</li>
<li id="S3.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i2.p1" class="ltx_para">
<p id="S3.I1.i2.p1.1" class="ltx_p"><span id="S3.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Heuristic Decomposition</span>: Inspired by McCabe’s cyclomatic complexity <cite class="ltx_cite ltx_citemacro_cite">McCabe (<a href="#bib.bib22" title="" class="ltx_ref">1976</a>)</cite>, a widely adopted metric for measuring code complexity in software engineering, we implement a heuristic baseline to decompose a complex program into simpler pieces. Specifically, we decompose each if-statement as well as for or while loops into a separate function since these code structures contribute to higher cyclomatic complexity. We then generate post-hoc subtask descriptions using GPT-4.</p>
</div>
</li>
<li id="S3.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i3.p1" class="ltx_para">
<p id="S3.I1.i3.p1.1" class="ltx_p"><span id="S3.I1.i3.p1.1.1" class="ltx_text ltx_font_bold">Vanilla Decomposition</span>: It prompts a code language model <math id="S3.I1.i3.p1.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="S3.I1.i3.p1.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.I1.i3.p1.1.m1.1.1" xref="S3.I1.i3.p1.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="S3.I1.i3.p1.1.m1.1b"><ci id="S3.I1.i3.p1.1.m1.1.1.cmml" xref="S3.I1.i3.p1.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i3.p1.1.m1.1c">\mathcal{M}</annotation></semantics></math> to perform decomposition with basic author-crafted demonstrations without learning from human feedback.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="S3.SS1.SSS0.Px4" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Annotation Procedure</h5>

<div id="S3.SS1.SSS0.Px4.p1" class="ltx_para">
<p id="S3.SS1.SSS0.Px4.p1.1" class="ltx_p">We conduct experiments based on an internal Online Judge system. Problems are randomly assigned to each annotator while ensuring that they have not seen the assigned problem before (if they have, the problem will be reassigned) and never repeatedly debug the same problem. Next, given a competition-level problem, several exemplified public test cases, and a solution, labelers are required to perform debugging. See Appendix <a href="#A3" title="Appendix C Additional Human Annotation Details ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a> for more annotation details.</p>
</div>
</section>
<section id="S3.SS1.SSS0.Px5" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Implementation Details</h5>

<div id="S3.SS1.SSS0.Px5.p1" class="ltx_para">
<p id="S3.SS1.SSS0.Px5.p1.1" class="ltx_p">We use GPT-4 <cite class="ltx_cite ltx_citemacro_cite">OpenAI (<a href="#bib.bib23" title="" class="ltx_ref">2023</a>)</cite> as our default backbone model <math id="S3.SS1.SSS0.Px5.p1.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="S3.SS1.SSS0.Px5.p1.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.SSS0.Px5.p1.1.m1.1.1" xref="S3.SS1.SSS0.Px5.p1.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.SSS0.Px5.p1.1.m1.1b"><ci id="S3.SS1.SSS0.Px5.p1.1.m1.1.1.cmml" xref="S3.SS1.SSS0.Px5.p1.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.SSS0.Px5.p1.1.m1.1c">\mathcal{M}</annotation></semantics></math> for code generation and decomposition due to its superior in-context learning capabilities. To focus our evaluation on the impact of decomposition, we ensure the consistency between initial and decomposed solutions based on their outputs on test cases. If the consistency check fails, we reject the decomposition and retry within a sampling budget. The consistency check is applied in all baselines for fair comparisons. See Appendix <a href="#A1.SS1" title="A.1 Consistency of Decomposition ‣ Appendix A Implementation Details ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A.1</span></a> for more implementation details.</p>
</div>
</section>
</section>
<section id="S3.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.2 </span>Assisting Human Supervision</h3>

<div id="S3.SS2.p1" class="ltx_para">
<p id="S3.SS2.p1.1" class="ltx_p">We evaluate the effectiveness of decomposition in aiding humans to repair programs and provide reliable supervision signals.</p>
</div>
<section id="S3.SS2.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">3.2.1 </span>Quantitative Analysis</h4>

<section id="S3.SS2.SSS1.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Supervising competitive programming is extremely challenging</h5>

<div id="S3.SS2.SSS1.Px1.p1" class="ltx_para">
<p id="S3.SS2.SSS1.Px1.p1.1" class="ltx_p">In our experiments, highly experienced experts spend at least 68 minutes repairing 56.7% of the model-generated programs, and non-experts spend at least 74 minutes repairing 30% of the model-generated programs. These results indicate that competitive programming serves as a good testbed for studying scalable oversight.</p>
</div>
</section>
<section id="S3.SS2.SSS1.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Decomposition improves human efficiency</h5>

<div id="S3.SS2.SSS1.Px2.p1" class="ltx_para">
<p id="S3.SS2.SSS1.Px2.p1.3" class="ltx_p">Figure <a href="#S3.F4" title="Figure 4 ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> demonstrates that humans assisted with our decomposition model significantly outperform those without the assistance in terms of the speed to repair model-generated programs. For instance, the time required to collect repaired programs with 68% test case accuracy from non-experts is reduced from 40 minutes to 12 minutes, achieving a 3.3<math id="S3.SS2.SSS1.Px2.p1.1.m1.1" class="ltx_Math" alttext="\times" display="inline"><semantics id="S3.SS2.SSS1.Px2.p1.1.m1.1a"><mo id="S3.SS2.SSS1.Px2.p1.1.m1.1.1" xref="S3.SS2.SSS1.Px2.p1.1.m1.1.1.cmml">×</mo><annotation-xml encoding="MathML-Content" id="S3.SS2.SSS1.Px2.p1.1.m1.1b"><times id="S3.SS2.SSS1.Px2.p1.1.m1.1.1.cmml" xref="S3.SS2.SSS1.Px2.p1.1.m1.1.1"></times></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.SSS1.Px2.p1.1.m1.1c">\times</annotation></semantics></math> speedup. Paired <math id="S3.SS2.SSS1.Px2.p1.2.m2.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S3.SS2.SSS1.Px2.p1.2.m2.1a"><mi id="S3.SS2.SSS1.Px2.p1.2.m2.1.1" xref="S3.SS2.SSS1.Px2.p1.2.m2.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.SSS1.Px2.p1.2.m2.1b"><ci id="S3.SS2.SSS1.Px2.p1.2.m2.1.1.cmml" xref="S3.SS2.SSS1.Px2.p1.2.m2.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.SSS1.Px2.p1.2.m2.1c">t</annotation></semantics></math>-tests confirm that these efficiency gains are statistically significant (<math id="S3.SS2.SSS1.Px2.p1.3.m3.1" class="ltx_Math" alttext="p&lt;0.005" display="inline"><semantics id="S3.SS2.SSS1.Px2.p1.3.m3.1a"><mrow id="S3.SS2.SSS1.Px2.p1.3.m3.1.1" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.cmml"><mi id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.2" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.2.cmml">p</mi><mo id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.1" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.1.cmml">&lt;</mo><mn id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.3" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.3.cmml">0.005</mn></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.SSS1.Px2.p1.3.m3.1b"><apply id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.cmml" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1"><lt id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.1.cmml" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.1"></lt><ci id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.2.cmml" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.2">𝑝</ci><cn type="float" id="S3.SS2.SSS1.Px2.p1.3.m3.1.1.3.cmml" xref="S3.SS2.SSS1.Px2.p1.3.m3.1.1.3">0.005</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.SSS1.Px2.p1.3.m3.1c">p&lt;0.005</annotation></semantics></math>) for both experts and non-experts. These results indicate that our decomposition model can effectively improve the efficiency of human labelers.</p>
</div>
</section>
<section id="S3.SS2.SSS1.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Humans provide higher-quality labels with decomposition</h5>

<div id="S3.SS2.SSS1.Px3.p1" class="ltx_para">
<p id="S3.SS2.SSS1.Px3.p1.2" class="ltx_p">Beyond efficiency, we evaluate how decomposition impacts the final quality of human labels. As shown in Figure <a href="#S3.F4" title="Figure 4 ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, our decomposition model enables humans to repair more model-generated programs. For instance, decomposition improves the strict accuracy of repaired programs from 56.7% to 90% for experts and from 30.0% to 63.3% for non-experts. Paired <math id="S3.SS2.SSS1.Px3.p1.1.m1.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S3.SS2.SSS1.Px3.p1.1.m1.1a"><mi id="S3.SS2.SSS1.Px3.p1.1.m1.1.1" xref="S3.SS2.SSS1.Px3.p1.1.m1.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.SSS1.Px3.p1.1.m1.1b"><ci id="S3.SS2.SSS1.Px3.p1.1.m1.1.1.cmml" xref="S3.SS2.SSS1.Px3.p1.1.m1.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.SSS1.Px3.p1.1.m1.1c">t</annotation></semantics></math>-tests confirm the significance (<math id="S3.SS2.SSS1.Px3.p1.2.m2.1" class="ltx_Math" alttext="p&lt;0.01" display="inline"><semantics id="S3.SS2.SSS1.Px3.p1.2.m2.1a"><mrow id="S3.SS2.SSS1.Px3.p1.2.m2.1.1" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.cmml"><mi id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.2" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.2.cmml">p</mi><mo id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.1" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.1.cmml">&lt;</mo><mn id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.3" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.3.cmml">0.01</mn></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.SSS1.Px3.p1.2.m2.1b"><apply id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.cmml" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1"><lt id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.1.cmml" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.1"></lt><ci id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.2.cmml" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.2">𝑝</ci><cn type="float" id="S3.SS2.SSS1.Px3.p1.2.m2.1.1.3.cmml" xref="S3.SS2.SSS1.Px3.p1.2.m2.1.1.3">0.01</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.SSS1.Px3.p1.2.m2.1c">p&lt;0.01</annotation></semantics></math>) of these improvements. This verifies the effectiveness of decomposition in aiding humans to tackle complex tasks by breaking them down into more manageable subtasks.</p>
</div>
</section>
<section id="S3.SS2.SSS1.Px4" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Decomposition enables non-experts to be comparable with experts</h5>

<div id="S3.SS2.SSS1.Px4.p1" class="ltx_para">
<p id="S3.SS2.SSS1.Px4.p1.1" class="ltx_p">Remarkably, when assisted with our decomposition model, non-experts achieve performance comparable to non-assisted experts in labeling efficiency and quality. This aligns with the overarching goal of scalable oversight, empowering human labelers to oversee models in superhuman tasks.</p>
</div>
<figure id="S3.F5" class="ltx_figure"><img src="/html/2406.04604/assets/pics/human_group.png" id="S3.F5.g1" class="ltx_graphics ltx_centering ltx_img_square" width="449" height="461" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 5: </span>Decomposition brings more benefits to human labelers on hard problems.</figcaption>
</figure>
</section>
<section id="S3.SS2.SSS1.Px5" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Decomposition benefits more on hard problems</h5>

<div id="S3.SS2.SSS1.Px5.p1" class="ltx_para">
<p id="S3.SS2.SSS1.Px5.p1.1" class="ltx_p">To further understand the impact of decomposition, we extract two subsets from our test data based on the time spent by non-experts to repair the model-generated programs: (1) Easy: problems that take less than 25 minutes. (2) Hard: problems that take more than 40 minutes. The results shown in Figure <a href="#S3.F5" title="Figure 5 ‣ Decomposition enables non-experts to be comparable with experts ‣ 3.2.1 Quantitative Analysis ‣ 3.2 Assisting Human Supervision ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a> reveal that easy problems benefit minimally from decomposition. However, when it comes to hard problems, decomposition significantly improves the labeling efficiency and final quality of human labelers. These results further underscore our decomposition’s utility in complex tasks.</p>
</div>
</section>
<section id="S3.SS2.SSS1.Px6" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Humans prefer our decomposition model</h5>

<div id="S3.SS2.SSS1.Px6.p1" class="ltx_para">
<p id="S3.SS2.SSS1.Px6.p1.1" class="ltx_p">As shown in Figure <a href="#S3.F4" title="Figure 4 ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> and Figure <a href="#S3.F4" title="Figure 4 ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, our decomposition model surpasses other decomposition baselines in enhancing human labeling efficiency and quality. We also ask labelers to consider whether the decomposition is helpful for them to repair the program in the post-survey. As shown in Figure <a href="#S3.F6" title="Figure 6 ‣ 3.2.2 Qualitative Analysis ‣ 3.2 Assisting Human Supervision ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>, labelers rate our decomposition results substantially more helpful than the baselines’. These results verify that LMs can learn to produce high-<span id="S3.SS2.SSS1.Px6.p1.1.1" class="ltx_text ltx_font_typewriter">AssistV</span> decompositions by learning from human repair experiences.</p>
</div>
</section>
</section>
<section id="S3.SS2.SSS2" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">3.2.2 </span>Qualitative Analysis</h4>

<div id="S3.SS2.SSS2.p1" class="ltx_para">
<p id="S3.SS2.SSS2.p1.1" class="ltx_p">To further gain insight into what our decomposition model has learned from human feedback to enhance its utility, we analyze the collected human repairing traces alongside the corresponding critique, from which we draw the following qualitative observations.</p>
</div>
<figure id="S3.F6" class="ltx_figure"><img src="/html/2406.04604/assets/pics/helpfulness.png" id="S3.F6.g1" class="ltx_graphics ltx_centering ltx_img_square" width="419" height="503" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 6: </span>Helpfulness of decompositions evaluated by human labelers after repairing.</figcaption>
</figure>
<section id="S3.SS2.SSS2.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Assisting humans by highlighting boundary conditions</h5>

<div id="S3.SS2.SSS2.Px1.p1" class="ltx_para">
<p id="S3.SS2.SSS2.Px1.p1.4" class="ltx_p">We observe that decomposing complex programs around boundary conditions effectively aids humans. As boundary conditions typically involve more straightforward logic than the core problem, humans can more easily understand and repair them. For instance, in one training demonstration (Figure <a href="#A5.F13" title="Figure 13 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">13</span></a>),  <svg id="S3.SS2.SSS2.Px1.p1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="97.8"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(48.9,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 45.85 9.69 L -45.85 9.69 C -47.38 9.69 -48.62 8.45 -48.62 6.92 L -48.62 -1.4 C -48.62 -2.93 -47.38 -4.17 -45.85 -4.17 L 45.85 -4.17 C 47.38 -4.17 48.62 -2.93 48.62 -1.4 L 48.62 6.92 C 48.62 8.45 47.38 9.69 45.85 9.69 Z M -48.62 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -44.01 0)" fill="#000000" stroke="#000000"><foreignObject width="88.4" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px1.p1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">check_validity</span></foreignObject></g></g></svg> highlights all boundary conditions, earning positive human feedback for enabling them to quickly identify that problems are located in  <svg id="S3.SS2.SSS2.Px1.p1.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="177.44"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(88.72,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 85.67 9.69 L -85.67 9.69 C -87.2 9.69 -88.44 8.45 -88.44 6.92 L -88.44 -1.4 C -88.44 -2.93 -87.2 -4.17 -85.67 -4.17 L 85.67 -4.17 C 87.2 -4.17 88.44 -2.93 88.44 -1.4 L 88.44 6.92 C 88.44 8.45 87.2 9.69 85.67 9.69 Z M -88.44 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -83.83 0)" fill="#000000" stroke="#000000"><foreignObject width="168.43" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px1.p1.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">calculate_minimum_moves</span></foreignObject></g></g></svg>. This principle is learned by our decomposition model, as evidenced in Figure <a href="#A5.F14" title="Figure 14 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">14</span></a>, where it highlights two boundary conditions in  <svg id="S3.SS2.SSS2.Px1.p1.3.pic3" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="114.82"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(57.41,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 54.37 9.69 L -54.37 9.69 C -55.9 9.69 -57.13 8.45 -57.13 6.92 L -57.13 -1.4 C -57.13 -2.93 -55.9 -4.17 -54.37 -4.17 L 54.37 -4.17 C 55.9 -4.17 57.13 -2.93 57.13 -1.4 L 57.13 6.92 C 57.13 8.45 55.9 9.69 54.37 9.69 Z M -57.13 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -52.52 0)" fill="#000000" stroke="#000000"><foreignObject width="105.43" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px1.p1.3.pic3.1.1.1.1.1" class="ltx_text ltx_font_typewriter">check_zero_piles</span></foreignObject></g></g></svg> and  <svg id="S3.SS2.SSS2.Px1.p1.4.pic4" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="120.24"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(60.12,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 57.08 9.69 L -57.08 9.69 C -58.61 9.69 -59.84 8.45 -59.84 6.92 L -59.84 -1.4 C -59.84 -2.93 -58.61 -4.17 -57.08 -4.17 L 57.08 -4.17 C 58.61 -4.17 59.84 -2.93 59.84 -1.4 L 59.84 6.92 C 59.84 8.45 58.61 9.69 57.08 9.69 Z M -59.84 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -55.23 0)" fill="#000000" stroke="#000000"><foreignObject width="110.85" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px1.p1.4.pic4.1.1.1.1.1" class="ltx_text ltx_font_typewriter">check_same_piles</span></foreignObject></g></g></svg>, leading humans to quickly realize the missing third boundary condition.</p>
</div>
</section>
<section id="S3.SS2.SSS2.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Assisting humans by creating simpler subtasks</h5>

<div id="S3.SS2.SSS2.Px2.p1" class="ltx_para">
<p id="S3.SS2.SSS2.Px2.p1.4" class="ltx_p">Decomposition reduces human workload by breaking down the initial solution into multiple simpler pieces, each corresponding to a specific subtask. For instance, in one training demonstration (Figure <a href="#A5.F15" title="Figure 15 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">15</span></a>),  <svg id="S3.SS2.SSS2.Px2.p1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="73.2"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(36.6,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 33.55 9.69 L -33.55 9.69 C -35.08 9.69 -36.32 8.45 -36.32 6.92 L -36.32 -1.4 C -36.32 -2.93 -35.08 -4.17 -33.55 -4.17 L 33.55 -4.17 C 35.08 -4.17 36.32 -2.93 36.32 -1.4 L 36.32 6.92 C 36.32 8.45 35.08 9.69 33.55 9.69 Z M -36.32 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -31.71 0)" fill="#000000" stroke="#000000"><foreignObject width="63.42" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px2.p1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">find_cycle</span></foreignObject></g></g></svg> simplifies the complex  <svg id="S3.SS2.SSS2.Px2.p1.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="78.65"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(39.33,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 36.28 9.69 L -36.28 9.69 C -37.81 9.69 -39.05 8.45 -39.05 6.92 L -39.05 -1.4 C -39.05 -2.93 -37.81 -4.17 -36.28 -4.17 L 36.28 -4.17 C 37.81 -4.17 39.05 -2.93 39.05 -1.4 L 39.05 6.92 C 39.05 8.45 37.81 9.69 36.28 9.69 Z M -39.05 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -34.44 0)" fill="#000000" stroke="#000000"><foreignObject width="68.88" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px2.p1.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">find_cycles</span></foreignObject></g></g></svg>, and  <svg id="S3.SS2.SSS2.Px2.p1.3.pic3" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="145.53"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(72.77,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 69.72 9.69 L -69.72 9.69 C -71.25 9.69 -72.49 8.45 -72.49 6.92 L -72.49 -1.4 C -72.49 -2.93 -71.25 -4.17 -69.72 -4.17 L 69.72 -4.17 C 71.25 -4.17 72.49 -2.93 72.49 -1.4 L 72.49 6.92 C 72.49 8.45 71.25 9.69 69.72 9.69 Z M -72.49 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -67.88 0)" fill="#000000" stroke="#000000"><foreignObject width="135.37" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px2.p1.3.pic3.1.1.1.1.1" class="ltx_text ltx_font_typewriter">generate_permutation</span></foreignObject></g></g></svg> helps humans locate the actual bugs in a simple subtask. Similarly, in Figure <a href="#A5.F16" title="Figure 16 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">16</span></a>, our decomposition model creates a simple task  <svg id="S3.SS2.SSS2.Px2.p1.4.pic4" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="145.19"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(72.59,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 69.55 9.69 L -69.55 9.69 C -71.08 9.69 -72.32 8.45 -72.32 6.92 L -72.32 -1.4 C -72.32 -2.93 -71.08 -4.17 -69.55 -4.17 L 69.55 -4.17 C 71.08 -4.17 72.32 -2.93 72.32 -1.4 L 72.32 6.92 C 72.32 8.45 71.08 9.69 69.55 9.69 Z M -72.32 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -67.71 0)" fill="#000000" stroke="#000000"><foreignObject width="135.8" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px2.p1.4.pic4.1.1.1.1.1" class="ltx_text ltx_font_typewriter">calculate_participants</span></foreignObject></g></g></svg> which effectively isolates bugs. In addition, our decomposition model learns to decompose code pieces that humans typically struggle with (e.g., nested loops, binary search, and dynamic programming), as exemplified in Figure <a href="#A5.F15" title="Figure 15 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">15</span></a> and Figure <a href="#A5.F17" title="Figure 17 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">17</span></a>.</p>
</div>
</section>
<section id="S3.SS2.SSS2.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Assisting humans by presenting clear high-level logic</h5>

<div id="S3.SS2.SSS2.Px3.p1" class="ltx_para">
<p id="S3.SS2.SSS2.Px3.p1.2" class="ltx_p">Decomposition’s ability to offer clear high-level logic can accelerate comprehension and bug identification before delving into low-level details, as demonstrated in Figure <a href="#A5.F17" title="Figure 17 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">17</span></a>. Next, Figure <a href="#A5.F18" title="Figure 18 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">18</span></a> illustrates our decomposition model’s integration of this principle by creating two subtasks  <svg id="S3.SS2.SSS2.Px3.p1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="89.07"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(44.54,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 41.49 9.69 L -41.49 9.69 C -43.02 9.69 -44.26 8.45 -44.26 6.92 L -44.26 -1.4 C -44.26 -2.93 -43.02 -4.17 -41.49 -4.17 L 41.49 -4.17 C 43.02 -4.17 44.26 -2.93 44.26 -1.4 L 44.26 6.92 C 44.26 8.45 43.02 9.69 41.49 9.69 Z M -44.26 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -39.65 0)" fill="#000000" stroke="#000000"><foreignObject width="79.29" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px3.p1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">toggle_doors</span></foreignObject></g></g></svg> and  <svg id="S3.SS2.SSS2.Px3.p1.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="127.89"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(63.95,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 60.9 9.69 L -60.9 9.69 C -62.43 9.69 -63.67 8.45 -63.67 6.92 L -63.67 -1.4 C -63.67 -2.93 -62.43 -4.17 -60.9 -4.17 L 60.9 -4.17 C 62.43 -4.17 63.67 -2.93 63.67 -1.4 L 63.67 6.92 C 63.67 8.45 62.43 9.69 60.9 9.69 Z M -63.67 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -59.06 0)" fill="#000000" stroke="#000000"><foreignObject width="117.73" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S3.SS2.SSS2.Px3.p1.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">toggle_single_door</span></foreignObject></g></g></svg>. This high-level logic indicates that the current solution addresses each door independently, thereby enabling humans to locate bugs directly since doors are interrelated in the problem context.</p>
</div>
</section>
</section>
</section>
<section id="S3.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.3 </span>Assisting AI Supervision</h3>

<div id="S3.SS3.p1" class="ltx_para">
<p id="S3.SS3.p1.1" class="ltx_p">With the verified effectiveness of decomposition in assisting human supervision, we further explore its potential to assist AI supervision. We suspect that some patterns of decompositions shown in Section <a href="#S3.SS2.SSS2" title="3.2.2 Qualitative Analysis ‣ 3.2 Assisting Human Supervision ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.2.2</span></a> (e.g., creating simpler subtasks and presenting clear high-level logic) can also help AI to evaluate (i.e., discriminate programs’ correctness) and repair programs. We achieve AI supervision by prompting a code language model <math id="S3.SS3.p1.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="S3.SS3.p1.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS3.p1.1.m1.1.1" xref="S3.SS3.p1.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p1.1.m1.1b"><ci id="S3.SS3.p1.1.m1.1.1.cmml" xref="S3.SS3.p1.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p1.1.m1.1c">\mathcal{M}</annotation></semantics></math> to perform discrimination and repair <cite class="ltx_cite ltx_citemacro_cite">Saunders et al. (<a href="#bib.bib28" title="" class="ltx_ref">2022</a>); Madaan et al. (<a href="#bib.bib21" title="" class="ltx_ref">2023</a>); Chen et al. (<a href="#bib.bib7" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
<figure id="S3.T1" class="ltx_table">
<div id="S3.T1.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:293.5pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(57.2pt,-38.7pt) scale(1.35871708456546,1.35871708456546) ;">
<table id="S3.T1.1.1" class="ltx_tabular ltx_align_middle">
<tr id="S3.T1.1.1.1" class="ltx_tr">
<td id="S3.T1.1.1.1.1" class="ltx_td ltx_align_left ltx_border_tt" rowspan="2"><span id="S3.T1.1.1.1.1.1" class="ltx_text ltx_font_bold">Program</span></td>
<td id="S3.T1.1.1.1.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="S3.T1.1.1.1.2.1" class="ltx_text ltx_font_bold">Discrimination</span></td>
<td id="S3.T1.1.1.1.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="S3.T1.1.1.1.3.1" class="ltx_text ltx_font_bold">Repair</span></td>
</tr>
<tr id="S3.T1.1.1.2" class="ltx_tr">
<td id="S3.T1.1.1.2.1" class="ltx_td ltx_align_center"><span id="S3.T1.1.1.2.1.1" class="ltx_text ltx_font_bold">Acc</span></td>
<td id="S3.T1.1.1.2.2" class="ltx_td ltx_align_center"><span id="S3.T1.1.1.2.2.1" class="ltx_text ltx_font_bold">Acc (Strict)</span></td>
<td id="S3.T1.1.1.2.3" class="ltx_td ltx_align_center"><span id="S3.T1.1.1.2.3.1" class="ltx_text ltx_font_bold">Acc (Test Case)</span></td>
</tr>
<tr id="S3.T1.1.1.3" class="ltx_tr">
<td id="S3.T1.1.1.3.1" class="ltx_td ltx_align_center ltx_border_t" style="background-color:#F2F2F2;" colspan="4"><span id="S3.T1.1.1.3.1.1" class="ltx_text ltx_font_italic">APPS</span></td>
</tr>
<tr id="S3.T1.1.1.4" class="ltx_tr">
<td id="S3.T1.1.1.4.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.1.4.1.1" class="ltx_text ltx_font_bold">Initial</span></td>
<td id="S3.T1.1.1.4.2" class="ltx_td ltx_align_center ltx_border_t">10.2</td>
<td id="S3.T1.1.1.4.3" class="ltx_td ltx_align_center ltx_border_t">18.3 <span id="S3.T1.1.1.4.3.1" class="ltx_text" style="background-color:#FFFBCC;">(+0.0)</span>
</td>
<td id="S3.T1.1.1.4.4" class="ltx_td ltx_align_center ltx_border_t">41.5 <span id="S3.T1.1.1.4.4.1" class="ltx_text" style="background-color:#FBEBEB;">(-0.3)</span>
</td>
</tr>
<tr id="S3.T1.1.1.5" class="ltx_tr">
<td id="S3.T1.1.1.5.1" class="ltx_td ltx_align_left"><span id="S3.T1.1.1.5.1.1" class="ltx_text ltx_font_bold">Heuristic Decomp</span></td>
<td id="S3.T1.1.1.5.2" class="ltx_td ltx_align_center">30.0</td>
<td id="S3.T1.1.1.5.3" class="ltx_td ltx_align_center">16.7 <span id="S3.T1.1.1.5.3.1" class="ltx_text" style="background-color:#F4CCCC;">(-1.6)</span>
</td>
<td id="S3.T1.1.1.5.4" class="ltx_td ltx_align_center">39.2 <span id="S3.T1.1.1.5.4.1" class="ltx_text" style="background-color:#F4CCCC;">(-2.6)</span>
</td>
</tr>
<tr id="S3.T1.1.1.6" class="ltx_tr">
<td id="S3.T1.1.1.6.1" class="ltx_td ltx_align_left"><span id="S3.T1.1.1.6.1.1" class="ltx_text ltx_font_bold">Vanilla Decomp</span></td>
<td id="S3.T1.1.1.6.2" class="ltx_td ltx_align_center">32.7</td>
<td id="S3.T1.1.1.6.3" class="ltx_td ltx_align_center">19.2 <span id="S3.T1.1.1.6.3.1" class="ltx_text" style="background-color:#DFFFDF;">(+0.9)</span>
</td>
<td id="S3.T1.1.1.6.4" class="ltx_td ltx_align_center">42.7 <span id="S3.T1.1.1.6.4.1" class="ltx_text" style="background-color:#DFFFDF;">(+0.9)</span>
</td>
</tr>
<tr id="S3.T1.1.1.7" class="ltx_tr">
<td id="S3.T1.1.1.7.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.1.7.1.1" class="ltx_text ltx_font_bold">Assistive Decomp</span></td>
<td id="S3.T1.1.1.7.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S3.T1.1.1.7.2.1" class="ltx_text ltx_font_bold">42.9</span></td>
<td id="S3.T1.1.1.7.3" class="ltx_td ltx_align_center ltx_border_t">
<span id="S3.T1.1.1.7.3.1" class="ltx_text ltx_font_bold">21.7</span> <span id="S3.T1.1.1.7.3.2" class="ltx_text" style="background-color:#BFFFBE;">(<span id="S3.T1.1.1.7.3.2.1" class="ltx_text ltx_font_bold">+2.9</span>)</span>
</td>
<td id="S3.T1.1.1.7.4" class="ltx_td ltx_align_center ltx_border_t">
<span id="S3.T1.1.1.7.4.1" class="ltx_text ltx_font_bold">47.4</span> <span id="S3.T1.1.1.7.4.2" class="ltx_text" style="background-color:#BFFFBE;">(<span id="S3.T1.1.1.7.4.2.1" class="ltx_text ltx_font_bold">+5.6</span>)</span>
</td>
</tr>
<tr id="S3.T1.1.1.8" class="ltx_tr">
<td id="S3.T1.1.1.8.1" class="ltx_td ltx_align_center ltx_border_t" style="background-color:#F2F2F2;" colspan="4"><span id="S3.T1.1.1.8.1.1" class="ltx_text ltx_font_italic">Code-Contests</span></td>
</tr>
<tr id="S3.T1.1.1.9" class="ltx_tr">
<td id="S3.T1.1.1.9.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S3.T1.1.1.9.1.1" class="ltx_text ltx_font_bold">Initial</span></td>
<td id="S3.T1.1.1.9.2" class="ltx_td ltx_align_center ltx_border_t">26.7</td>
<td id="S3.T1.1.1.9.3" class="ltx_td ltx_align_center ltx_border_t">6.3 <span id="S3.T1.1.1.9.3.1" class="ltx_text" style="background-color:#FFFBCC;">(+0.0)</span>
</td>
<td id="S3.T1.1.1.9.4" class="ltx_td ltx_align_center ltx_border_t">20.1 <span id="S3.T1.1.1.9.4.1" class="ltx_text" style="background-color:#F2FFD2;">(+1.3)</span>
</td>
</tr>
<tr id="S3.T1.1.1.10" class="ltx_tr">
<td id="S3.T1.1.1.10.1" class="ltx_td ltx_align_left"><span id="S3.T1.1.1.10.1.1" class="ltx_text ltx_font_bold">Heuristic Decomp</span></td>
<td id="S3.T1.1.1.10.2" class="ltx_td ltx_align_center">46.7</td>
<td id="S3.T1.1.1.10.3" class="ltx_td ltx_align_center">7.3 <span id="S3.T1.1.1.10.3.1" class="ltx_text" style="background-color:#F2FFD2;">(+1.0)</span>
</td>
<td id="S3.T1.1.1.10.4" class="ltx_td ltx_align_center">19.3 <span id="S3.T1.1.1.10.4.1" class="ltx_text" style="background-color:#FFFBCC;">(+0.5)</span>
</td>
</tr>
<tr id="S3.T1.1.1.11" class="ltx_tr">
<td id="S3.T1.1.1.11.1" class="ltx_td ltx_align_left"><span id="S3.T1.1.1.11.1.1" class="ltx_text ltx_font_bold">Vanilla Decomp</span></td>
<td id="S3.T1.1.1.11.2" class="ltx_td ltx_align_center">53.1</td>
<td id="S3.T1.1.1.11.3" class="ltx_td ltx_align_center">8.3 <span id="S3.T1.1.1.11.3.1" class="ltx_text" style="background-color:#DFFFDF;">(+2.0)</span>
</td>
<td id="S3.T1.1.1.11.4" class="ltx_td ltx_align_center">21.4 <span id="S3.T1.1.1.11.4.1" class="ltx_text" style="background-color:#DFFFDF;">(+2.6)</span>
</td>
</tr>
<tr id="S3.T1.1.1.12" class="ltx_tr">
<td id="S3.T1.1.1.12.1" class="ltx_td ltx_align_left ltx_border_bb ltx_border_t"><span id="S3.T1.1.1.12.1.1" class="ltx_text ltx_font_bold">Assistive Decomp</span></td>
<td id="S3.T1.1.1.12.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><span id="S3.T1.1.1.12.2.1" class="ltx_text ltx_font_bold">62.5</span></td>
<td id="S3.T1.1.1.12.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="S3.T1.1.1.12.3.1" class="ltx_text ltx_font_bold">12.5</span> <span id="S3.T1.1.1.12.3.2" class="ltx_text" style="background-color:#BFFFBE;">(<span id="S3.T1.1.1.12.3.2.1" class="ltx_text ltx_font_bold">+6.2</span>)</span>
</td>
<td id="S3.T1.1.1.12.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="S3.T1.1.1.12.4.1" class="ltx_text ltx_font_bold">26.3</span> <span id="S3.T1.1.1.12.4.2" class="ltx_text" style="background-color:#BFFFBE;">(+<span id="S3.T1.1.1.12.4.2.1" class="ltx_text ltx_font_bold">7.5</span>)</span>
</td>
</tr>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 1: </span>Decomposition aids AI systems to discriminate and repair programs, improving discrimination accuracy and repaired solutions’ accuracy. Parenthetical values denote the accuracy variation from non-repair programs.</figcaption>
</figure>
<figure id="S3.F7" class="ltx_figure"><img src="/html/2406.04604/assets/pics/ai_weak_to_strong_cr.png" id="S3.F7.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="419" height="333" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 7: </span>Decomposition aids AI systems to provide self-supervision, where GPT-4 discriminates its own outputs, and weak-to-strong supervision, where GPT-3.5-turbo discriminates GPT-4-generated programs.</figcaption>
</figure>
<section id="S3.SS3.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Decomposition facilitates accurate self-supervision</h5>

<div id="S3.SS3.SSS0.Px1.p1" class="ltx_para">
<p id="S3.SS3.SSS0.Px1.p1.1" class="ltx_p">We first investigate decomposition’s impact on self-supervision, where we use GPT-4 to supervise itself. Results in Table <a href="#S3.T1" title="Table 1 ‣ 3.3 Assisting AI Supervision ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> reveal that GPT-4 struggles to discriminate and repair programs generated by itself, echoing recent findings in LLMs’ self-correction abilities <cite class="ltx_cite ltx_citemacro_cite">Huang et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023a</a>)</cite>. However, decomposing the complex programs with our model improves both discrimination accuracy and repaired solution quality by a large margin. For instance, discrimination accuracy improves from 10.2% to 42.9% on APPS and from 26.7% to 62.5% on Code-Contests.</p>
</div>
</section>
<section id="S3.SS3.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Decomposition facilitates accurate weak-to-strong supervision</h5>

<div id="S3.SS3.SSS0.Px2.p1" class="ltx_para">
<p id="S3.SS3.SSS0.Px2.p1.1" class="ltx_p">Having verified the effectiveness of decomposition in aiding non-expert human labelers, we now investigate its potential to assist weak models in supervising strong models. We evaluate GPT-3.5-turbo’s ability to discriminate the programs generated by GPT-4. The results in Figure <a href="#S3.F7" title="Figure 7 ‣ 3.3 Assisting AI Supervision ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> reveal that our method leads to more accurate weak-to-strong AI supervision. For instance, it improves GPT-3.5-turbo’s discrimination accuracy from 6.1% to 22.4% on APPS, surpassing the non-assisted accuracy of GPT-4 (10.2%).</p>
</div>
</section>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Analysis</h2>

<figure id="S4.T2" class="ltx_table">
<table id="S4.T2.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tr id="S4.T2.1.2" class="ltx_tr">
<td id="S4.T2.1.2.1" class="ltx_td ltx_align_left ltx_border_tt"><span id="S4.T2.1.2.1.1" class="ltx_text ltx_font_bold">Method</span></td>
<td id="S4.T2.1.2.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T2.1.2.2.1" class="ltx_text ltx_font_bold">Accuracy</span></td>
</tr>
<tr id="S4.T2.1.3" class="ltx_tr">
<td id="S4.T2.1.3.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S4.T2.1.3.1.1" class="ltx_text ltx_font_bold">Intuitive Human Preference</span></td>
<td id="S4.T2.1.3.2" class="ltx_td ltx_align_center ltx_border_t">49.5</td>
</tr>
<tr id="S4.T2.1.4" class="ltx_tr">
<td id="S4.T2.1.4.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S4.T2.1.4.1.1" class="ltx_text ltx_font_bold">Cyclomatic Complexity</span></td>
<td id="S4.T2.1.4.2" class="ltx_td ltx_align_center ltx_border_t">48.4</td>
</tr>
<tr id="S4.T2.1.5" class="ltx_tr">
<td id="S4.T2.1.5.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S4.T2.1.5.1.1" class="ltx_text ltx_font_bold">GPT-3.5-turbo Zero-shot</span></td>
<td id="S4.T2.1.5.2" class="ltx_td ltx_align_center ltx_border_t">54.3</td>
</tr>
<tr id="S4.T2.1.6" class="ltx_tr">
<td id="S4.T2.1.6.1" class="ltx_td ltx_align_left"><span id="S4.T2.1.6.1.1" class="ltx_text ltx_font_bold">GPT-3.5-turbo Few-shot</span></td>
<td id="S4.T2.1.6.2" class="ltx_td ltx_align_center">62.5</td>
</tr>
<tr id="S4.T2.1.7" class="ltx_tr">
<td id="S4.T2.1.7.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S4.T2.1.7.1.1" class="ltx_text ltx_font_bold">GPT-4 Zero-shot</span></td>
<td id="S4.T2.1.7.2" class="ltx_td ltx_align_center ltx_border_t">66.7</td>
</tr>
<tr id="S4.T2.1.1" class="ltx_tr">
<td id="S4.T2.1.1.1" class="ltx_td ltx_align_left ltx_border_bb"><span id="S4.T2.1.1.1.1" class="ltx_text ltx_font_bold">GPT-4 Few-shot (<math id="S4.T2.1.1.1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S4.T2.1.1.1.1.m1.1a"><msub id="S4.T2.1.1.1.1.m1.1.1" xref="S4.T2.1.1.1.1.m1.1.1.cmml"><mi id="S4.T2.1.1.1.1.m1.1.1.2" xref="S4.T2.1.1.1.1.m1.1.1.2.cmml">π</mi><mtext class="ltx_mathvariant_bold" id="S4.T2.1.1.1.1.m1.1.1.3" xref="S4.T2.1.1.1.1.m1.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.T2.1.1.1.1.m1.1b"><apply id="S4.T2.1.1.1.1.m1.1.1.cmml" xref="S4.T2.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.T2.1.1.1.1.m1.1.1.1.cmml" xref="S4.T2.1.1.1.1.m1.1.1">subscript</csymbol><ci id="S4.T2.1.1.1.1.m1.1.1.2.cmml" xref="S4.T2.1.1.1.1.m1.1.1.2">𝜋</ci><ci id="S4.T2.1.1.1.1.m1.1.1.3a.cmml" xref="S4.T2.1.1.1.1.m1.1.1.3"><mtext class="ltx_mathvariant_bold" mathsize="70%" id="S4.T2.1.1.1.1.m1.1.1.3.cmml" xref="S4.T2.1.1.1.1.m1.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.1.1.1.1.m1.1c">\pi_{\text{rank}}</annotation></semantics></math>)</span></td>
<td id="S4.T2.1.1.2" class="ltx_td ltx_align_center ltx_border_bb"><span id="S4.T2.1.1.2.1" class="ltx_text ltx_font_bold">78.1</span></td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 2: </span>Accuracy of the rank model in predicting the assistive value of decompositions in actual human repair processes.</figcaption>
</figure>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Validity of the Rank Model</h3>

<div id="S4.SS1.p1" class="ltx_para">
<p id="S4.SS1.p1.1" class="ltx_p">We evaluate the effectiveness of our ranking model <math id="S4.SS1.p1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S4.SS1.p1.1.m1.1a"><msub id="S4.SS1.p1.1.m1.1.1" xref="S4.SS1.p1.1.m1.1.1.cmml"><mi id="S4.SS1.p1.1.m1.1.1.2" xref="S4.SS1.p1.1.m1.1.1.2.cmml">π</mi><mtext id="S4.SS1.p1.1.m1.1.1.3" xref="S4.SS1.p1.1.m1.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.p1.1.m1.1b"><apply id="S4.SS1.p1.1.m1.1.1.cmml" xref="S4.SS1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS1.p1.1.m1.1.1.1.cmml" xref="S4.SS1.p1.1.m1.1.1">subscript</csymbol><ci id="S4.SS1.p1.1.m1.1.1.2.cmml" xref="S4.SS1.p1.1.m1.1.1.2">𝜋</ci><ci id="S4.SS1.p1.1.m1.1.1.3a.cmml" xref="S4.SS1.p1.1.m1.1.1.3"><mtext mathsize="70%" id="S4.SS1.p1.1.m1.1.1.3.cmml" xref="S4.SS1.p1.1.m1.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p1.1.m1.1c">\pi_{\text{rank}}</annotation></semantics></math> in predicting the assistive value of decompositions. We construct paired decompositions annotated with real assistive value labels as the test data. The results shown in Table <a href="#S4.T2" title="Table 2 ‣ 4 Analysis ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> indicate that LLMs can learn to select higher-<span id="S4.SS1.p1.1.1" class="ltx_text ltx_font_typewriter">AssistV</span> decomposition for assisting humans via in-context learning, surpassing prior heuristic or zero-shot baselines. Notably, we ask humans to provide preferences on paired decompositions without repairing them, which is shown to align poorly with their actual assistive value for human repair. These results highlight the importance of measuring assistive value based on actual human supervision experiences.</p>
</div>
<div id="S4.SS1.p2" class="ltx_para">
<p id="S4.SS1.p2.1" class="ltx_p">See Appendix <a href="#A2.SS4" title="B.4 Analysis of the Rank Model ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B.4</span></a> for more details about these ranking baselines.</p>
</div>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Ablations on the Critic Model</h3>

<div id="S4.SS2.p1" class="ltx_para">
<p id="S4.SS2.p1.3" class="ltx_p">We perform an ablation to explore the impact of our critic model <math id="S4.SS2.p1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{critic}}" display="inline"><semantics id="S4.SS2.p1.1.m1.1a"><msub id="S4.SS2.p1.1.m1.1.1" xref="S4.SS2.p1.1.m1.1.1.cmml"><mi id="S4.SS2.p1.1.m1.1.1.2" xref="S4.SS2.p1.1.m1.1.1.2.cmml">π</mi><mtext id="S4.SS2.p1.1.m1.1.1.3" xref="S4.SS2.p1.1.m1.1.1.3a.cmml">critic</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.p1.1.m1.1b"><apply id="S4.SS2.p1.1.m1.1.1.cmml" xref="S4.SS2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS2.p1.1.m1.1.1.1.cmml" xref="S4.SS2.p1.1.m1.1.1">subscript</csymbol><ci id="S4.SS2.p1.1.m1.1.1.2.cmml" xref="S4.SS2.p1.1.m1.1.1.2">𝜋</ci><ci id="S4.SS2.p1.1.m1.1.1.3a.cmml" xref="S4.SS2.p1.1.m1.1.1.3"><mtext mathsize="70%" id="S4.SS2.p1.1.m1.1.1.3.cmml" xref="S4.SS2.p1.1.m1.1.1.3">critic</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p1.1.m1.1c">\pi_{\text{critic}}</annotation></semantics></math>. In the ablated version, we directly generate refined decompositions without incorporating predicted critiques. We use <math id="S4.SS2.p1.2.m2.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="S4.SS2.p1.2.m2.1a"><msub id="S4.SS2.p1.2.m2.1.1" xref="S4.SS2.p1.2.m2.1.1.cmml"><mi id="S4.SS2.p1.2.m2.1.1.2" xref="S4.SS2.p1.2.m2.1.1.2.cmml">π</mi><mtext id="S4.SS2.p1.2.m2.1.1.3" xref="S4.SS2.p1.2.m2.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.p1.2.m2.1b"><apply id="S4.SS2.p1.2.m2.1.1.cmml" xref="S4.SS2.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS2.p1.2.m2.1.1.1.cmml" xref="S4.SS2.p1.2.m2.1.1">subscript</csymbol><ci id="S4.SS2.p1.2.m2.1.1.2.cmml" xref="S4.SS2.p1.2.m2.1.1.2">𝜋</ci><ci id="S4.SS2.p1.2.m2.1.1.3a.cmml" xref="S4.SS2.p1.2.m2.1.1.3"><mtext mathsize="70%" id="S4.SS2.p1.2.m2.1.1.3.cmml" xref="S4.SS2.p1.2.m2.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p1.2.m2.1c">\pi_{\text{rank}}</annotation></semantics></math> as a proxy to evaluate the assistive value of decompositions due to its reasonable performance (Section <a href="#S4.SS1" title="4.1 Validity of the Rank Model ‣ 4 Analysis ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1</span></a>). The win rates between models with and without <math id="S4.SS2.p1.3.m3.1" class="ltx_Math" alttext="\pi_{\text{critic}}" display="inline"><semantics id="S4.SS2.p1.3.m3.1a"><msub id="S4.SS2.p1.3.m3.1.1" xref="S4.SS2.p1.3.m3.1.1.cmml"><mi id="S4.SS2.p1.3.m3.1.1.2" xref="S4.SS2.p1.3.m3.1.1.2.cmml">π</mi><mtext id="S4.SS2.p1.3.m3.1.1.3" xref="S4.SS2.p1.3.m3.1.1.3a.cmml">critic</mtext></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.p1.3.m3.1b"><apply id="S4.SS2.p1.3.m3.1.1.cmml" xref="S4.SS2.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS2.p1.3.m3.1.1.1.cmml" xref="S4.SS2.p1.3.m3.1.1">subscript</csymbol><ci id="S4.SS2.p1.3.m3.1.1.2.cmml" xref="S4.SS2.p1.3.m3.1.1.2">𝜋</ci><ci id="S4.SS2.p1.3.m3.1.1.3a.cmml" xref="S4.SS2.p1.3.m3.1.1.3"><mtext mathsize="70%" id="S4.SS2.p1.3.m3.1.1.3.cmml" xref="S4.SS2.p1.3.m3.1.1.3">critic</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p1.3.m3.1c">\pi_{\text{critic}}</annotation></semantics></math> are 58.8% and 35.3%, respectively. These results indicate the effectiveness of learning from informative natural language human feedback.</p>
</div>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>Distilled Decomposition Model</h3>

<div id="S4.SS3.p1" class="ltx_para">
<p id="S4.SS3.p1.4" class="ltx_p">In this section, we aim to study whether moderate-size open-source LLMs can also learn to generate better decompositions to assist human or AI supervision while reducing API costs and enhancing reproducibility. We hence distill the knowledge of assistive decomposition from proprietary LLMs into an in-house model <math id="S4.SS3.p1.1.m1.1" class="ltx_Math" alttext="\pi_{\theta}" display="inline"><semantics id="S4.SS3.p1.1.m1.1a"><msub id="S4.SS3.p1.1.m1.1.1" xref="S4.SS3.p1.1.m1.1.1.cmml"><mi id="S4.SS3.p1.1.m1.1.1.2" xref="S4.SS3.p1.1.m1.1.1.2.cmml">π</mi><mi id="S4.SS3.p1.1.m1.1.1.3" xref="S4.SS3.p1.1.m1.1.1.3.cmml">θ</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.1.m1.1b"><apply id="S4.SS3.p1.1.m1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.1.m1.1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1">subscript</csymbol><ci id="S4.SS3.p1.1.m1.1.1.2.cmml" xref="S4.SS3.p1.1.m1.1.1.2">𝜋</ci><ci id="S4.SS3.p1.1.m1.1.1.3.cmml" xref="S4.SS3.p1.1.m1.1.1.3">𝜃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.1.m1.1c">\pi_{\theta}</annotation></semantics></math>. Specifically, we create supervised data <math id="S4.SS3.p1.2.m2.3" class="ltx_Math" alttext="D=\{(P,A,A_{d}^{*})\}" display="inline"><semantics id="S4.SS3.p1.2.m2.3a"><mrow id="S4.SS3.p1.2.m2.3.3" xref="S4.SS3.p1.2.m2.3.3.cmml"><mi id="S4.SS3.p1.2.m2.3.3.3" xref="S4.SS3.p1.2.m2.3.3.3.cmml">D</mi><mo id="S4.SS3.p1.2.m2.3.3.2" xref="S4.SS3.p1.2.m2.3.3.2.cmml">=</mo><mrow id="S4.SS3.p1.2.m2.3.3.1.1" xref="S4.SS3.p1.2.m2.3.3.1.2.cmml"><mo stretchy="false" id="S4.SS3.p1.2.m2.3.3.1.1.2" xref="S4.SS3.p1.2.m2.3.3.1.2.cmml">{</mo><mrow id="S4.SS3.p1.2.m2.3.3.1.1.1.1" xref="S4.SS3.p1.2.m2.3.3.1.1.1.2.cmml"><mo stretchy="false" id="S4.SS3.p1.2.m2.3.3.1.1.1.1.2" xref="S4.SS3.p1.2.m2.3.3.1.1.1.2.cmml">(</mo><mi id="S4.SS3.p1.2.m2.1.1" xref="S4.SS3.p1.2.m2.1.1.cmml">P</mi><mo id="S4.SS3.p1.2.m2.3.3.1.1.1.1.3" xref="S4.SS3.p1.2.m2.3.3.1.1.1.2.cmml">,</mo><mi id="S4.SS3.p1.2.m2.2.2" xref="S4.SS3.p1.2.m2.2.2.cmml">A</mi><mo id="S4.SS3.p1.2.m2.3.3.1.1.1.1.4" xref="S4.SS3.p1.2.m2.3.3.1.1.1.2.cmml">,</mo><msubsup id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.cmml"><mi id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.2" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.2.cmml">A</mi><mi id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.3" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.3.cmml">d</mi><mo id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.3" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.3.cmml">∗</mo></msubsup><mo stretchy="false" id="S4.SS3.p1.2.m2.3.3.1.1.1.1.5" xref="S4.SS3.p1.2.m2.3.3.1.1.1.2.cmml">)</mo></mrow><mo stretchy="false" id="S4.SS3.p1.2.m2.3.3.1.1.3" xref="S4.SS3.p1.2.m2.3.3.1.2.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.2.m2.3b"><apply id="S4.SS3.p1.2.m2.3.3.cmml" xref="S4.SS3.p1.2.m2.3.3"><eq id="S4.SS3.p1.2.m2.3.3.2.cmml" xref="S4.SS3.p1.2.m2.3.3.2"></eq><ci id="S4.SS3.p1.2.m2.3.3.3.cmml" xref="S4.SS3.p1.2.m2.3.3.3">𝐷</ci><set id="S4.SS3.p1.2.m2.3.3.1.2.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1"><vector id="S4.SS3.p1.2.m2.3.3.1.1.1.2.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1"><ci id="S4.SS3.p1.2.m2.1.1.cmml" xref="S4.SS3.p1.2.m2.1.1">𝑃</ci><ci id="S4.SS3.p1.2.m2.2.2.cmml" xref="S4.SS3.p1.2.m2.2.2">𝐴</ci><apply id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.1.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1">superscript</csymbol><apply id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.1.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1">subscript</csymbol><ci id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.2.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.2">𝐴</ci><ci id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.3.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.2.3">𝑑</ci></apply><times id="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.3.cmml" xref="S4.SS3.p1.2.m2.3.3.1.1.1.1.1.3"></times></apply></vector></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.2.m2.3c">D=\{(P,A,A_{d}^{*})\}</annotation></semantics></math> with LLMs as illustrated in Section <a href="#S2.SS3" title="2.3 Models ‣ 2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.3</span></a>, and optimizing <math id="S4.SS3.p1.3.m3.1" class="ltx_Math" alttext="\pi_{\theta}" display="inline"><semantics id="S4.SS3.p1.3.m3.1a"><msub id="S4.SS3.p1.3.m3.1.1" xref="S4.SS3.p1.3.m3.1.1.cmml"><mi id="S4.SS3.p1.3.m3.1.1.2" xref="S4.SS3.p1.3.m3.1.1.2.cmml">π</mi><mi id="S4.SS3.p1.3.m3.1.1.3" xref="S4.SS3.p1.3.m3.1.1.3.cmml">θ</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.3.m3.1b"><apply id="S4.SS3.p1.3.m3.1.1.cmml" xref="S4.SS3.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.3.m3.1.1.1.cmml" xref="S4.SS3.p1.3.m3.1.1">subscript</csymbol><ci id="S4.SS3.p1.3.m3.1.1.2.cmml" xref="S4.SS3.p1.3.m3.1.1.2">𝜋</ci><ci id="S4.SS3.p1.3.m3.1.1.3.cmml" xref="S4.SS3.p1.3.m3.1.1.3">𝜃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.3.m3.1c">\pi_{\theta}</annotation></semantics></math> over <math id="S4.SS3.p1.4.m4.1" class="ltx_Math" alttext="D" display="inline"><semantics id="S4.SS3.p1.4.m4.1a"><mi id="S4.SS3.p1.4.m4.1.1" xref="S4.SS3.p1.4.m4.1.1.cmml">D</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.4.m4.1b"><ci id="S4.SS3.p1.4.m4.1.1.cmml" xref="S4.SS3.p1.4.m4.1.1">𝐷</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.4.m4.1c">D</annotation></semantics></math> via the standard MLE loss:</p>
</div>
<div id="S4.SS3.p2" class="ltx_para">
<table id="S4.Ex1" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S4.Ex1.m1.4" class="ltx_Math" alttext="\mathcal{L}_{MLE}=-\sum_{t=1}^{|A_{d}^{*}|}\text{log}\pi_{\theta}(A^{*}_{d_{t}}|A^{*}_{d_{&lt;t}},A,P)" display="block"><semantics id="S4.Ex1.m1.4a"><mrow id="S4.Ex1.m1.4.4" xref="S4.Ex1.m1.4.4.cmml"><msub id="S4.Ex1.m1.4.4.3" xref="S4.Ex1.m1.4.4.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S4.Ex1.m1.4.4.3.2" xref="S4.Ex1.m1.4.4.3.2.cmml">ℒ</mi><mrow id="S4.Ex1.m1.4.4.3.3" xref="S4.Ex1.m1.4.4.3.3.cmml"><mi id="S4.Ex1.m1.4.4.3.3.2" xref="S4.Ex1.m1.4.4.3.3.2.cmml">M</mi><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.4.4.3.3.1" xref="S4.Ex1.m1.4.4.3.3.1.cmml">​</mo><mi id="S4.Ex1.m1.4.4.3.3.3" xref="S4.Ex1.m1.4.4.3.3.3.cmml">L</mi><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.4.4.3.3.1a" xref="S4.Ex1.m1.4.4.3.3.1.cmml">​</mo><mi id="S4.Ex1.m1.4.4.3.3.4" xref="S4.Ex1.m1.4.4.3.3.4.cmml">E</mi></mrow></msub><mo id="S4.Ex1.m1.4.4.2" xref="S4.Ex1.m1.4.4.2.cmml">=</mo><mrow id="S4.Ex1.m1.4.4.1" xref="S4.Ex1.m1.4.4.1.cmml"><mo id="S4.Ex1.m1.4.4.1a" xref="S4.Ex1.m1.4.4.1.cmml">−</mo><mrow id="S4.Ex1.m1.4.4.1.1" xref="S4.Ex1.m1.4.4.1.1.cmml"><munderover id="S4.Ex1.m1.4.4.1.1.2" xref="S4.Ex1.m1.4.4.1.1.2.cmml"><mo movablelimits="false" id="S4.Ex1.m1.4.4.1.1.2.2.2" xref="S4.Ex1.m1.4.4.1.1.2.2.2.cmml">∑</mo><mrow id="S4.Ex1.m1.4.4.1.1.2.2.3" xref="S4.Ex1.m1.4.4.1.1.2.2.3.cmml"><mi id="S4.Ex1.m1.4.4.1.1.2.2.3.2" xref="S4.Ex1.m1.4.4.1.1.2.2.3.2.cmml">t</mi><mo id="S4.Ex1.m1.4.4.1.1.2.2.3.1" xref="S4.Ex1.m1.4.4.1.1.2.2.3.1.cmml">=</mo><mn id="S4.Ex1.m1.4.4.1.1.2.2.3.3" xref="S4.Ex1.m1.4.4.1.1.2.2.3.3.cmml">1</mn></mrow><mrow id="S4.Ex1.m1.1.1.1.1" xref="S4.Ex1.m1.1.1.1.2.cmml"><mo stretchy="false" id="S4.Ex1.m1.1.1.1.1.2" xref="S4.Ex1.m1.1.1.1.2.1.cmml">|</mo><msubsup id="S4.Ex1.m1.1.1.1.1.1" xref="S4.Ex1.m1.1.1.1.1.1.cmml"><mi id="S4.Ex1.m1.1.1.1.1.1.2.2" xref="S4.Ex1.m1.1.1.1.1.1.2.2.cmml">A</mi><mi id="S4.Ex1.m1.1.1.1.1.1.2.3" xref="S4.Ex1.m1.1.1.1.1.1.2.3.cmml">d</mi><mo id="S4.Ex1.m1.1.1.1.1.1.3" xref="S4.Ex1.m1.1.1.1.1.1.3.cmml">∗</mo></msubsup><mo stretchy="false" id="S4.Ex1.m1.1.1.1.1.3" xref="S4.Ex1.m1.1.1.1.2.1.cmml">|</mo></mrow></munderover><mrow id="S4.Ex1.m1.4.4.1.1.1" xref="S4.Ex1.m1.4.4.1.1.1.cmml"><mtext id="S4.Ex1.m1.4.4.1.1.1.3" xref="S4.Ex1.m1.4.4.1.1.1.3a.cmml">log</mtext><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.4.4.1.1.1.2" xref="S4.Ex1.m1.4.4.1.1.1.2.cmml">​</mo><msub id="S4.Ex1.m1.4.4.1.1.1.4" xref="S4.Ex1.m1.4.4.1.1.1.4.cmml"><mi id="S4.Ex1.m1.4.4.1.1.1.4.2" xref="S4.Ex1.m1.4.4.1.1.1.4.2.cmml">π</mi><mi id="S4.Ex1.m1.4.4.1.1.1.4.3" xref="S4.Ex1.m1.4.4.1.1.1.4.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S4.Ex1.m1.4.4.1.1.1.2a" xref="S4.Ex1.m1.4.4.1.1.1.2.cmml">​</mo><mrow id="S4.Ex1.m1.4.4.1.1.1.1.1" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S4.Ex1.m1.4.4.1.1.1.1.1.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.cmml">(</mo><mrow id="S4.Ex1.m1.4.4.1.1.1.1.1.1" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.cmml"><msubsup id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.cmml"><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.2.cmml">A</mi><msub id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.cmml"><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.2.cmml">d</mi><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.3.cmml">t</mi></msub><mo id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.3.cmml">∗</mo></msubsup><mo fence="false" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.2.cmml">|</mo><mrow id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.2.cmml"><msubsup id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.cmml"><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.2.cmml">A</mi><msub id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.cmml"><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.2.cmml">d</mi><mrow id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.cmml"><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.2.cmml"></mi><mo id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.1" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.1.cmml">&lt;</mo><mi id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.3.cmml">t</mi></mrow></msub><mo id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.3.cmml">∗</mo></msubsup><mo id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.2" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.2.cmml">,</mo><mi id="S4.Ex1.m1.2.2" xref="S4.Ex1.m1.2.2.cmml">A</mi><mo id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.2.cmml">,</mo><mi id="S4.Ex1.m1.3.3" xref="S4.Ex1.m1.3.3.cmml">P</mi></mrow></mrow><mo stretchy="false" id="S4.Ex1.m1.4.4.1.1.1.1.1.3" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.Ex1.m1.4b"><apply id="S4.Ex1.m1.4.4.cmml" xref="S4.Ex1.m1.4.4"><eq id="S4.Ex1.m1.4.4.2.cmml" xref="S4.Ex1.m1.4.4.2"></eq><apply id="S4.Ex1.m1.4.4.3.cmml" xref="S4.Ex1.m1.4.4.3"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.3.1.cmml" xref="S4.Ex1.m1.4.4.3">subscript</csymbol><ci id="S4.Ex1.m1.4.4.3.2.cmml" xref="S4.Ex1.m1.4.4.3.2">ℒ</ci><apply id="S4.Ex1.m1.4.4.3.3.cmml" xref="S4.Ex1.m1.4.4.3.3"><times id="S4.Ex1.m1.4.4.3.3.1.cmml" xref="S4.Ex1.m1.4.4.3.3.1"></times><ci id="S4.Ex1.m1.4.4.3.3.2.cmml" xref="S4.Ex1.m1.4.4.3.3.2">𝑀</ci><ci id="S4.Ex1.m1.4.4.3.3.3.cmml" xref="S4.Ex1.m1.4.4.3.3.3">𝐿</ci><ci id="S4.Ex1.m1.4.4.3.3.4.cmml" xref="S4.Ex1.m1.4.4.3.3.4">𝐸</ci></apply></apply><apply id="S4.Ex1.m1.4.4.1.cmml" xref="S4.Ex1.m1.4.4.1"><minus id="S4.Ex1.m1.4.4.1.2.cmml" xref="S4.Ex1.m1.4.4.1"></minus><apply id="S4.Ex1.m1.4.4.1.1.cmml" xref="S4.Ex1.m1.4.4.1.1"><apply id="S4.Ex1.m1.4.4.1.1.2.cmml" xref="S4.Ex1.m1.4.4.1.1.2"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.2.1.cmml" xref="S4.Ex1.m1.4.4.1.1.2">superscript</csymbol><apply id="S4.Ex1.m1.4.4.1.1.2.2.cmml" xref="S4.Ex1.m1.4.4.1.1.2"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.2.2.1.cmml" xref="S4.Ex1.m1.4.4.1.1.2">subscript</csymbol><sum id="S4.Ex1.m1.4.4.1.1.2.2.2.cmml" xref="S4.Ex1.m1.4.4.1.1.2.2.2"></sum><apply id="S4.Ex1.m1.4.4.1.1.2.2.3.cmml" xref="S4.Ex1.m1.4.4.1.1.2.2.3"><eq id="S4.Ex1.m1.4.4.1.1.2.2.3.1.cmml" xref="S4.Ex1.m1.4.4.1.1.2.2.3.1"></eq><ci id="S4.Ex1.m1.4.4.1.1.2.2.3.2.cmml" xref="S4.Ex1.m1.4.4.1.1.2.2.3.2">𝑡</ci><cn type="integer" id="S4.Ex1.m1.4.4.1.1.2.2.3.3.cmml" xref="S4.Ex1.m1.4.4.1.1.2.2.3.3">1</cn></apply></apply><apply id="S4.Ex1.m1.1.1.1.2.cmml" xref="S4.Ex1.m1.1.1.1.1"><abs id="S4.Ex1.m1.1.1.1.2.1.cmml" xref="S4.Ex1.m1.1.1.1.1.2"></abs><apply id="S4.Ex1.m1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.1.1.1.1.1">superscript</csymbol><apply id="S4.Ex1.m1.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.1.1.1.1.1.2.1.cmml" xref="S4.Ex1.m1.1.1.1.1.1">subscript</csymbol><ci id="S4.Ex1.m1.1.1.1.1.1.2.2.cmml" xref="S4.Ex1.m1.1.1.1.1.1.2.2">𝐴</ci><ci id="S4.Ex1.m1.1.1.1.1.1.2.3.cmml" xref="S4.Ex1.m1.1.1.1.1.1.2.3">𝑑</ci></apply><times id="S4.Ex1.m1.1.1.1.1.1.3.cmml" xref="S4.Ex1.m1.1.1.1.1.1.3"></times></apply></apply></apply><apply id="S4.Ex1.m1.4.4.1.1.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1"><times id="S4.Ex1.m1.4.4.1.1.1.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.2"></times><ci id="S4.Ex1.m1.4.4.1.1.1.3a.cmml" xref="S4.Ex1.m1.4.4.1.1.1.3"><mtext id="S4.Ex1.m1.4.4.1.1.1.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.3">log</mtext></ci><apply id="S4.Ex1.m1.4.4.1.1.1.4.cmml" xref="S4.Ex1.m1.4.4.1.1.1.4"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.4.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.4">subscript</csymbol><ci id="S4.Ex1.m1.4.4.1.1.1.4.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.4.2">𝜋</ci><ci id="S4.Ex1.m1.4.4.1.1.1.4.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.4.3">𝜃</ci></apply><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1"><csymbol cd="latexml" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.2">conditional</csymbol><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3">subscript</csymbol><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3">superscript</csymbol><ci id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.2">𝐴</ci><times id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.2.3"></times></apply><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3">subscript</csymbol><ci id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.2">𝑑</ci><ci id="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.3.3.3">𝑡</ci></apply></apply><list id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1"><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1">subscript</csymbol><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1">superscript</csymbol><ci id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.2">𝐴</ci><times id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.2.3"></times></apply><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3">subscript</csymbol><ci id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.2">𝑑</ci><apply id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3"><lt id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.1.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.1"></lt><csymbol cd="latexml" id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.2.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.2">absent</csymbol><ci id="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.3.cmml" xref="S4.Ex1.m1.4.4.1.1.1.1.1.1.1.1.1.3.3.3">𝑡</ci></apply></apply></apply><ci id="S4.Ex1.m1.2.2.cmml" xref="S4.Ex1.m1.2.2">𝐴</ci><ci id="S4.Ex1.m1.3.3.cmml" xref="S4.Ex1.m1.3.3">𝑃</ci></list></apply></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Ex1.m1.4c">\mathcal{L}_{MLE}=-\sum_{t=1}^{|A_{d}^{*}|}\text{log}\pi_{\theta}(A^{*}_{d_{t}}|A^{*}_{d_{&lt;t}},A,P)</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S4.SS3.p2.3" class="ltx_p">where <math id="S4.SS3.p2.1.m1.1" class="ltx_Math" alttext="P" display="inline"><semantics id="S4.SS3.p2.1.m1.1a"><mi id="S4.SS3.p2.1.m1.1.1" xref="S4.SS3.p2.1.m1.1.1.cmml">P</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.1.m1.1b"><ci id="S4.SS3.p2.1.m1.1.1.cmml" xref="S4.SS3.p2.1.m1.1.1">𝑃</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.1.m1.1c">P</annotation></semantics></math>, <math id="S4.SS3.p2.2.m2.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S4.SS3.p2.2.m2.1a"><mi id="S4.SS3.p2.2.m2.1.1" xref="S4.SS3.p2.2.m2.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.2.m2.1b"><ci id="S4.SS3.p2.2.m2.1.1.cmml" xref="S4.SS3.p2.2.m2.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.2.m2.1c">A</annotation></semantics></math>, and <math id="S4.SS3.p2.3.m3.1" class="ltx_Math" alttext="A_{d}^{*}" display="inline"><semantics id="S4.SS3.p2.3.m3.1a"><msubsup id="S4.SS3.p2.3.m3.1.1" xref="S4.SS3.p2.3.m3.1.1.cmml"><mi id="S4.SS3.p2.3.m3.1.1.2.2" xref="S4.SS3.p2.3.m3.1.1.2.2.cmml">A</mi><mi id="S4.SS3.p2.3.m3.1.1.2.3" xref="S4.SS3.p2.3.m3.1.1.2.3.cmml">d</mi><mo id="S4.SS3.p2.3.m3.1.1.3" xref="S4.SS3.p2.3.m3.1.1.3.cmml">∗</mo></msubsup><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.3.m3.1b"><apply id="S4.SS3.p2.3.m3.1.1.cmml" xref="S4.SS3.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS3.p2.3.m3.1.1.1.cmml" xref="S4.SS3.p2.3.m3.1.1">superscript</csymbol><apply id="S4.SS3.p2.3.m3.1.1.2.cmml" xref="S4.SS3.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS3.p2.3.m3.1.1.2.1.cmml" xref="S4.SS3.p2.3.m3.1.1">subscript</csymbol><ci id="S4.SS3.p2.3.m3.1.1.2.2.cmml" xref="S4.SS3.p2.3.m3.1.1.2.2">𝐴</ci><ci id="S4.SS3.p2.3.m3.1.1.2.3.cmml" xref="S4.SS3.p2.3.m3.1.1.2.3">𝑑</ci></apply><times id="S4.SS3.p2.3.m3.1.1.3.cmml" xref="S4.SS3.p2.3.m3.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.3.m3.1c">A_{d}^{*}</annotation></semantics></math> denote the problem, the initial model-generated solution, and the decomposed solution, respectively.</p>
</div>
<div id="S4.SS3.p3" class="ltx_para">
<p id="S4.SS3.p3.1" class="ltx_p">We evaluate the performance of the Code-LLaMA-based <cite class="ltx_cite ltx_citemacro_cite">Roziere et al. (<a href="#bib.bib27" title="" class="ltx_ref">2023</a>)</cite> distilled decomposition model in aiding human and AI supervision, finding it outperforms the vanilla decomposition based on Code-LLaMA and even GPT-4. These results reveal the effectiveness of distilling the knowledge of assistive decomposition from GPT-4 to Code-LLaMA. See Appendix <a href="#A2.SS5" title="B.5 Analysis of the Distilled Decomposition Model ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B.5</span></a> for detailed results.</p>
</div>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Related Work</h2>

<section id="S5.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.1 </span>Code Generation</h3>

<div id="S5.SS1.p1" class="ltx_para">
<p id="S5.SS1.p1.1" class="ltx_p">Language models have achieved impressive performance in generating simple code pieces <cite class="ltx_cite ltx_citemacro_cite">Chen et al. (<a href="#bib.bib6" title="" class="ltx_ref">2021</a>); Austin et al. (<a href="#bib.bib2" title="" class="ltx_ref">2021</a>)</cite> by learning from human demonstrations <cite class="ltx_cite ltx_citemacro_cite">Li et al. (<a href="#bib.bib19" title="" class="ltx_ref">2023</a>)</cite>, and are being used to generate increasingly complex programs <cite class="ltx_cite ltx_citemacro_cite">Hendrycks et al. (<a href="#bib.bib13" title="" class="ltx_ref">2021</a>); Li et al. (<a href="#bib.bib20" title="" class="ltx_ref">2022</a>)</cite>. In this context, our paper studies how to aid humans in efficiently providing reliable supervision to train models that don’t write buggy programs in complex and high-impact scenarios.</p>
</div>
</section>
<section id="S5.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.2 </span>AI-assisted Programming</h3>

<div id="S5.SS2.p1" class="ltx_para">
<p id="S5.SS2.p1.1" class="ltx_p">Prior works have explored various forms to assist human programmers, including code completion <cite class="ltx_cite ltx_citemacro_cite">Github (<a href="#bib.bib12" title="" class="ltx_ref">2021</a>)</cite>, bug location <cite class="ltx_cite ltx_citemacro_cite">Xie et al. (<a href="#bib.bib34" title="" class="ltx_ref">2016</a>)</cite>, and program repair <cite class="ltx_cite ltx_citemacro_cite">Joshi et al. (<a href="#bib.bib16" title="" class="ltx_ref">2023</a>)</cite>. However, these methods rely on assistant models’ capacity for accurate code generation or repair—a requirement unmet in our setup, where models face challenges in performing either task. We thus explore an alternative form of assistance via task decomposition, which alleviates the burden on assistant models.</p>
</div>
</section>
<section id="S5.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.3 </span>Task Decomposition</h3>

<div id="S5.SS3.p1" class="ltx_para">
<p id="S5.SS3.p1.1" class="ltx_p">Task decomposition has been extensively studied for tackling complex tasks, primarily focusing on enhancing model performance <cite class="ltx_cite ltx_citemacro_cite">Khot et al. (<a href="#bib.bib17" title="" class="ltx_ref">2022</a>); Dua et al. (<a href="#bib.bib11" title="" class="ltx_ref">2022</a>); Yao et al. (<a href="#bib.bib35" title="" class="ltx_ref">2022</a>)</cite> and enabling efficient searching over the subtask space <cite class="ltx_cite ltx_citemacro_cite">Zelikman et al. (<a href="#bib.bib36" title="" class="ltx_ref">2023</a>)</cite>. In contrast, our paper studies decomposition from a human-centric perspective, aiming to facilitate human supervision over complex tasks, akin to <cite class="ltx_cite ltx_citemacro_citet">Wu et al. (<a href="#bib.bib33" title="" class="ltx_ref">2021</a>)</cite>’s work in aiding human supervision over book summarization. Concerning the implementation of decomposition, prior works mainly rely on fixed heuristic rules (e.g., decomposing each <math id="S5.SS3.p1.1.m1.1" class="ltx_Math" alttext="N" display="inline"><semantics id="S5.SS3.p1.1.m1.1a"><mi id="S5.SS3.p1.1.m1.1.1" xref="S5.SS3.p1.1.m1.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="S5.SS3.p1.1.m1.1b"><ci id="S5.SS3.p1.1.m1.1.1.cmml" xref="S5.SS3.p1.1.m1.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p1.1.m1.1c">N</annotation></semantics></math>-line of codes) or author-crafted demonstrations <cite class="ltx_cite ltx_citemacro_cite">Wu et al. (<a href="#bib.bib33" title="" class="ltx_ref">2021</a>); Khot et al. (<a href="#bib.bib17" title="" class="ltx_ref">2022</a>); Zelikman et al. (<a href="#bib.bib36" title="" class="ltx_ref">2023</a>)</cite>. In this paper, we take the first step towards advanced decomposition by learning to improve a novel objective: assistive value, which exactly measures the feasibility and speed of human repair in the actual annotation process.</p>
</div>
</section>
<section id="S5.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.4 </span>Learning from Human Feedback</h3>

<div id="S5.SS4.p1" class="ltx_para">
<p id="S5.SS4.p1.1" class="ltx_p">Learning from human feedback has been widely adopted in recent works for developing human-friendly general language models <cite class="ltx_cite ltx_citemacro_cite">Ouyang et al. (<a href="#bib.bib24" title="" class="ltx_ref">2022</a>)</cite>. Our work can also be understood as introducing this concept to the development of assistant models. Specifically, our introduced objective—assistive value–could also be introduced as a unique type of human feedback, which is gathered from actual human problem-solving experiences. Therefore, the collected human feedback could be more aligned with the ultimate goal of AI assistants: enhancing human problem-solving performance.</p>
</div>
</section>
<section id="S5.SS5" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.5 </span>Scalable Oversight</h3>

<div id="S5.SS5.p1" class="ltx_para">
<p id="S5.SS5.p1.1" class="ltx_p">Advancements in LLMs have intensified the need for scalable, reliable human oversight on complex tasks that reach or even exceed the capabilities of human experts. Addressing this, prior research has explored various assistance methods, such as self-critiquing <cite class="ltx_cite ltx_citemacro_cite">Saunders et al. (<a href="#bib.bib28" title="" class="ltx_ref">2022</a>)</cite>, AI debate <cite class="ltx_cite ltx_citemacro_cite">Parrish et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>)</cite>, and decomposition <cite class="ltx_cite ltx_citemacro_cite">Christiano et al. (<a href="#bib.bib8" title="" class="ltx_ref">2018</a>); Wu et al. (<a href="#bib.bib33" title="" class="ltx_ref">2021</a>)</cite>. However, previous works overlook the feedback from actual assisted humans, potentially leading to reduced helpfulness of the assistance to humans <cite class="ltx_cite ltx_citemacro_cite">Xie et al. (<a href="#bib.bib34" title="" class="ltx_ref">2016</a>); Parrish et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>)</cite>.
To bridge this gap, we propose to learn assistance models from human feedback.</p>
</div>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Conclusion</h2>

<div id="S6.p1" class="ltx_para">
<p id="S6.p1.1" class="ltx_p">This paper focuses on assisting humans in supervising LMs on complex problems by automated task decomposition. We introduce a novel objective for learning task decomposition: assistive value (<span id="S6.p1.1.1" class="ltx_text ltx_font_typewriter">AssistV</span>), which measures the feasibility and speed of humans to repair a decomposed solution. We collect a dataset of decompositions, measure their <span id="S6.p1.1.2" class="ltx_text ltx_font_typewriter">AssistV</span>, and ask human annotators to provide a natural language critique on what makes a decomposition (not) helpful. We then learn to critique, refine, and rank decompositions to generate high-<span id="S6.p1.1.3" class="ltx_text ltx_font_typewriter">AssistV</span> decompositions. Experiment results demonstrate that our method can effectively assist humans in providing higher-quality supervision with significantly less time. Notably, our method assists non-experts in matching unassisted experts. Overall, we show that even when LMs cannot solve the problem themselves, they can still learn to assist humans by learning from human repair experiences. These results highlight the potential of learning-based methods to assist humans with scalable oversight.</p>
</div>
</section>
<section id="Sx1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Limitations and Future Work</h2>

<div id="Sx1.p1" class="ltx_para">
<p id="Sx1.p1.1" class="ltx_p">One limitation of our work is that our non-expert participants, who could solve 30% of the problems while spending 74 minutes, are still more skilled than novice programmers (e.g., programmers who can only solve Leetcode easy-level problems) or non-programmers. Nonetheless, we demonstrate the effectiveness of our decomposition model, which successfully enables assisted non-experts to be comparable with non-assisted experts. We leave the investigation of leveraging decomposition to assist novice programmers or even non-programmers as future work.</p>
</div>
<div id="Sx1.p2" class="ltx_para">
<p id="Sx1.p2.1" class="ltx_p">Another limitation of our paper is that we did not conduct experiments on misleading benchmarks, which mainly consist of subtle errors that humans tend to overlook. The effectiveness of decomposition in aiding humans in detecting and fixing these subtle errors is worth studying in future work.</p>
</div>
<div id="Sx1.p3" class="ltx_para">
<p id="Sx1.p3.1" class="ltx_p">Moreover, we only conduct experiments based on in-context learning and supervised learning. Given the reasonable performance of the rank model (Section <a href="#S4.SS1" title="4.1 Validity of the Rank Model ‣ 4 Analysis ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1</span></a>), fine-tuning the decomposition model with reinforcement learning is worth studying in the future.</p>
</div>
<div id="Sx1.p4" class="ltx_para">
<p id="Sx1.p4.1" class="ltx_p">Finally, our introduced objective <span id="Sx1.p4.1.1" class="ltx_text ltx_font_typewriter">AssistV</span>, which highlights the possibility of learning an effective assistance model from human problem-solving experiences, can be further extended to other assistance forms beyond decomposition. For example, recent works explore using model-generated explanations to assist humans in complex decision-making tasks (e.g., complex question answering <cite class="ltx_cite ltx_citemacro_cite">Rein et al. (<a href="#bib.bib26" title="" class="ltx_ref">2023</a>)</cite>). However, the practical benefits of model explanations are mixed as they may introduce additional human workload for understanding the explanation or even mislead humans to incorrect results <cite class="ltx_cite ltx_citemacro_cite">Parrish et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>)</cite>. Therefore, future works can explore what kind of explanations can better assist humans by measuring the <span id="Sx1.p4.1.2" class="ltx_text ltx_font_typewriter">AssistV</span> of different explanations and learning to improve <span id="Sx1.p4.1.3" class="ltx_text ltx_font_typewriter">AssistV</span>.</p>
</div>
</section>
<section id="Sx2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Ethics Statement</h2>

<div id="Sx2.p1" class="ltx_para">
<p id="Sx2.p1.1" class="ltx_p">We recruit annotators mainly from college students. We inform annotators in advance how their annotation data will be collected and used. We pay them 30 USD per hour, which is higher than the average wage of the local residents.</p>
</div>
</section>
<section id="Sx3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Acknowledgements</h2>

<div id="Sx3.p1" class="ltx_para">
<p id="Sx3.p1.1" class="ltx_p">This work was supported by the National Science Foundation for Distinguished Young Scholars (with No. 62125604) and the NSFC projects (with No. 62306160). This work was also supported by China National Postdoctoral Program for Innovative Talents (No. BX20230194) and China Postdoctoral Science Foundation (No. 2023M731952).</p>
</div>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Amodei et al. (2016)</span>
<span class="ltx_bibblock">
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016.

</span>
<span class="ltx_bibblock">Concrete problems in ai safety.

</span>
<span class="ltx_bibblock"><em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1606.06565</em>.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Austin et al. (2021)</span>
<span class="ltx_bibblock">
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021.

</span>
<span class="ltx_bibblock">Program synthesis with large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2108.07732</em>.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bai et al. (2022)</span>
<span class="ltx_bibblock">
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022.

</span>
<span class="ltx_bibblock">Constitutional ai: Harmlessness from ai feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2212.08073</em>.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bradley (1997)</span>
<span class="ltx_bibblock">
Andrew P Bradley. 1997.

</span>
<span class="ltx_bibblock">The use of the area under the roc curve in the evaluation of machine learning algorithms.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">Pattern recognition</em>, 30(7):1145–1159.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Charitsis et al. (2023)</span>
<span class="ltx_bibblock">
Charis Charitsis, Chris Piech, and John C Mitchell. 2023.

</span>
<span class="ltx_bibblock">Detecting the reasons for program decomposition in cs1 and evaluating their impact.

</span>
<span class="ltx_bibblock">In <em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1</em>, pages 1014–1020.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2021)</span>
<span class="ltx_bibblock">
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021.

</span>
<span class="ltx_bibblock">Evaluating large language models trained on code.

</span>
<span class="ltx_bibblock"><em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2107.03374</em>.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2023)</span>
<span class="ltx_bibblock">
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023.

</span>
<span class="ltx_bibblock">Teaching large language models to self-debug.

</span>
<span class="ltx_bibblock"><em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2304.05128</em>.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Christiano et al. (2018)</span>
<span class="ltx_bibblock">
Paul Christiano, Buck Shlegeris, and Dario Amodei. 2018.

</span>
<span class="ltx_bibblock">Supervising strong learners by amplifying weak experts.

</span>
<span class="ltx_bibblock"><em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1810.08575</em>.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Connolly and Dean (1997)</span>
<span class="ltx_bibblock">
Terry Connolly and Doug Dean. 1997.

</span>
<span class="ltx_bibblock">Decomposed versus holistic estimates of effort required for software writing tasks.

</span>
<span class="ltx_bibblock"><em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">Management Science</em>, 43(7):1029–1045.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Correa et al. (2023)</span>
<span class="ltx_bibblock">
Carlos G Correa, Mark K Ho, Frederick Callaway, Nathaniel D Daw, and Thomas L Griffiths. 2023.

</span>
<span class="ltx_bibblock">Humans decompose tasks by trading off utility and computational cost.

</span>
<span class="ltx_bibblock"><em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">PLOS Computational Biology</em>, 19(6):e1011087.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dua et al. (2022)</span>
<span class="ltx_bibblock">
Dheeru Dua, Shivanshu Gupta, Sameer Singh, and Matt Gardner. 2022.

</span>
<span class="ltx_bibblock">Successive prompting for decomposing complex questions.

</span>
<span class="ltx_bibblock"><em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2212.04092</em>.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Github (2021)</span>
<span class="ltx_bibblock">
Github. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/" title="" class="ltx_ref ltx_href">Introducing github copilot</a>.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hendrycks et al. (2021)</span>
<span class="ltx_bibblock">
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. 2021.

</span>
<span class="ltx_bibblock">Measuring coding challenge competence with apps.

</span>
<span class="ltx_bibblock"><em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2105.09938</em>.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Huang et al. (2023a)</span>
<span class="ltx_bibblock">
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023a.

</span>
<span class="ltx_bibblock">Large language models cannot self-correct reasoning yet.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.01798</em>.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Huang et al. (2023b)</span>
<span class="ltx_bibblock">
Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2023b.

</span>
<span class="ltx_bibblock">Benchmarking large language models as ai research agents.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.03302</em>.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Joshi et al. (2023)</span>
<span class="ltx_bibblock">
Harshit Joshi, José Cambronero Sanchez, Sumit Gulwani, Vu Le, Gust Verbruggen, and Ivan Radiček. 2023.

</span>
<span class="ltx_bibblock">Repair is nearly generation: Multilingual program repair with llms.

</span>
<span class="ltx_bibblock">In <em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">Proceedings of the AAAI Conference on Artificial Intelligence</em>, volume 37, pages 5131–5140.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Khot et al. (2022)</span>
<span class="ltx_bibblock">
Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022.

</span>
<span class="ltx_bibblock">Decomposed prompting: A modular approach for solving complex tasks.

</span>
<span class="ltx_bibblock"><em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2210.02406</em>.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lee and Anderson (2001)</span>
<span class="ltx_bibblock">
Frank J Lee and John R Anderson. 2001.

</span>
<span class="ltx_bibblock">Does learning a complex task have to be complex?: A study in learning decomposition.

</span>
<span class="ltx_bibblock"><em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">Cognitive psychology</em>, 42(3):267–316.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023)</span>
<span class="ltx_bibblock">
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023.

</span>
<span class="ltx_bibblock">Starcoder: may the source be with you!

</span>
<span class="ltx_bibblock"><em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.06161</em>.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2022)</span>
<span class="ltx_bibblock">
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022.

</span>
<span class="ltx_bibblock">Competition-level code generation with alphacode.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">Science</em>, 378(6624):1092–1097.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Madaan et al. (2023)</span>
<span class="ltx_bibblock">
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023.

</span>
<span class="ltx_bibblock">Self-refine: Iterative refinement with self-feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2303.17651</em>.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">McCabe (1976)</span>
<span class="ltx_bibblock">
Thomas J McCabe. 1976.

</span>
<span class="ltx_bibblock">A complexity measure.

</span>
<span class="ltx_bibblock"><em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">IEEE Transactions on software Engineering</em>, (4):308–320.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI (2023)</span>
<span class="ltx_bibblock">
OpenAI. 2023.

</span>
<span class="ltx_bibblock">GPT-4 technical report.

</span>
<span class="ltx_bibblock"><em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2303.08774</em>.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ouyang et al. (2022)</span>
<span class="ltx_bibblock">
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022.

</span>
<span class="ltx_bibblock">Training language models to follow instructions with human feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 35:27730–27744.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Parrish et al. (2022)</span>
<span class="ltx_bibblock">
Alicia Parrish, Harsh Trivedi, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Amanpreet Singh Saimbhi, and Samuel R Bowman. 2022.

</span>
<span class="ltx_bibblock">Two-turn debate doesn’t help humans answer hard reading comprehension questions.

</span>
<span class="ltx_bibblock"><em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2210.10860</em>.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Rein et al. (2023)</span>
<span class="ltx_bibblock">
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023.

</span>
<span class="ltx_bibblock">Gpqa: A graduate-level google-proof q&amp;a benchmark.

</span>
<span class="ltx_bibblock"><em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2311.12022</em>.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Roziere et al. (2023)</span>
<span class="ltx_bibblock">
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023.

</span>
<span class="ltx_bibblock">Code llama: Open foundation models for code.

</span>
<span class="ltx_bibblock"><em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2308.12950</em>.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Saunders et al. (2022)</span>
<span class="ltx_bibblock">
William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022.

</span>
<span class="ltx_bibblock">Self-critiquing models for assisting human evaluators.

</span>
<span class="ltx_bibblock"><em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2206.05802</em>.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Selby (2015)</span>
<span class="ltx_bibblock">
Cynthia C Selby. 2015.

</span>
<span class="ltx_bibblock">Relationships: computational thinking, pedagogy of programming, and bloom’s taxonomy.

</span>
<span class="ltx_bibblock">In <em id="bib.bib29.1.1" class="ltx_emph ltx_font_italic">Proceedings of the workshop in primary and secondary computing education</em>, pages 80–87.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sun et al. (2023)</span>
<span class="ltx_bibblock">
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. 2023.

</span>
<span class="ltx_bibblock">Principle-driven self-alignment of language models from scratch with minimal human supervision.

</span>
<span class="ltx_bibblock"><em id="bib.bib30.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.03047</em>.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Trinh et al. (2024)</span>
<span class="ltx_bibblock">
Trieu H Trinh, Yuhuai Wu, Quoc V Le, He He, and Thang Luong. 2024.

</span>
<span class="ltx_bibblock">Solving olympiad geometry without human demonstrations.

</span>
<span class="ltx_bibblock"><em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">Nature</em>, 625(7995):476–482.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Vaithilingam et al. (2022)</span>
<span class="ltx_bibblock">
Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. 2022.

</span>
<span class="ltx_bibblock">Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib32.1.1" class="ltx_emph ltx_font_italic">Chi conference on human factors in computing systems extended abstracts</em>, pages 1–7.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wu et al. (2021)</span>
<span class="ltx_bibblock">
Jeff Wu, Long Ouyang, Daniel M Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano. 2021.

</span>
<span class="ltx_bibblock">Recursively summarizing books with human feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib33.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2109.10862</em>.

</span>
</li>
<li id="bib.bib34" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xie et al. (2016)</span>
<span class="ltx_bibblock">
Xiaoyuan Xie, Zicong Liu, Shuo Song, Zhenyu Chen, Jifeng Xuan, and Baowen Xu. 2016.

</span>
<span class="ltx_bibblock">Revisit of automatic debugging via human focus-tracking analysis.

</span>
<span class="ltx_bibblock">In <em id="bib.bib34.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 38th International Conference on Software Engineering</em>, pages 808–819.

</span>
</li>
<li id="bib.bib35" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yao et al. (2022)</span>
<span class="ltx_bibblock">
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022.

</span>
<span class="ltx_bibblock">React: Synergizing reasoning and acting in language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib35.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2210.03629</em>.

</span>
</li>
<li id="bib.bib36" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zelikman et al. (2023)</span>
<span class="ltx_bibblock">
Eric Zelikman, Qian Huang, Gabriel Poesia, Noah Goodman, and Nick Haber. 2023.

</span>
<span class="ltx_bibblock">Parsel: Algorithmic reasoning with language models by composing decompositions.

</span>
<span class="ltx_bibblock">In <em id="bib.bib36.1.1" class="ltx_emph ltx_font_italic">Thirty-seventh Conference on Neural Information Processing Systems</em>.

</span>
</li>
</ul>
</section>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Implementation Details</h2>

<section id="A1.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.1 </span>Consistency of Decomposition</h3>

<div id="A1.SS1.p1" class="ltx_para">
<p id="A1.SS1.p1.1" class="ltx_p">To ensure consistency between initial and decomposed solutions, we set the max retry time <math id="A1.SS1.p1.1.m1.1" class="ltx_Math" alttext="N=8" display="inline"><semantics id="A1.SS1.p1.1.m1.1a"><mrow id="A1.SS1.p1.1.m1.1.1" xref="A1.SS1.p1.1.m1.1.1.cmml"><mi id="A1.SS1.p1.1.m1.1.1.2" xref="A1.SS1.p1.1.m1.1.1.2.cmml">N</mi><mo id="A1.SS1.p1.1.m1.1.1.1" xref="A1.SS1.p1.1.m1.1.1.1.cmml">=</mo><mn id="A1.SS1.p1.1.m1.1.1.3" xref="A1.SS1.p1.1.m1.1.1.3.cmml">8</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS1.p1.1.m1.1b"><apply id="A1.SS1.p1.1.m1.1.1.cmml" xref="A1.SS1.p1.1.m1.1.1"><eq id="A1.SS1.p1.1.m1.1.1.1.cmml" xref="A1.SS1.p1.1.m1.1.1.1"></eq><ci id="A1.SS1.p1.1.m1.1.1.2.cmml" xref="A1.SS1.p1.1.m1.1.1.2">𝑁</ci><cn type="integer" id="A1.SS1.p1.1.m1.1.1.3.cmml" xref="A1.SS1.p1.1.m1.1.1.3">8</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p1.1.m1.1c">N=8</annotation></semantics></math>. If the consistency check fails in the end, we directly use the initial solution as the final output. Empirically, we find the consistency ratio of the decomposition generated by few-shot prompting with Code-LLaMA-13B, GPT-3.5-turbo, and GPT-4 is 92.9%, 97.6%, and 98.8%, respectively.</p>
</div>
</section>
<section id="A1.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.2 </span>Data Collection</h3>

<section id="A1.SS2.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Data Preparation</h5>

<div id="A1.SS2.SSS0.Px1.p1" class="ltx_para">
<p id="A1.SS2.SSS0.Px1.p1.1" class="ltx_p">We use vanilla decomposition to sample <math id="A1.SS2.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="K=5" display="inline"><semantics id="A1.SS2.SSS0.Px1.p1.1.m1.1a"><mrow id="A1.SS2.SSS0.Px1.p1.1.m1.1.1" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.cmml"><mi id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.2" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.2.cmml">K</mi><mo id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.1" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.1.cmml">=</mo><mn id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.3" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.3.cmml">5</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.SSS0.Px1.p1.1.m1.1b"><apply id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.cmml" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1"><eq id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.1.cmml" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.1"></eq><ci id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.2.cmml" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.2">𝐾</ci><cn type="integer" id="A1.SS2.SSS0.Px1.p1.1.m1.1.1.3.cmml" xref="A1.SS2.SSS0.Px1.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.SSS0.Px1.p1.1.m1.1c">K=5</annotation></semantics></math> different decomposed solutions from various code generation models, including Code-LLaMA-13B, GPT-3.5-turbo, and GPT-4. We use top-p sampling with a default temperature 0.5.</p>
</div>
</section>
<section id="A1.SS2.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Constructing Pair-wise Demonstrations</h5>

<div id="A1.SS2.SSS0.Px2.p1" class="ltx_para">
<p id="A1.SS2.SSS0.Px2.p1.1" class="ltx_p">For constructing pair-wise decompositions, since we adopt a few-shot learning setup in our experiments, we manually inspect the collected human critiques and perform matching. And we finally collect five pair-wise decompositions as the training data for in-context learning.</p>
</div>
<div id="A1.SS2.SSS0.Px2.p2" class="ltx_para">
<p id="A1.SS2.SSS0.Px2.p2.4" class="ltx_p">Considering that matching critiques based on semantic similarity is not challenging for modern LMs, we further explore automatically matching decompositions based on language models. Specifically, given two decomposed solutions with the corresponding human critiques: <math id="A1.SS2.SSS0.Px2.p2.1.m1.2" class="ltx_Math" alttext="(A_{d}^{1},C^{1})" display="inline"><semantics id="A1.SS2.SSS0.Px2.p2.1.m1.2a"><mrow id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.3.cmml"><mo stretchy="false" id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.3" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.3.cmml">(</mo><msubsup id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.cmml"><mi id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.2" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.2.cmml">A</mi><mi id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.3" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.3.cmml">d</mi><mn id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.3" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.3.cmml">1</mn></msubsup><mo id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.4" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.3.cmml">,</mo><msup id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.cmml"><mi id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.2" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.2.cmml">C</mi><mn id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.3" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.3.cmml">1</mn></msup><mo stretchy="false" id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.5" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.3.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.SSS0.Px2.p2.1.m1.2b"><interval closure="open" id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.3.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2"><apply id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1">superscript</csymbol><apply id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.1.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1">subscript</csymbol><ci id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.2.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.2">𝐴</ci><ci id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.3.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.2.3">𝑑</ci></apply><cn type="integer" id="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.3.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.1.1.1.1.3">1</cn></apply><apply id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.1.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2">superscript</csymbol><ci id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.2.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.2">𝐶</ci><cn type="integer" id="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.3.cmml" xref="A1.SS2.SSS0.Px2.p2.1.m1.2.2.2.2.3">1</cn></apply></interval></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.SSS0.Px2.p2.1.m1.2c">(A_{d}^{1},C^{1})</annotation></semantics></math>, <math id="A1.SS2.SSS0.Px2.p2.2.m2.2" class="ltx_Math" alttext="(A_{d}^{2},C^{2})" display="inline"><semantics id="A1.SS2.SSS0.Px2.p2.2.m2.2a"><mrow id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.3.cmml"><mo stretchy="false" id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.3" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.3.cmml">(</mo><msubsup id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.cmml"><mi id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.2" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.2.cmml">A</mi><mi id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.3" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.3.cmml">d</mi><mn id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.3" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.3.cmml">2</mn></msubsup><mo id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.4" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.3.cmml">,</mo><msup id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.cmml"><mi id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.2" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.2.cmml">C</mi><mn id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.3" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.3.cmml">2</mn></msup><mo stretchy="false" id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.5" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.3.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="A1.SS2.SSS0.Px2.p2.2.m2.2b"><interval closure="open" id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.3.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2"><apply id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1">superscript</csymbol><apply id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.1.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1">subscript</csymbol><ci id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.2.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.2">𝐴</ci><ci id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.3.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.2.3">𝑑</ci></apply><cn type="integer" id="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.3.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.1.1.1.1.3">2</cn></apply><apply id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.1.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2">superscript</csymbol><ci id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.2.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.2">𝐶</ci><cn type="integer" id="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.3.cmml" xref="A1.SS2.SSS0.Px2.p2.2.m2.2.2.2.2.3">2</cn></apply></interval></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.SSS0.Px2.p2.2.m2.2c">(A_{d}^{2},C^{2})</annotation></semantics></math>, we prompt GPT-3.5-turbo to evaluate if <math id="A1.SS2.SSS0.Px2.p2.3.m3.1" class="ltx_Math" alttext="C^{1}" display="inline"><semantics id="A1.SS2.SSS0.Px2.p2.3.m3.1a"><msup id="A1.SS2.SSS0.Px2.p2.3.m3.1.1" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1.cmml"><mi id="A1.SS2.SSS0.Px2.p2.3.m3.1.1.2" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1.2.cmml">C</mi><mn id="A1.SS2.SSS0.Px2.p2.3.m3.1.1.3" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS2.SSS0.Px2.p2.3.m3.1b"><apply id="A1.SS2.SSS0.Px2.p2.3.m3.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.3.m3.1.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1">superscript</csymbol><ci id="A1.SS2.SSS0.Px2.p2.3.m3.1.1.2.cmml" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1.2">𝐶</ci><cn type="integer" id="A1.SS2.SSS0.Px2.p2.3.m3.1.1.3.cmml" xref="A1.SS2.SSS0.Px2.p2.3.m3.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.SSS0.Px2.p2.3.m3.1c">C^{1}</annotation></semantics></math> matches <math id="A1.SS2.SSS0.Px2.p2.4.m4.1" class="ltx_Math" alttext="C^{2}" display="inline"><semantics id="A1.SS2.SSS0.Px2.p2.4.m4.1a"><msup id="A1.SS2.SSS0.Px2.p2.4.m4.1.1" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1.cmml"><mi id="A1.SS2.SSS0.Px2.p2.4.m4.1.1.2" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1.2.cmml">C</mi><mn id="A1.SS2.SSS0.Px2.p2.4.m4.1.1.3" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1.3.cmml">2</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS2.SSS0.Px2.p2.4.m4.1b"><apply id="A1.SS2.SSS0.Px2.p2.4.m4.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1"><csymbol cd="ambiguous" id="A1.SS2.SSS0.Px2.p2.4.m4.1.1.1.cmml" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1">superscript</csymbol><ci id="A1.SS2.SSS0.Px2.p2.4.m4.1.1.2.cmml" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1.2">𝐶</ci><cn type="integer" id="A1.SS2.SSS0.Px2.p2.4.m4.1.1.3.cmml" xref="A1.SS2.SSS0.Px2.p2.4.m4.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS2.SSS0.Px2.p2.4.m4.1c">C^{2}</annotation></semantics></math>. We construct a test set that contains 50 decomposition pairs for evaluating the automatic matching performance, where the golden labels are derived from manual matching results. We find that GPT-3.5-turbo achieves a perfect accuracy of 100%. These results indicate that it is promising to construct pair-wise demonstrations automatically.</p>
</div>
</section>
</section>
<section id="A1.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.3 </span>Benchmark</h3>

<div id="A1.SS3.p1" class="ltx_para">
<p id="A1.SS3.p1.1" class="ltx_p">We obtain competition-level problems from Code-Contests and APPS. We use the whole test set of Code-Contests that consists of 96 problems and randomly sample 120 problems from the competition-level subset of APPs. When conducting human experiments, we first filter those problems where the generated solution directly passes all the hidden test cases and randomly sample 30 problems as the test data.</p>
</div>
</section>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Additional Results</h2>

<section id="A2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">B.1 </span>Comparing Human Propose with Human Repair</h3>

<figure id="A2.F8" class="ltx_figure"><img src="/html/2406.04604/assets/pics/scratch.png" id="A2.F8.g1" class="ltx_graphics ltx_centering ltx_img_square" width="538" height="536" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 8: </span>Comparing the performance of human labelers when asking them to write programs from scratch or repairing model-generated programs.</figcaption>
</figure>
<div id="A2.SS1.p1" class="ltx_para">
<p id="A2.SS1.p1.1" class="ltx_p">We adopt a propose-and-repair pipeline to obtain labels for code generation. Instead of burdening humans with creating programs from scratch, we leverage the coding capabilities of modern LLMs to propose initial programs and then ask humans to repair them. This pipeline is motivated by the recognition that modern LLMs can effectively solve a considerable portion of test cases, making them a practical starting point for reducing the human workload and streamlining the data collection process <cite class="ltx_cite ltx_citemacro_cite">Vaithilingam et al. (<a href="#bib.bib32" title="" class="ltx_ref">2022</a>)</cite>.</p>
</div>
<div id="A2.SS1.p2" class="ltx_para">
<p id="A2.SS1.p2.1" class="ltx_p">We also empirically verify the effectiveness of this pipeline by comparing it with a baseline where human labelers are required to write programs from scratch. We construct two test sets for competitive programming, consisting of 10 and 18 problems that code generation models (e.g., GPT-4 in our experiments) succeed or fail to solve, respectively. We then study human labelers’ efficiency on these two test sets. For problems that models can directly solve, non-expert humans also achieve 100% accuracy while still spending on average 24.8 minutes. For problems that models fail to solve, the results are shown in Figure <a href="#A2.F8" title="Figure 8 ‣ B.1 Comparing Human Propose with Human Repair ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>. We can see that human labelers can also more easily collect a high-quality program label based on an initial model-generated program, yielding higher labeling quality and efficiency.</p>
</div>
</section>
<section id="A2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">B.2 </span>Directly Generating Decomposed Solutions</h3>

<div id="A2.SS2.p1" class="ltx_para">
<p id="A2.SS2.p1.1" class="ltx_p">In our experiments, we follow a two-stage framework to first generate an initial solution and then decompose it. We choose not to generate decomposed solutions for these two reasons. First, the two-stage framework enables us to focus on the impact of decomposition. Otherwise, directly generating decomposed solutions may introduce differences in solution contents, which can also impact human repair experiences. Second, we observe that code generation models might perform worse when prompted to generate decomposed solutions directly. Table <a href="#A2.T3" title="Table 3 ‣ B.2 Directly Generating Decomposed Solutions ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> presents the results, from which we can observe a substantial decrease in accuracy when directly prompting GPT-4 to generate decomposed solutions. We conjecture this might be due to the fact that most codes in these language models’ pre-training corpus are not well modularized and decomposed.</p>
</div>
<figure id="A2.T3" class="ltx_table">
<div id="A2.T3.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:87.9pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(83.7pt,-17.0pt) scale(1.62861764657675,1.62861764657675) ;">
<table id="A2.T3.1.1" class="ltx_tabular ltx_align_middle">
<tr id="A2.T3.1.1.1" class="ltx_tr">
<td id="A2.T3.1.1.1.1" class="ltx_td ltx_align_left ltx_border_tt"><span id="A2.T3.1.1.1.1.1" class="ltx_text ltx_font_bold">Prompt</span></td>
<td id="A2.T3.1.1.1.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="A2.T3.1.1.1.2.1" class="ltx_text ltx_font_bold">Strict Accuracy</span></td>
<td id="A2.T3.1.1.1.3" class="ltx_td ltx_align_center ltx_border_tt"><span id="A2.T3.1.1.1.3.1" class="ltx_text ltx_font_bold">Test Case Average</span></td>
</tr>
<tr id="A2.T3.1.1.2" class="ltx_tr">
<td id="A2.T3.1.1.2.1" class="ltx_td ltx_align_left ltx_border_t">Non-decomposed</td>
<td id="A2.T3.1.1.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="A2.T3.1.1.2.2.1" class="ltx_text ltx_font_bold">18.3</span></td>
<td id="A2.T3.1.1.2.3" class="ltx_td ltx_align_center ltx_border_t"><span id="A2.T3.1.1.2.3.1" class="ltx_text ltx_font_bold">41.5</span></td>
</tr>
<tr id="A2.T3.1.1.3" class="ltx_tr">
<td id="A2.T3.1.1.3.1" class="ltx_td ltx_align_left ltx_border_bb">Decomposed</td>
<td id="A2.T3.1.1.3.2" class="ltx_td ltx_align_center ltx_border_bb">16.7</td>
<td id="A2.T3.1.1.3.3" class="ltx_td ltx_align_center ltx_border_bb">37.4</td>
</tr>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 3: </span>Prompting GPT-4 to generate non-decomposed solutions and decomposed solutions. We evaluate the accuracy of the generated solutions on APPS.</figcaption>
</figure>
</section>
<section id="A2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">B.3 </span>Evaluating Code Decomposition with Software Engineering Metrics</h3>

<div id="A2.SS3.p1" class="ltx_para">
<p id="A2.SS3.p1.1" class="ltx_p">We adopt the following four metrics to evaluate decomposition, which are widely adopted in software engineering:</p>
<ul id="A2.I1" class="ltx_itemize">
<li id="A2.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I1.i1.p1" class="ltx_para">
<p id="A2.I1.i1.p1.1" class="ltx_p">Func Number: It calculates the average number of functions (i.e., subtasks)</p>
</div>
</li>
<li id="A2.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I1.i2.p1" class="ltx_para">
<p id="A2.I1.i2.p1.1" class="ltx_p">Avg Complexity: It first calculates the average cyclomatic complexity among all pieces in a single program and then takes an average across all programs.</p>
</div>
</li>
<li id="A2.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I1.i3.p1" class="ltx_para">
<p id="A2.I1.i3.p1.1" class="ltx_p">Max Complexity: It first calculates the maximum cyclomatic complexity among all pieces in a single program and then takes an average across all programs.</p>
</div>
</li>
<li id="A2.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I1.i4.p1" class="ltx_para">
<p id="A2.I1.i4.p1.1" class="ltx_p">Global Max Complexity: It calculates the maximum cyclomatic complexity among all pieces across all programs.</p>
</div>
</li>
</ul>
</div>
<div id="A2.SS3.p2" class="ltx_para">
<p id="A2.SS3.p2.1" class="ltx_p">From the results shown in Table <a href="#A2.T4" title="Table 4 ‣ B.3 Evaluating Code Decomposition with Software Engineering Metrics ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, we can see that by learning from human feedback, our decomposition model produces more subtasks and lower complexity than the vanilla decomposition baseline. In addition, while heuristic decomposition results in the largest number of subtasks and the lowest cyclomatic complexity, it does not effectively assist humans in practice. These results indicate that these naive objectives for code decomposition (e.g., function number, cyclomatic complexity) are not well aligned with the assistive value in the actual human supervision process.</p>
</div>
<figure id="A2.T4" class="ltx_table">
<div id="A2.T4.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:433.6pt;height:291.3pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(56.1pt,-37.7pt) scale(1.3486797752849,1.3486797752849) ;">
<table id="A2.T4.1.1" class="ltx_tabular ltx_align_middle">
<tr id="A2.T4.1.1.1" class="ltx_tr">
<td id="A2.T4.1.1.1.1" class="ltx_td ltx_align_left ltx_border_tt" rowspan="2"><span id="A2.T4.1.1.1.1.1" class="ltx_text ltx_font_bold">Program</span></td>
<td id="A2.T4.1.1.1.2" class="ltx_td ltx_align_center ltx_border_tt" rowspan="2"><span id="A2.T4.1.1.1.2.1" class="ltx_text ltx_font_bold">Func Number</span></td>
<td id="A2.T4.1.1.1.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="3"><span id="A2.T4.1.1.1.3.1" class="ltx_text ltx_font_bold">Complexity</span></td>
</tr>
<tr id="A2.T4.1.1.2" class="ltx_tr">
<td id="A2.T4.1.1.2.1" class="ltx_td ltx_align_center"><span id="A2.T4.1.1.2.1.1" class="ltx_text ltx_font_bold">Avg</span></td>
<td id="A2.T4.1.1.2.2" class="ltx_td ltx_align_center"><span id="A2.T4.1.1.2.2.1" class="ltx_text ltx_font_bold">Max</span></td>
<td id="A2.T4.1.1.2.3" class="ltx_td ltx_align_center"><span id="A2.T4.1.1.2.3.1" class="ltx_text ltx_font_bold">Global Max</span></td>
</tr>
<tr id="A2.T4.1.1.3" class="ltx_tr">
<td id="A2.T4.1.1.3.1" class="ltx_td ltx_align_center ltx_border_t" style="background-color:#F2F2F2;" colspan="5"><span id="A2.T4.1.1.3.1.1" class="ltx_text ltx_font_italic" style="background-color:#F2F2F2;">APPS</span></td>
</tr>
<tr id="A2.T4.1.1.4" class="ltx_tr">
<td id="A2.T4.1.1.4.1" class="ltx_td ltx_align_left ltx_border_t">Initial</td>
<td id="A2.T4.1.1.4.2" class="ltx_td ltx_align_center ltx_border_t">0.4</td>
<td id="A2.T4.1.1.4.3" class="ltx_td ltx_align_center ltx_border_t">5.5</td>
<td id="A2.T4.1.1.4.4" class="ltx_td ltx_align_center ltx_border_t">5.7</td>
<td id="A2.T4.1.1.4.5" class="ltx_td ltx_align_center ltx_border_t">17.0</td>
</tr>
<tr id="A2.T4.1.1.5" class="ltx_tr">
<td id="A2.T4.1.1.5.1" class="ltx_td ltx_align_left">Heuristic Decomp</td>
<td id="A2.T4.1.1.5.2" class="ltx_td ltx_align_center">4.5</td>
<td id="A2.T4.1.1.5.3" class="ltx_td ltx_align_center">2.0</td>
<td id="A2.T4.1.1.5.4" class="ltx_td ltx_align_center">2.3</td>
<td id="A2.T4.1.1.5.5" class="ltx_td ltx_align_center">4.0</td>
</tr>
<tr id="A2.T4.1.1.6" class="ltx_tr">
<td id="A2.T4.1.1.6.1" class="ltx_td ltx_align_left">Vanilla Decomp</td>
<td id="A2.T4.1.1.6.2" class="ltx_td ltx_align_center">2.8</td>
<td id="A2.T4.1.1.6.3" class="ltx_td ltx_align_center">2.5</td>
<td id="A2.T4.1.1.6.4" class="ltx_td ltx_align_center">4.0</td>
<td id="A2.T4.1.1.6.5" class="ltx_td ltx_align_center">12.0</td>
</tr>
<tr id="A2.T4.1.1.7" class="ltx_tr">
<td id="A2.T4.1.1.7.1" class="ltx_td ltx_align_left">Human-Centric Decomp</td>
<td id="A2.T4.1.1.7.2" class="ltx_td ltx_align_center">3.1</td>
<td id="A2.T4.1.1.7.3" class="ltx_td ltx_align_center">2.4</td>
<td id="A2.T4.1.1.7.4" class="ltx_td ltx_align_center">3.8</td>
<td id="A2.T4.1.1.7.5" class="ltx_td ltx_align_center">10.0</td>
</tr>
<tr id="A2.T4.1.1.8" class="ltx_tr">
<td id="A2.T4.1.1.8.1" class="ltx_td ltx_align_center ltx_border_t" style="background-color:#F2F2F2;" colspan="5"><span id="A2.T4.1.1.8.1.1" class="ltx_text ltx_font_italic" style="background-color:#F2F2F2;">Code-Contests</span></td>
</tr>
<tr id="A2.T4.1.1.9" class="ltx_tr">
<td id="A2.T4.1.1.9.1" class="ltx_td ltx_align_left ltx_border_t">Initial</td>
<td id="A2.T4.1.1.9.2" class="ltx_td ltx_align_center ltx_border_t">0.4</td>
<td id="A2.T4.1.1.9.3" class="ltx_td ltx_align_center ltx_border_t">5.1</td>
<td id="A2.T4.1.1.9.4" class="ltx_td ltx_align_center ltx_border_t">5.7</td>
<td id="A2.T4.1.1.9.5" class="ltx_td ltx_align_center ltx_border_t">18.0</td>
</tr>
<tr id="A2.T4.1.1.10" class="ltx_tr">
<td id="A2.T4.1.1.10.1" class="ltx_td ltx_align_left">Heuristic Decomp</td>
<td id="A2.T4.1.1.10.2" class="ltx_td ltx_align_center">5.2</td>
<td id="A2.T4.1.1.10.3" class="ltx_td ltx_align_center">2.0</td>
<td id="A2.T4.1.1.10.4" class="ltx_td ltx_align_center">2.3</td>
<td id="A2.T4.1.1.10.5" class="ltx_td ltx_align_center">5.0</td>
</tr>
<tr id="A2.T4.1.1.11" class="ltx_tr">
<td id="A2.T4.1.1.11.1" class="ltx_td ltx_align_left">Vanilla Decomp</td>
<td id="A2.T4.1.1.11.2" class="ltx_td ltx_align_center">2.9</td>
<td id="A2.T4.1.1.11.3" class="ltx_td ltx_align_center">2.7</td>
<td id="A2.T4.1.1.11.4" class="ltx_td ltx_align_center">4.2</td>
<td id="A2.T4.1.1.11.5" class="ltx_td ltx_align_center">12.0</td>
</tr>
<tr id="A2.T4.1.1.12" class="ltx_tr">
<td id="A2.T4.1.1.12.1" class="ltx_td ltx_align_left ltx_border_bb">Human-Centric Decomp</td>
<td id="A2.T4.1.1.12.2" class="ltx_td ltx_align_center ltx_border_bb">4.3</td>
<td id="A2.T4.1.1.12.3" class="ltx_td ltx_align_center ltx_border_bb">2.2</td>
<td id="A2.T4.1.1.12.4" class="ltx_td ltx_align_center ltx_border_bb">3.7</td>
<td id="A2.T4.1.1.12.5" class="ltx_td ltx_align_center ltx_border_bb">10.0</td>
</tr>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 4: </span>Statistics of initial programs and decomposed programs. “Func number” denotes the average number of functions (i.e., subtasks), “Complexity” denotes the value of cyclomatic complexity. </figcaption>
</figure>
</section>
<section id="A2.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">B.4 </span>Analysis of the Rank Model</h3>

<div id="A2.SS4.p1" class="ltx_para">
<p id="A2.SS4.p1.3" class="ltx_p">Given a question <math id="A2.SS4.p1.1.m1.1" class="ltx_Math" alttext="Q" display="inline"><semantics id="A2.SS4.p1.1.m1.1a"><mi id="A2.SS4.p1.1.m1.1.1" xref="A2.SS4.p1.1.m1.1.1.cmml">Q</mi><annotation-xml encoding="MathML-Content" id="A2.SS4.p1.1.m1.1b"><ci id="A2.SS4.p1.1.m1.1.1.cmml" xref="A2.SS4.p1.1.m1.1.1">𝑄</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.SS4.p1.1.m1.1c">Q</annotation></semantics></math>, and a pair of decomposed solutions <math id="A2.SS4.p1.2.m2.1" class="ltx_Math" alttext="A^{1}_{d}" display="inline"><semantics id="A2.SS4.p1.2.m2.1a"><msubsup id="A2.SS4.p1.2.m2.1.1" xref="A2.SS4.p1.2.m2.1.1.cmml"><mi id="A2.SS4.p1.2.m2.1.1.2.2" xref="A2.SS4.p1.2.m2.1.1.2.2.cmml">A</mi><mi id="A2.SS4.p1.2.m2.1.1.3" xref="A2.SS4.p1.2.m2.1.1.3.cmml">d</mi><mn id="A2.SS4.p1.2.m2.1.1.2.3" xref="A2.SS4.p1.2.m2.1.1.2.3.cmml">1</mn></msubsup><annotation-xml encoding="MathML-Content" id="A2.SS4.p1.2.m2.1b"><apply id="A2.SS4.p1.2.m2.1.1.cmml" xref="A2.SS4.p1.2.m2.1.1"><csymbol cd="ambiguous" id="A2.SS4.p1.2.m2.1.1.1.cmml" xref="A2.SS4.p1.2.m2.1.1">subscript</csymbol><apply id="A2.SS4.p1.2.m2.1.1.2.cmml" xref="A2.SS4.p1.2.m2.1.1"><csymbol cd="ambiguous" id="A2.SS4.p1.2.m2.1.1.2.1.cmml" xref="A2.SS4.p1.2.m2.1.1">superscript</csymbol><ci id="A2.SS4.p1.2.m2.1.1.2.2.cmml" xref="A2.SS4.p1.2.m2.1.1.2.2">𝐴</ci><cn type="integer" id="A2.SS4.p1.2.m2.1.1.2.3.cmml" xref="A2.SS4.p1.2.m2.1.1.2.3">1</cn></apply><ci id="A2.SS4.p1.2.m2.1.1.3.cmml" xref="A2.SS4.p1.2.m2.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS4.p1.2.m2.1c">A^{1}_{d}</annotation></semantics></math> and <math id="A2.SS4.p1.3.m3.1" class="ltx_Math" alttext="A^{2}_{d}" display="inline"><semantics id="A2.SS4.p1.3.m3.1a"><msubsup id="A2.SS4.p1.3.m3.1.1" xref="A2.SS4.p1.3.m3.1.1.cmml"><mi id="A2.SS4.p1.3.m3.1.1.2.2" xref="A2.SS4.p1.3.m3.1.1.2.2.cmml">A</mi><mi id="A2.SS4.p1.3.m3.1.1.3" xref="A2.SS4.p1.3.m3.1.1.3.cmml">d</mi><mn id="A2.SS4.p1.3.m3.1.1.2.3" xref="A2.SS4.p1.3.m3.1.1.2.3.cmml">2</mn></msubsup><annotation-xml encoding="MathML-Content" id="A2.SS4.p1.3.m3.1b"><apply id="A2.SS4.p1.3.m3.1.1.cmml" xref="A2.SS4.p1.3.m3.1.1"><csymbol cd="ambiguous" id="A2.SS4.p1.3.m3.1.1.1.cmml" xref="A2.SS4.p1.3.m3.1.1">subscript</csymbol><apply id="A2.SS4.p1.3.m3.1.1.2.cmml" xref="A2.SS4.p1.3.m3.1.1"><csymbol cd="ambiguous" id="A2.SS4.p1.3.m3.1.1.2.1.cmml" xref="A2.SS4.p1.3.m3.1.1">superscript</csymbol><ci id="A2.SS4.p1.3.m3.1.1.2.2.cmml" xref="A2.SS4.p1.3.m3.1.1.2.2">𝐴</ci><cn type="integer" id="A2.SS4.p1.3.m3.1.1.2.3.cmml" xref="A2.SS4.p1.3.m3.1.1.2.3">2</cn></apply><ci id="A2.SS4.p1.3.m3.1.1.3.cmml" xref="A2.SS4.p1.3.m3.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS4.p1.3.m3.1c">A^{2}_{d}</annotation></semantics></math>, we adopt the following four methods to predict which decomposed solution leads to higher assistive value:</p>
<ul id="A2.I2" class="ltx_itemize">
<li id="A2.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I2.i1.p1" class="ltx_para">
<p id="A2.I2.i1.p1.1" class="ltx_p"><span id="A2.I2.i1.p1.1.1" class="ltx_text ltx_font_bold">Intuitive Human Preference</span>: ask human programmers to give an intuitive preference for which decomposition can lead to higher human repair performance (i.e., higher assistive value). For each decomposition pair, we ask three annotators to give a preference. We adopt majority voting to make final decisions among three annotators.</p>
</div>
</li>
<li id="A2.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I2.i2.p1" class="ltx_para">
<p id="A2.I2.i2.p1.1" class="ltx_p"><span id="A2.I2.i2.p1.1.1" class="ltx_text ltx_font_bold">Cyclomatic Complexity</span>: consider the solution with a lower cyclomatic complexity as the better one.</p>
</div>
</li>
<li id="A2.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I2.i3.p1" class="ltx_para">
<p id="A2.I2.i3.p1.1" class="ltx_p"><span id="A2.I2.i3.p1.1.1" class="ltx_text ltx_font_bold">Zero-shot</span>: prompt a code language model <math id="A2.I2.i3.p1.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="A2.I2.i3.p1.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="A2.I2.i3.p1.1.m1.1.1" xref="A2.I2.i3.p1.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="A2.I2.i3.p1.1.m1.1b"><ci id="A2.I2.i3.p1.1.m1.1.1.cmml" xref="A2.I2.i3.p1.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.I2.i3.p1.1.m1.1c">\mathcal{M}</annotation></semantics></math> to select a more effective decomposition in a zero-shot setting.</p>
</div>
</li>
<li id="A2.I2.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="A2.I2.i4.p1" class="ltx_para">
<p id="A2.I2.i4.p1.2" class="ltx_p"><span id="A2.I2.i4.p1.1.1" class="ltx_text ltx_font_bold">Few-shot (<math id="A2.I2.i4.p1.1.1.m1.1" class="ltx_Math" alttext="\pi_{\text{rank}}" display="inline"><semantics id="A2.I2.i4.p1.1.1.m1.1a"><msub id="A2.I2.i4.p1.1.1.m1.1.1" xref="A2.I2.i4.p1.1.1.m1.1.1.cmml"><mi id="A2.I2.i4.p1.1.1.m1.1.1.2" xref="A2.I2.i4.p1.1.1.m1.1.1.2.cmml">π</mi><mtext class="ltx_mathvariant_bold" id="A2.I2.i4.p1.1.1.m1.1.1.3" xref="A2.I2.i4.p1.1.1.m1.1.1.3a.cmml">rank</mtext></msub><annotation-xml encoding="MathML-Content" id="A2.I2.i4.p1.1.1.m1.1b"><apply id="A2.I2.i4.p1.1.1.m1.1.1.cmml" xref="A2.I2.i4.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A2.I2.i4.p1.1.1.m1.1.1.1.cmml" xref="A2.I2.i4.p1.1.1.m1.1.1">subscript</csymbol><ci id="A2.I2.i4.p1.1.1.m1.1.1.2.cmml" xref="A2.I2.i4.p1.1.1.m1.1.1.2">𝜋</ci><ci id="A2.I2.i4.p1.1.1.m1.1.1.3a.cmml" xref="A2.I2.i4.p1.1.1.m1.1.1.3"><mtext class="ltx_mathvariant_bold" mathsize="70%" id="A2.I2.i4.p1.1.1.m1.1.1.3.cmml" xref="A2.I2.i4.p1.1.1.m1.1.1.3">rank</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.I2.i4.p1.1.1.m1.1c">\pi_{\text{rank}}</annotation></semantics></math>)</span>: prompt a code language model <math id="A2.I2.i4.p1.2.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="A2.I2.i4.p1.2.m1.1a"><mi class="ltx_font_mathcaligraphic" id="A2.I2.i4.p1.2.m1.1.1" xref="A2.I2.i4.p1.2.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="A2.I2.i4.p1.2.m1.1b"><ci id="A2.I2.i4.p1.2.m1.1.1.cmml" xref="A2.I2.i4.p1.2.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.I2.i4.p1.2.m1.1c">\mathcal{M}</annotation></semantics></math> to select a more effective decomposition in a few-shot setting, where the few-shot demonstrations are collected from the feedback of real human labelers as illustrated in Section <a href="#S2" title="2 Methodology ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="A2.SS5" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">B.5 </span>Analysis of the Distilled Decomposition Model</h3>

<div id="A2.SS5.p1" class="ltx_para">
<p id="A2.SS5.p1.1" class="ltx_p">We compare the distilled assistive decomposition model based on Code-LLaMA-13B with the vanilla decomposition based on Code-LLaMA-13B and GPT-4, as well as its GPT-4-based teacher model. We conduct experiments to assist both human supervision and AI supervision.</p>
</div>
<section id="A2.SS5.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Assisting Human Supervision</h5>

<figure id="A2.F9" class="ltx_figure"><img src="/html/2406.04604/assets/x4.png" id="A2.F9.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="415" height="192" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 9: </span>Comparison of the distilled assistive Code-LLaMA model against three other decomposition models. We use our rank model as a proxy evaluator.</figcaption>
</figure>
<figure id="A2.F10" class="ltx_figure"><img src="/html/2406.04604/assets/pics/distill.png" id="A2.F10.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="538" height="400" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 10: </span>Assisting AI supervision with the distilled decomposition model.</figcaption>
</figure>
<div id="A2.SS5.SSS0.Px1.p1" class="ltx_para">
<p id="A2.SS5.SSS0.Px1.p1.1" class="ltx_p">Based on the reasonable accuracy of our rank model in predicting the actual assistive value of decompositions (Section <a href="#S4.SS1" title="4.1 Validity of the Rank Model ‣ 4 Analysis ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1</span></a>), we use the prediction of our rank model as a proxy to evaluate the distilled decomposition model. The results shown in Figure <a href="#A2.F9" title="Figure 9 ‣ Assisting Human Supervision ‣ B.5 Analysis of the Distilled Decomposition Model ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a> demonstrate that the distilled assistive Code-LLaMA model substantially outperforms vanilla Code-LLaMA and moderately outperforms vanilla GPT-4, thanks to the internalized high-<span id="A2.SS5.SSS0.Px1.p1.1.1" class="ltx_text ltx_font_typewriter">AssistV</span> decomposition knowledge in the generated data from its teacher model.</p>
</div>
</section>
<section id="A2.SS5.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Assisting AI Supervision</h5>

<div id="A2.SS5.SSS0.Px2.p1" class="ltx_para">
<p id="A2.SS5.SSS0.Px2.p1.1" class="ltx_p">As illustrated in Section <a href="#S3.SS3.SSS0.Px1" title="Decomposition facilitates accurate self-supervision ‣ 3.3 Assisting AI Supervision ‣ 3 Experiments ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.3</span></a>, we evaluate the distilled decomposition model’s performance in assisting AI systems in providing two forms of supervision: discrimination and repair. As shown in Figure <a href="#A2.F10" title="Figure 10 ‣ Assisting Human Supervision ‣ B.5 Analysis of the Distilled Decomposition Model ‣ Appendix B Additional Results ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>, the distilled decomposition model leads to more accurate AI supervision than the vanilla Code-LLaMA-13B model and even the vanilla GPT-4.</p>
</div>
</section>
</section>
</section>
<section id="A3" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix C </span>Additional Human Annotation Details</h2>

<section id="A3.SS0.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Labeler selection</h5>

<div id="A3.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="A3.SS0.SSS0.Px1.p1.1" class="ltx_p">Our labelers are mainly hired from college students. To ensure the honesty of labelers, we track their debugging trajectories and filter those who plagiarize online golden solutions. As for group slicing, besides conducting a pre-survey to collect their self-evaluation on the programming level, we further examine their level based on their performance during the warm-up test.</p>
</div>
</section>
<section id="A3.SS0.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Labeling instruction</h5>

<div id="A3.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="A3.SS0.SSS0.Px2.p1.1" class="ltx_p">We present the summary of our labeling instructions in Table <a href="#A3.T5" title="Table 5 ‣ Labeling instruction ‣ Appendix C Additional Human Annotation Details ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</p>
</div>
<figure id="A3.T5" class="ltx_table">
<table id="A3.T5.1" class="ltx_tabular ltx_centering ltx_align_middle">
<tr id="A3.T5.1.1" class="ltx_tr">
<td id="A3.T5.1.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_tt">
<span id="A3.T5.1.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="A3.T5.1.1.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A3.T5.1.1.1.1.1.1" class="ltx_text ltx_font_bold">Instruction:</span></span>
</span>
</td>
</tr>
<tr id="A3.T5.1.2" class="ltx_tr">
<td id="A3.T5.1.2.1" class="ltx_td ltx_align_justify ltx_align_top">
<span id="A3.T5.1.2.1.1" class="ltx_inline-block ltx_align_top">
<span id="A3.T5.1.2.1.1.1" class="ltx_p" style="width:433.6pt;">You are given an algorithmic coding problem and a model-generated solution. Your job is to debug the solution and improve its accuracy.</span>
</span>
</td>
</tr>
<tr id="A3.T5.1.3" class="ltx_tr">
<td id="A3.T5.1.3.1" class="ltx_td ltx_align_justify ltx_align_top">
<span id="A3.T5.1.3.1.1" class="ltx_inline-block ltx_align_top">
<span id="A3.T5.1.3.1.1.1" class="ltx_p" style="width:433.6pt;">During debugging, you can actively submit your code and run your custom test cases. If the solution has a modular structure with multiple subfunctions, leverage it to accelerate your debugging. For example, you can check the presented high-level logic before inspecting low-level implementations, and you can perform function-level debugging.</span>
</span>
</td>
</tr>
<tr id="A3.T5.1.4" class="ltx_tr">
<td id="A3.T5.1.4.1" class="ltx_td ltx_align_justify ltx_align_top">
<span id="A3.T5.1.4.1.1" class="ltx_inline-block ltx_align_top">
<span id="A3.T5.1.4.1.1.1" class="ltx_p" style="width:433.6pt;">There are two criteria to stop debugging:
(1) The repaired solution passes all hidden test cases.
(2) The debugging time already exceeds 30 minutes. This timeframe is imposed to avoid endless debugging.</span>
</span>
</td>
</tr>
<tr id="A3.T5.1.5" class="ltx_tr">
<td id="A3.T5.1.5.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_t">
<span id="A3.T5.1.5.1.1" class="ltx_inline-block ltx_align_top">
<span id="A3.T5.1.5.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A3.T5.1.5.1.1.1.1" class="ltx_text ltx_font_bold">Survey:</span></span>
</span>
</td>
</tr>
<tr id="A3.T5.1.6" class="ltx_tr">
<td id="A3.T5.1.6.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_bb">
<span id="A3.T5.1.6.1.1" class="ltx_inline-block ltx_align_top">
<span id="A3.T5.1.6.1.1.1" class="ltx_p" style="width:433.6pt;">Review your debugging process and answer the following questions.


<span id="A3.I1" class="ltx_itemize">
<span id="A3.I1.i1" class="ltx_item" style="list-style-type:none;"><span class="ltx_tag ltx_tag_item">•</span> 
<span id="A3.I1.i1.p1" class="ltx_para">
<span id="A3.I1.i1.p1.1" class="ltx_p">Fixed Bugs: What bugs have you fixed during debugging?</span>
</span></span>
<span id="A3.I1.i2" class="ltx_item" style="list-style-type:none;padding-top:-2.8pt;"><span class="ltx_tag ltx_tag_item">•</span> 
<span id="A3.I1.i2.p1" class="ltx_para">
<span id="A3.I1.i2.p1.1" class="ltx_p">Critique on Decomposition: how does the current decomposition impede or improve your debugging efficiency?</span>
</span></span>
<span id="A3.I1.i3" class="ltx_item" style="list-style-type:none;padding-top:-2.8pt;"><span class="ltx_tag ltx_tag_item">•</span> 
<span id="A3.I1.i3.p1" class="ltx_para">
<span id="A3.I1.i3.p1.1" class="ltx_p">Other Assistance Forms: what other assistance forms do you need based on your debugging experience?</span>
</span></span>
</span></span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 5: </span>Summary of our labeling instruction and survey.</figcaption>
</figure>
<figure id="A3.SS0.SSS0.Px2.tab1" class="ltx_table">
</figure>
</section>
<section id="A3.SS0.SSS0.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Interface</h5>

<div id="A3.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="A3.SS0.SSS0.Px3.p1.1" class="ltx_p">In Figure <a href="#A3.F12" title="Figure 12 ‣ Interface ‣ Appendix C Additional Human Annotation Details ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">12</span></a>, we present screenshots of our interface.</p>
</div>
<figure id="A3.F11.sf1" class="ltx_figure"><img src="/html/2406.04604/assets/pics/interface-main.png" id="A3.F11.sf1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="598" height="452" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">(a) </span></figcaption>
</figure>
<figure id="A3.F12" class="ltx_figure">
<figure id="A3.F12.sf1" class="ltx_figure ltx_align_center"><img src="/html/2406.04604/assets/pics/interface.png" id="A3.F12.sf1.g1" class="ltx_graphics ltx_centering ltx_img_portrait" width="359" height="829" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">(a) </span></figcaption>
</figure>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 12: </span>Screenshots of our labeling interface. (a) After entering the OJ system, labelers can see a brief labeling instruction and a problem list. (b) For each problem, annotators are required to perform the debugging task. During debugging, our OJ system supports annotators to run their custom test cases.</figcaption>
</figure>
</section>
</section>
<section id="A4" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix D </span>Example Prompts</h2>

<section id="A4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">D.1 </span>Generating Initial Solution</h3>

<div id="A4.SS1.p1" class="ltx_para">
<p id="A4.SS1.p1.1" class="ltx_p">For generating the Initial solution, we adopt the prompt in Table <a href="#A4.T6" title="Table 6 ‣ D.1 Generating Initial Solution ‣ Appendix D Example Prompts ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>.</p>
</div>
<figure id="A4.T6" class="ltx_table">
<table id="A4.T6.30" class="ltx_tabular ltx_centering ltx_align_middle">
<tr id="A4.T6.30.31" class="ltx_tr">
<td id="A4.T6.30.31.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r ltx_border_t">
<span id="A4.T6.30.31.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T6.30.31.1.1.1" class="ltx_p" style="width:411.9pt;"><span id="A4.T6.30.31.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T6.30.30" class="ltx_tr">
<td id="A4.T6.30.30.30" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T6.30.30.30.30" class="ltx_inline-block ltx_align_top">
<span id="A4.T6.3.3.3.3.3" class="ltx_p" style="width:411.9pt;">You are given a tree consisting of <math id="A4.T6.1.1.1.1.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A4.T6.1.1.1.1.1.m1.1a"><mi id="A4.T6.1.1.1.1.1.m1.1.1" xref="A4.T6.1.1.1.1.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A4.T6.1.1.1.1.1.m1.1b"><ci id="A4.T6.1.1.1.1.1.m1.1.1.cmml" xref="A4.T6.1.1.1.1.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.1.1.1.1.1.m1.1c">n</annotation></semantics></math> vertices. A number is written on each vertex; the number on vertex <math id="A4.T6.2.2.2.2.2.m2.1" class="ltx_Math" alttext="i" display="inline"><semantics id="A4.T6.2.2.2.2.2.m2.1a"><mi id="A4.T6.2.2.2.2.2.m2.1.1" xref="A4.T6.2.2.2.2.2.m2.1.1.cmml">i</mi><annotation-xml encoding="MathML-Content" id="A4.T6.2.2.2.2.2.m2.1b"><ci id="A4.T6.2.2.2.2.2.m2.1.1.cmml" xref="A4.T6.2.2.2.2.2.m2.1.1">𝑖</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.2.2.2.2.2.m2.1c">i</annotation></semantics></math> is equal to <math id="A4.T6.3.3.3.3.3.m3.1" class="ltx_Math" alttext="a_{i}" display="inline"><semantics id="A4.T6.3.3.3.3.3.m3.1a"><msub id="A4.T6.3.3.3.3.3.m3.1.1" xref="A4.T6.3.3.3.3.3.m3.1.1.cmml"><mi id="A4.T6.3.3.3.3.3.m3.1.1.2" xref="A4.T6.3.3.3.3.3.m3.1.1.2.cmml">a</mi><mi id="A4.T6.3.3.3.3.3.m3.1.1.3" xref="A4.T6.3.3.3.3.3.m3.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A4.T6.3.3.3.3.3.m3.1b"><apply id="A4.T6.3.3.3.3.3.m3.1.1.cmml" xref="A4.T6.3.3.3.3.3.m3.1.1"><csymbol cd="ambiguous" id="A4.T6.3.3.3.3.3.m3.1.1.1.cmml" xref="A4.T6.3.3.3.3.3.m3.1.1">subscript</csymbol><ci id="A4.T6.3.3.3.3.3.m3.1.1.2.cmml" xref="A4.T6.3.3.3.3.3.m3.1.1.2">𝑎</ci><ci id="A4.T6.3.3.3.3.3.m3.1.1.3.cmml" xref="A4.T6.3.3.3.3.3.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.3.3.3.3.3.m3.1c">a_{i}</annotation></semantics></math>.</span>
<span id="A4.T6.11.11.11.11.11" class="ltx_p">Let’s denote the function <math id="A4.T6.4.4.4.4.4.m1.2" class="ltx_Math" alttext="g(x,y)" display="inline"><semantics id="A4.T6.4.4.4.4.4.m1.2a"><mrow id="A4.T6.4.4.4.4.4.m1.2.3" xref="A4.T6.4.4.4.4.4.m1.2.3.cmml"><mi id="A4.T6.4.4.4.4.4.m1.2.3.2" xref="A4.T6.4.4.4.4.4.m1.2.3.2.cmml">g</mi><mo lspace="0em" rspace="0em" id="A4.T6.4.4.4.4.4.m1.2.3.1" xref="A4.T6.4.4.4.4.4.m1.2.3.1.cmml">​</mo><mrow id="A4.T6.4.4.4.4.4.m1.2.3.3.2" xref="A4.T6.4.4.4.4.4.m1.2.3.3.1.cmml"><mo stretchy="false" id="A4.T6.4.4.4.4.4.m1.2.3.3.2.1" xref="A4.T6.4.4.4.4.4.m1.2.3.3.1.cmml">(</mo><mi id="A4.T6.4.4.4.4.4.m1.1.1" xref="A4.T6.4.4.4.4.4.m1.1.1.cmml">x</mi><mo id="A4.T6.4.4.4.4.4.m1.2.3.3.2.2" xref="A4.T6.4.4.4.4.4.m1.2.3.3.1.cmml">,</mo><mi id="A4.T6.4.4.4.4.4.m1.2.2" xref="A4.T6.4.4.4.4.4.m1.2.2.cmml">y</mi><mo stretchy="false" id="A4.T6.4.4.4.4.4.m1.2.3.3.2.3" xref="A4.T6.4.4.4.4.4.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.4.4.4.4.4.m1.2b"><apply id="A4.T6.4.4.4.4.4.m1.2.3.cmml" xref="A4.T6.4.4.4.4.4.m1.2.3"><times id="A4.T6.4.4.4.4.4.m1.2.3.1.cmml" xref="A4.T6.4.4.4.4.4.m1.2.3.1"></times><ci id="A4.T6.4.4.4.4.4.m1.2.3.2.cmml" xref="A4.T6.4.4.4.4.4.m1.2.3.2">𝑔</ci><interval closure="open" id="A4.T6.4.4.4.4.4.m1.2.3.3.1.cmml" xref="A4.T6.4.4.4.4.4.m1.2.3.3.2"><ci id="A4.T6.4.4.4.4.4.m1.1.1.cmml" xref="A4.T6.4.4.4.4.4.m1.1.1">𝑥</ci><ci id="A4.T6.4.4.4.4.4.m1.2.2.cmml" xref="A4.T6.4.4.4.4.4.m1.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.4.4.4.4.4.m1.2c">g(x,y)</annotation></semantics></math> as the greatest common divisor of the numbers written on the vertices belonging to the simple path from vertex <math id="A4.T6.5.5.5.5.5.m2.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T6.5.5.5.5.5.m2.1a"><mi id="A4.T6.5.5.5.5.5.m2.1.1" xref="A4.T6.5.5.5.5.5.m2.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T6.5.5.5.5.5.m2.1b"><ci id="A4.T6.5.5.5.5.5.m2.1.1.cmml" xref="A4.T6.5.5.5.5.5.m2.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.5.5.5.5.5.m2.1c">x</annotation></semantics></math> to vertex <math id="A4.T6.6.6.6.6.6.m3.1" class="ltx_Math" alttext="y" display="inline"><semantics id="A4.T6.6.6.6.6.6.m3.1a"><mi id="A4.T6.6.6.6.6.6.m3.1.1" xref="A4.T6.6.6.6.6.6.m3.1.1.cmml">y</mi><annotation-xml encoding="MathML-Content" id="A4.T6.6.6.6.6.6.m3.1b"><ci id="A4.T6.6.6.6.6.6.m3.1.1.cmml" xref="A4.T6.6.6.6.6.6.m3.1.1">𝑦</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.6.6.6.6.6.m3.1c">y</annotation></semantics></math> (including these two vertices). Also let’s denote <math id="A4.T6.7.7.7.7.7.m4.2" class="ltx_Math" alttext="dist(x,y)" display="inline"><semantics id="A4.T6.7.7.7.7.7.m4.2a"><mrow id="A4.T6.7.7.7.7.7.m4.2.3" xref="A4.T6.7.7.7.7.7.m4.2.3.cmml"><mi id="A4.T6.7.7.7.7.7.m4.2.3.2" xref="A4.T6.7.7.7.7.7.m4.2.3.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T6.7.7.7.7.7.m4.2.3.1" xref="A4.T6.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mi id="A4.T6.7.7.7.7.7.m4.2.3.3" xref="A4.T6.7.7.7.7.7.m4.2.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T6.7.7.7.7.7.m4.2.3.1a" xref="A4.T6.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mi id="A4.T6.7.7.7.7.7.m4.2.3.4" xref="A4.T6.7.7.7.7.7.m4.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T6.7.7.7.7.7.m4.2.3.1b" xref="A4.T6.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mi id="A4.T6.7.7.7.7.7.m4.2.3.5" xref="A4.T6.7.7.7.7.7.m4.2.3.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T6.7.7.7.7.7.m4.2.3.1c" xref="A4.T6.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mrow id="A4.T6.7.7.7.7.7.m4.2.3.6.2" xref="A4.T6.7.7.7.7.7.m4.2.3.6.1.cmml"><mo stretchy="false" id="A4.T6.7.7.7.7.7.m4.2.3.6.2.1" xref="A4.T6.7.7.7.7.7.m4.2.3.6.1.cmml">(</mo><mi id="A4.T6.7.7.7.7.7.m4.1.1" xref="A4.T6.7.7.7.7.7.m4.1.1.cmml">x</mi><mo id="A4.T6.7.7.7.7.7.m4.2.3.6.2.2" xref="A4.T6.7.7.7.7.7.m4.2.3.6.1.cmml">,</mo><mi id="A4.T6.7.7.7.7.7.m4.2.2" xref="A4.T6.7.7.7.7.7.m4.2.2.cmml">y</mi><mo stretchy="false" id="A4.T6.7.7.7.7.7.m4.2.3.6.2.3" xref="A4.T6.7.7.7.7.7.m4.2.3.6.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.7.7.7.7.7.m4.2b"><apply id="A4.T6.7.7.7.7.7.m4.2.3.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3"><times id="A4.T6.7.7.7.7.7.m4.2.3.1.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3.1"></times><ci id="A4.T6.7.7.7.7.7.m4.2.3.2.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3.2">𝑑</ci><ci id="A4.T6.7.7.7.7.7.m4.2.3.3.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3.3">𝑖</ci><ci id="A4.T6.7.7.7.7.7.m4.2.3.4.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3.4">𝑠</ci><ci id="A4.T6.7.7.7.7.7.m4.2.3.5.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3.5">𝑡</ci><interval closure="open" id="A4.T6.7.7.7.7.7.m4.2.3.6.1.cmml" xref="A4.T6.7.7.7.7.7.m4.2.3.6.2"><ci id="A4.T6.7.7.7.7.7.m4.1.1.cmml" xref="A4.T6.7.7.7.7.7.m4.1.1">𝑥</ci><ci id="A4.T6.7.7.7.7.7.m4.2.2.cmml" xref="A4.T6.7.7.7.7.7.m4.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.7.7.7.7.7.m4.2c">dist(x,y)</annotation></semantics></math> as the number of vertices on the simple path between vertices <math id="A4.T6.8.8.8.8.8.m5.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T6.8.8.8.8.8.m5.1a"><mi id="A4.T6.8.8.8.8.8.m5.1.1" xref="A4.T6.8.8.8.8.8.m5.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T6.8.8.8.8.8.m5.1b"><ci id="A4.T6.8.8.8.8.8.m5.1.1.cmml" xref="A4.T6.8.8.8.8.8.m5.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.8.8.8.8.8.m5.1c">x</annotation></semantics></math> and <math id="A4.T6.9.9.9.9.9.m6.1" class="ltx_Math" alttext="y" display="inline"><semantics id="A4.T6.9.9.9.9.9.m6.1a"><mi id="A4.T6.9.9.9.9.9.m6.1.1" xref="A4.T6.9.9.9.9.9.m6.1.1.cmml">y</mi><annotation-xml encoding="MathML-Content" id="A4.T6.9.9.9.9.9.m6.1b"><ci id="A4.T6.9.9.9.9.9.m6.1.1.cmml" xref="A4.T6.9.9.9.9.9.m6.1.1">𝑦</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.9.9.9.9.9.m6.1c">y</annotation></semantics></math>, including the endpoints. <math id="A4.T6.10.10.10.10.10.m7.2" class="ltx_Math" alttext="dist(x,x)=1" display="inline"><semantics id="A4.T6.10.10.10.10.10.m7.2a"><mrow id="A4.T6.10.10.10.10.10.m7.2.3" xref="A4.T6.10.10.10.10.10.m7.2.3.cmml"><mrow id="A4.T6.10.10.10.10.10.m7.2.3.2" xref="A4.T6.10.10.10.10.10.m7.2.3.2.cmml"><mi id="A4.T6.10.10.10.10.10.m7.2.3.2.2" xref="A4.T6.10.10.10.10.10.m7.2.3.2.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T6.10.10.10.10.10.m7.2.3.2.1" xref="A4.T6.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mi id="A4.T6.10.10.10.10.10.m7.2.3.2.3" xref="A4.T6.10.10.10.10.10.m7.2.3.2.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T6.10.10.10.10.10.m7.2.3.2.1a" xref="A4.T6.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mi id="A4.T6.10.10.10.10.10.m7.2.3.2.4" xref="A4.T6.10.10.10.10.10.m7.2.3.2.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T6.10.10.10.10.10.m7.2.3.2.1b" xref="A4.T6.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mi id="A4.T6.10.10.10.10.10.m7.2.3.2.5" xref="A4.T6.10.10.10.10.10.m7.2.3.2.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T6.10.10.10.10.10.m7.2.3.2.1c" xref="A4.T6.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mrow id="A4.T6.10.10.10.10.10.m7.2.3.2.6.2" xref="A4.T6.10.10.10.10.10.m7.2.3.2.6.1.cmml"><mo stretchy="false" id="A4.T6.10.10.10.10.10.m7.2.3.2.6.2.1" xref="A4.T6.10.10.10.10.10.m7.2.3.2.6.1.cmml">(</mo><mi id="A4.T6.10.10.10.10.10.m7.1.1" xref="A4.T6.10.10.10.10.10.m7.1.1.cmml">x</mi><mo id="A4.T6.10.10.10.10.10.m7.2.3.2.6.2.2" xref="A4.T6.10.10.10.10.10.m7.2.3.2.6.1.cmml">,</mo><mi id="A4.T6.10.10.10.10.10.m7.2.2" xref="A4.T6.10.10.10.10.10.m7.2.2.cmml">x</mi><mo stretchy="false" id="A4.T6.10.10.10.10.10.m7.2.3.2.6.2.3" xref="A4.T6.10.10.10.10.10.m7.2.3.2.6.1.cmml">)</mo></mrow></mrow><mo id="A4.T6.10.10.10.10.10.m7.2.3.1" xref="A4.T6.10.10.10.10.10.m7.2.3.1.cmml">=</mo><mn id="A4.T6.10.10.10.10.10.m7.2.3.3" xref="A4.T6.10.10.10.10.10.m7.2.3.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.10.10.10.10.10.m7.2b"><apply id="A4.T6.10.10.10.10.10.m7.2.3.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3"><eq id="A4.T6.10.10.10.10.10.m7.2.3.1.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.1"></eq><apply id="A4.T6.10.10.10.10.10.m7.2.3.2.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2"><times id="A4.T6.10.10.10.10.10.m7.2.3.2.1.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2.1"></times><ci id="A4.T6.10.10.10.10.10.m7.2.3.2.2.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2.2">𝑑</ci><ci id="A4.T6.10.10.10.10.10.m7.2.3.2.3.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2.3">𝑖</ci><ci id="A4.T6.10.10.10.10.10.m7.2.3.2.4.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2.4">𝑠</ci><ci id="A4.T6.10.10.10.10.10.m7.2.3.2.5.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2.5">𝑡</ci><interval closure="open" id="A4.T6.10.10.10.10.10.m7.2.3.2.6.1.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.2.6.2"><ci id="A4.T6.10.10.10.10.10.m7.1.1.cmml" xref="A4.T6.10.10.10.10.10.m7.1.1">𝑥</ci><ci id="A4.T6.10.10.10.10.10.m7.2.2.cmml" xref="A4.T6.10.10.10.10.10.m7.2.2">𝑥</ci></interval></apply><cn type="integer" id="A4.T6.10.10.10.10.10.m7.2.3.3.cmml" xref="A4.T6.10.10.10.10.10.m7.2.3.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.10.10.10.10.10.m7.2c">dist(x,x)=1</annotation></semantics></math> for every vertex <math id="A4.T6.11.11.11.11.11.m8.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T6.11.11.11.11.11.m8.1a"><mi id="A4.T6.11.11.11.11.11.m8.1.1" xref="A4.T6.11.11.11.11.11.m8.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T6.11.11.11.11.11.m8.1b"><ci id="A4.T6.11.11.11.11.11.m8.1.1.cmml" xref="A4.T6.11.11.11.11.11.m8.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.11.11.11.11.11.m8.1c">x</annotation></semantics></math>.</span>
<span id="A4.T6.13.13.13.13.13" class="ltx_p">Your task is to calculate the maximum value of <math id="A4.T6.12.12.12.12.12.m1.2" class="ltx_Math" alttext="dist(x,y)" display="inline"><semantics id="A4.T6.12.12.12.12.12.m1.2a"><mrow id="A4.T6.12.12.12.12.12.m1.2.3" xref="A4.T6.12.12.12.12.12.m1.2.3.cmml"><mi id="A4.T6.12.12.12.12.12.m1.2.3.2" xref="A4.T6.12.12.12.12.12.m1.2.3.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T6.12.12.12.12.12.m1.2.3.1" xref="A4.T6.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mi id="A4.T6.12.12.12.12.12.m1.2.3.3" xref="A4.T6.12.12.12.12.12.m1.2.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T6.12.12.12.12.12.m1.2.3.1a" xref="A4.T6.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mi id="A4.T6.12.12.12.12.12.m1.2.3.4" xref="A4.T6.12.12.12.12.12.m1.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T6.12.12.12.12.12.m1.2.3.1b" xref="A4.T6.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mi id="A4.T6.12.12.12.12.12.m1.2.3.5" xref="A4.T6.12.12.12.12.12.m1.2.3.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T6.12.12.12.12.12.m1.2.3.1c" xref="A4.T6.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mrow id="A4.T6.12.12.12.12.12.m1.2.3.6.2" xref="A4.T6.12.12.12.12.12.m1.2.3.6.1.cmml"><mo stretchy="false" id="A4.T6.12.12.12.12.12.m1.2.3.6.2.1" xref="A4.T6.12.12.12.12.12.m1.2.3.6.1.cmml">(</mo><mi id="A4.T6.12.12.12.12.12.m1.1.1" xref="A4.T6.12.12.12.12.12.m1.1.1.cmml">x</mi><mo id="A4.T6.12.12.12.12.12.m1.2.3.6.2.2" xref="A4.T6.12.12.12.12.12.m1.2.3.6.1.cmml">,</mo><mi id="A4.T6.12.12.12.12.12.m1.2.2" xref="A4.T6.12.12.12.12.12.m1.2.2.cmml">y</mi><mo stretchy="false" id="A4.T6.12.12.12.12.12.m1.2.3.6.2.3" xref="A4.T6.12.12.12.12.12.m1.2.3.6.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.12.12.12.12.12.m1.2b"><apply id="A4.T6.12.12.12.12.12.m1.2.3.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3"><times id="A4.T6.12.12.12.12.12.m1.2.3.1.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3.1"></times><ci id="A4.T6.12.12.12.12.12.m1.2.3.2.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3.2">𝑑</ci><ci id="A4.T6.12.12.12.12.12.m1.2.3.3.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3.3">𝑖</ci><ci id="A4.T6.12.12.12.12.12.m1.2.3.4.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3.4">𝑠</ci><ci id="A4.T6.12.12.12.12.12.m1.2.3.5.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3.5">𝑡</ci><interval closure="open" id="A4.T6.12.12.12.12.12.m1.2.3.6.1.cmml" xref="A4.T6.12.12.12.12.12.m1.2.3.6.2"><ci id="A4.T6.12.12.12.12.12.m1.1.1.cmml" xref="A4.T6.12.12.12.12.12.m1.1.1">𝑥</ci><ci id="A4.T6.12.12.12.12.12.m1.2.2.cmml" xref="A4.T6.12.12.12.12.12.m1.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.12.12.12.12.12.m1.2c">dist(x,y)</annotation></semantics></math> among such pairs of vertices that <math id="A4.T6.13.13.13.13.13.m2.2" class="ltx_Math" alttext="g(x,y)&gt;1" display="inline"><semantics id="A4.T6.13.13.13.13.13.m2.2a"><mrow id="A4.T6.13.13.13.13.13.m2.2.3" xref="A4.T6.13.13.13.13.13.m2.2.3.cmml"><mrow id="A4.T6.13.13.13.13.13.m2.2.3.2" xref="A4.T6.13.13.13.13.13.m2.2.3.2.cmml"><mi id="A4.T6.13.13.13.13.13.m2.2.3.2.2" xref="A4.T6.13.13.13.13.13.m2.2.3.2.2.cmml">g</mi><mo lspace="0em" rspace="0em" id="A4.T6.13.13.13.13.13.m2.2.3.2.1" xref="A4.T6.13.13.13.13.13.m2.2.3.2.1.cmml">​</mo><mrow id="A4.T6.13.13.13.13.13.m2.2.3.2.3.2" xref="A4.T6.13.13.13.13.13.m2.2.3.2.3.1.cmml"><mo stretchy="false" id="A4.T6.13.13.13.13.13.m2.2.3.2.3.2.1" xref="A4.T6.13.13.13.13.13.m2.2.3.2.3.1.cmml">(</mo><mi id="A4.T6.13.13.13.13.13.m2.1.1" xref="A4.T6.13.13.13.13.13.m2.1.1.cmml">x</mi><mo id="A4.T6.13.13.13.13.13.m2.2.3.2.3.2.2" xref="A4.T6.13.13.13.13.13.m2.2.3.2.3.1.cmml">,</mo><mi id="A4.T6.13.13.13.13.13.m2.2.2" xref="A4.T6.13.13.13.13.13.m2.2.2.cmml">y</mi><mo stretchy="false" id="A4.T6.13.13.13.13.13.m2.2.3.2.3.2.3" xref="A4.T6.13.13.13.13.13.m2.2.3.2.3.1.cmml">)</mo></mrow></mrow><mo id="A4.T6.13.13.13.13.13.m2.2.3.1" xref="A4.T6.13.13.13.13.13.m2.2.3.1.cmml">&gt;</mo><mn id="A4.T6.13.13.13.13.13.m2.2.3.3" xref="A4.T6.13.13.13.13.13.m2.2.3.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.13.13.13.13.13.m2.2b"><apply id="A4.T6.13.13.13.13.13.m2.2.3.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3"><gt id="A4.T6.13.13.13.13.13.m2.2.3.1.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3.1"></gt><apply id="A4.T6.13.13.13.13.13.m2.2.3.2.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3.2"><times id="A4.T6.13.13.13.13.13.m2.2.3.2.1.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3.2.1"></times><ci id="A4.T6.13.13.13.13.13.m2.2.3.2.2.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3.2.2">𝑔</ci><interval closure="open" id="A4.T6.13.13.13.13.13.m2.2.3.2.3.1.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3.2.3.2"><ci id="A4.T6.13.13.13.13.13.m2.1.1.cmml" xref="A4.T6.13.13.13.13.13.m2.1.1">𝑥</ci><ci id="A4.T6.13.13.13.13.13.m2.2.2.cmml" xref="A4.T6.13.13.13.13.13.m2.2.2">𝑦</ci></interval></apply><cn type="integer" id="A4.T6.13.13.13.13.13.m2.2.3.3.cmml" xref="A4.T6.13.13.13.13.13.m2.2.3.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.13.13.13.13.13.m2.2c">g(x,y)&gt;1</annotation></semantics></math>.</span>
<span id="A4.T6.30.30.30.30.31" class="ltx_p">—–Input—–</span>
<span id="A4.T6.15.15.15.15.15" class="ltx_p">The first line contains one integer <math id="A4.T6.14.14.14.14.14.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A4.T6.14.14.14.14.14.m1.1a"><mi id="A4.T6.14.14.14.14.14.m1.1.1" xref="A4.T6.14.14.14.14.14.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A4.T6.14.14.14.14.14.m1.1b"><ci id="A4.T6.14.14.14.14.14.m1.1.1.cmml" xref="A4.T6.14.14.14.14.14.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.14.14.14.14.14.m1.1c">n</annotation></semantics></math> — the number of vertices <math id="A4.T6.15.15.15.15.15.m2.1" class="ltx_Math" alttext="(1\leq n\leq 2\cdot 10^{5})" display="inline"><semantics id="A4.T6.15.15.15.15.15.m2.1a"><mrow id="A4.T6.15.15.15.15.15.m2.1.1.1" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.cmml"><mo stretchy="false" id="A4.T6.15.15.15.15.15.m2.1.1.1.2" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.cmml">(</mo><mrow id="A4.T6.15.15.15.15.15.m2.1.1.1.1" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.cmml"><mn id="A4.T6.15.15.15.15.15.m2.1.1.1.1.2" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.2.cmml">1</mn><mo id="A4.T6.15.15.15.15.15.m2.1.1.1.1.3" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.3.cmml">≤</mo><mi id="A4.T6.15.15.15.15.15.m2.1.1.1.1.4" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.4.cmml">n</mi><mo id="A4.T6.15.15.15.15.15.m2.1.1.1.1.5" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.5.cmml">≤</mo><mrow id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.cmml"><mn id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.2" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.2.cmml">2</mn><mo lspace="0.222em" rspace="0.222em" id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.1" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.1.cmml">⋅</mo><msup id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.cmml"><mn id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.2" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.2.cmml">10</mn><mn id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.3" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.3.cmml">5</mn></msup></mrow></mrow><mo stretchy="false" id="A4.T6.15.15.15.15.15.m2.1.1.1.3" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.15.15.15.15.15.m2.1b"><apply id="A4.T6.15.15.15.15.15.m2.1.1.1.1.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1"><and id="A4.T6.15.15.15.15.15.m2.1.1.1.1a.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1"></and><apply id="A4.T6.15.15.15.15.15.m2.1.1.1.1b.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1"><leq id="A4.T6.15.15.15.15.15.m2.1.1.1.1.3.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.3"></leq><cn type="integer" id="A4.T6.15.15.15.15.15.m2.1.1.1.1.2.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.2">1</cn><ci id="A4.T6.15.15.15.15.15.m2.1.1.1.1.4.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.4">𝑛</ci></apply><apply id="A4.T6.15.15.15.15.15.m2.1.1.1.1c.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1"><leq id="A4.T6.15.15.15.15.15.m2.1.1.1.1.5.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.5"></leq><share href="#A4.T6.15.15.15.15.15.m2.1.1.1.1.4.cmml" id="A4.T6.15.15.15.15.15.m2.1.1.1.1d.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1"></share><apply id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6"><ci id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.1.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.1">⋅</ci><cn type="integer" id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.2.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.2">2</cn><apply id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3"><csymbol cd="ambiguous" id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.1.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3">superscript</csymbol><cn type="integer" id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.2.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.2">10</cn><cn type="integer" id="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.3.cmml" xref="A4.T6.15.15.15.15.15.m2.1.1.1.1.6.3.3">5</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.15.15.15.15.15.m2.1c">(1\leq n\leq 2\cdot 10^{5})</annotation></semantics></math>.</span>
<span id="A4.T6.20.20.20.20.20" class="ltx_p">The second line contains <math id="A4.T6.16.16.16.16.16.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A4.T6.16.16.16.16.16.m1.1a"><mi id="A4.T6.16.16.16.16.16.m1.1.1" xref="A4.T6.16.16.16.16.16.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A4.T6.16.16.16.16.16.m1.1b"><ci id="A4.T6.16.16.16.16.16.m1.1.1.cmml" xref="A4.T6.16.16.16.16.16.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.16.16.16.16.16.m1.1c">n</annotation></semantics></math> integers <math id="A4.T6.17.17.17.17.17.m2.1" class="ltx_Math" alttext="a_{1}" display="inline"><semantics id="A4.T6.17.17.17.17.17.m2.1a"><msub id="A4.T6.17.17.17.17.17.m2.1.1" xref="A4.T6.17.17.17.17.17.m2.1.1.cmml"><mi id="A4.T6.17.17.17.17.17.m2.1.1.2" xref="A4.T6.17.17.17.17.17.m2.1.1.2.cmml">a</mi><mn id="A4.T6.17.17.17.17.17.m2.1.1.3" xref="A4.T6.17.17.17.17.17.m2.1.1.3.cmml">1</mn></msub><annotation-xml encoding="MathML-Content" id="A4.T6.17.17.17.17.17.m2.1b"><apply id="A4.T6.17.17.17.17.17.m2.1.1.cmml" xref="A4.T6.17.17.17.17.17.m2.1.1"><csymbol cd="ambiguous" id="A4.T6.17.17.17.17.17.m2.1.1.1.cmml" xref="A4.T6.17.17.17.17.17.m2.1.1">subscript</csymbol><ci id="A4.T6.17.17.17.17.17.m2.1.1.2.cmml" xref="A4.T6.17.17.17.17.17.m2.1.1.2">𝑎</ci><cn type="integer" id="A4.T6.17.17.17.17.17.m2.1.1.3.cmml" xref="A4.T6.17.17.17.17.17.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.17.17.17.17.17.m2.1c">a_{1}</annotation></semantics></math>, <math id="A4.T6.18.18.18.18.18.m3.1" class="ltx_Math" alttext="a_{2}" display="inline"><semantics id="A4.T6.18.18.18.18.18.m3.1a"><msub id="A4.T6.18.18.18.18.18.m3.1.1" xref="A4.T6.18.18.18.18.18.m3.1.1.cmml"><mi id="A4.T6.18.18.18.18.18.m3.1.1.2" xref="A4.T6.18.18.18.18.18.m3.1.1.2.cmml">a</mi><mn id="A4.T6.18.18.18.18.18.m3.1.1.3" xref="A4.T6.18.18.18.18.18.m3.1.1.3.cmml">2</mn></msub><annotation-xml encoding="MathML-Content" id="A4.T6.18.18.18.18.18.m3.1b"><apply id="A4.T6.18.18.18.18.18.m3.1.1.cmml" xref="A4.T6.18.18.18.18.18.m3.1.1"><csymbol cd="ambiguous" id="A4.T6.18.18.18.18.18.m3.1.1.1.cmml" xref="A4.T6.18.18.18.18.18.m3.1.1">subscript</csymbol><ci id="A4.T6.18.18.18.18.18.m3.1.1.2.cmml" xref="A4.T6.18.18.18.18.18.m3.1.1.2">𝑎</ci><cn type="integer" id="A4.T6.18.18.18.18.18.m3.1.1.3.cmml" xref="A4.T6.18.18.18.18.18.m3.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.18.18.18.18.18.m3.1c">a_{2}</annotation></semantics></math>, …, <math id="A4.T6.19.19.19.19.19.m4.1" class="ltx_Math" alttext="a_{n}" display="inline"><semantics id="A4.T6.19.19.19.19.19.m4.1a"><msub id="A4.T6.19.19.19.19.19.m4.1.1" xref="A4.T6.19.19.19.19.19.m4.1.1.cmml"><mi id="A4.T6.19.19.19.19.19.m4.1.1.2" xref="A4.T6.19.19.19.19.19.m4.1.1.2.cmml">a</mi><mi id="A4.T6.19.19.19.19.19.m4.1.1.3" xref="A4.T6.19.19.19.19.19.m4.1.1.3.cmml">n</mi></msub><annotation-xml encoding="MathML-Content" id="A4.T6.19.19.19.19.19.m4.1b"><apply id="A4.T6.19.19.19.19.19.m4.1.1.cmml" xref="A4.T6.19.19.19.19.19.m4.1.1"><csymbol cd="ambiguous" id="A4.T6.19.19.19.19.19.m4.1.1.1.cmml" xref="A4.T6.19.19.19.19.19.m4.1.1">subscript</csymbol><ci id="A4.T6.19.19.19.19.19.m4.1.1.2.cmml" xref="A4.T6.19.19.19.19.19.m4.1.1.2">𝑎</ci><ci id="A4.T6.19.19.19.19.19.m4.1.1.3.cmml" xref="A4.T6.19.19.19.19.19.m4.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.19.19.19.19.19.m4.1c">a_{n}</annotation></semantics></math> <math id="A4.T6.20.20.20.20.20.m5.1" class="ltx_Math" alttext="(1\leq a_{i}\leq 2\cdot 10^{5})" display="inline"><semantics id="A4.T6.20.20.20.20.20.m5.1a"><mrow id="A4.T6.20.20.20.20.20.m5.1.1.1" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.cmml"><mo stretchy="false" id="A4.T6.20.20.20.20.20.m5.1.1.1.2" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.cmml">(</mo><mrow id="A4.T6.20.20.20.20.20.m5.1.1.1.1" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.cmml"><mn id="A4.T6.20.20.20.20.20.m5.1.1.1.1.2" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.2.cmml">1</mn><mo id="A4.T6.20.20.20.20.20.m5.1.1.1.1.3" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.3.cmml">≤</mo><msub id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.cmml"><mi id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.2" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.2.cmml">a</mi><mi id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.3" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.3.cmml">i</mi></msub><mo id="A4.T6.20.20.20.20.20.m5.1.1.1.1.5" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.5.cmml">≤</mo><mrow id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.cmml"><mn id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.2" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.2.cmml">2</mn><mo lspace="0.222em" rspace="0.222em" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.1" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.1.cmml">⋅</mo><msup id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.cmml"><mn id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.2" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.2.cmml">10</mn><mn id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.3" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.3.cmml">5</mn></msup></mrow></mrow><mo stretchy="false" id="A4.T6.20.20.20.20.20.m5.1.1.1.3" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.20.20.20.20.20.m5.1b"><apply id="A4.T6.20.20.20.20.20.m5.1.1.1.1.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1"><and id="A4.T6.20.20.20.20.20.m5.1.1.1.1a.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1"></and><apply id="A4.T6.20.20.20.20.20.m5.1.1.1.1b.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1"><leq id="A4.T6.20.20.20.20.20.m5.1.1.1.1.3.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.3"></leq><cn type="integer" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.2.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.2">1</cn><apply id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4"><csymbol cd="ambiguous" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.1.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4">subscript</csymbol><ci id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.2.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.2">𝑎</ci><ci id="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.3.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.4.3">𝑖</ci></apply></apply><apply id="A4.T6.20.20.20.20.20.m5.1.1.1.1c.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1"><leq id="A4.T6.20.20.20.20.20.m5.1.1.1.1.5.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.5"></leq><share href="#A4.T6.20.20.20.20.20.m5.1.1.1.1.4.cmml" id="A4.T6.20.20.20.20.20.m5.1.1.1.1d.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1"></share><apply id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6"><ci id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.1.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.1">⋅</ci><cn type="integer" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.2.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.2">2</cn><apply id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3"><csymbol cd="ambiguous" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.1.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3">superscript</csymbol><cn type="integer" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.2.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.2">10</cn><cn type="integer" id="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.3.cmml" xref="A4.T6.20.20.20.20.20.m5.1.1.1.1.6.3.3">5</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.20.20.20.20.20.m5.1c">(1\leq a_{i}\leq 2\cdot 10^{5})</annotation></semantics></math> — the numbers written on vertices.</span>
<span id="A4.T6.26.26.26.26.26" class="ltx_p">Then <math id="A4.T6.21.21.21.21.21.m1.1" class="ltx_Math" alttext="n-1" display="inline"><semantics id="A4.T6.21.21.21.21.21.m1.1a"><mrow id="A4.T6.21.21.21.21.21.m1.1.1" xref="A4.T6.21.21.21.21.21.m1.1.1.cmml"><mi id="A4.T6.21.21.21.21.21.m1.1.1.2" xref="A4.T6.21.21.21.21.21.m1.1.1.2.cmml">n</mi><mo id="A4.T6.21.21.21.21.21.m1.1.1.1" xref="A4.T6.21.21.21.21.21.m1.1.1.1.cmml">−</mo><mn id="A4.T6.21.21.21.21.21.m1.1.1.3" xref="A4.T6.21.21.21.21.21.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.21.21.21.21.21.m1.1b"><apply id="A4.T6.21.21.21.21.21.m1.1.1.cmml" xref="A4.T6.21.21.21.21.21.m1.1.1"><minus id="A4.T6.21.21.21.21.21.m1.1.1.1.cmml" xref="A4.T6.21.21.21.21.21.m1.1.1.1"></minus><ci id="A4.T6.21.21.21.21.21.m1.1.1.2.cmml" xref="A4.T6.21.21.21.21.21.m1.1.1.2">𝑛</ci><cn type="integer" id="A4.T6.21.21.21.21.21.m1.1.1.3.cmml" xref="A4.T6.21.21.21.21.21.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.21.21.21.21.21.m1.1c">n-1</annotation></semantics></math> lines follow, each containing two integers <math id="A4.T6.22.22.22.22.22.m2.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T6.22.22.22.22.22.m2.1a"><mi id="A4.T6.22.22.22.22.22.m2.1.1" xref="A4.T6.22.22.22.22.22.m2.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T6.22.22.22.22.22.m2.1b"><ci id="A4.T6.22.22.22.22.22.m2.1.1.cmml" xref="A4.T6.22.22.22.22.22.m2.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.22.22.22.22.22.m2.1c">x</annotation></semantics></math> and <math id="A4.T6.23.23.23.23.23.m3.1" class="ltx_Math" alttext="y" display="inline"><semantics id="A4.T6.23.23.23.23.23.m3.1a"><mi id="A4.T6.23.23.23.23.23.m3.1.1" xref="A4.T6.23.23.23.23.23.m3.1.1.cmml">y</mi><annotation-xml encoding="MathML-Content" id="A4.T6.23.23.23.23.23.m3.1b"><ci id="A4.T6.23.23.23.23.23.m3.1.1.cmml" xref="A4.T6.23.23.23.23.23.m3.1.1">𝑦</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.23.23.23.23.23.m3.1c">y</annotation></semantics></math> <math id="A4.T6.24.24.24.24.24.m4.1" class="ltx_Math" alttext="(1\leq x,y\leq n,x\neq y)" display="inline"><semantics id="A4.T6.24.24.24.24.24.m4.1a"><mrow id="A4.T6.24.24.24.24.24.m4.1.1.1"><mo stretchy="false" id="A4.T6.24.24.24.24.24.m4.1.1.1.2">(</mo><mrow id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.3.cmml"><mrow id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.cmml"><mn id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.2" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.2.cmml">1</mn><mo id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.1" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.1.cmml">≤</mo><mi id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.3" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.3.cmml">x</mi></mrow><mo id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.3" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.3a.cmml">,</mo><mrow id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.3.cmml"><mrow id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.cmml"><mi id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.2" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.2.cmml">y</mi><mo id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.1" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.1.cmml">≤</mo><mi id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.3" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.3.cmml">n</mi></mrow><mo id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.3" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.3a.cmml">,</mo><mrow id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.cmml"><mi id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.2" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.2.cmml">x</mi><mo id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.1" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.1.cmml">≠</mo><mi id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.3" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.3.cmml">y</mi></mrow></mrow></mrow><mo stretchy="false" id="A4.T6.24.24.24.24.24.m4.1.1.1.3">)</mo></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.24.24.24.24.24.m4.1b"><apply id="A4.T6.24.24.24.24.24.m4.1.1.1.1.3.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2"><csymbol cd="ambiguous" id="A4.T6.24.24.24.24.24.m4.1.1.1.1.3a.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.3">formulae-sequence</csymbol><apply id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1"><leq id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.1.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.1"></leq><cn type="integer" id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.2.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.2">1</cn><ci id="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.3.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.1.1.3">𝑥</ci></apply><apply id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.3.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2"><csymbol cd="ambiguous" id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.3a.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.3">formulae-sequence</csymbol><apply id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1"><leq id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.1.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.1"></leq><ci id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.2.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.2">𝑦</ci><ci id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.3.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.1.1.3">𝑛</ci></apply><apply id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2"><neq id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.1.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.1"></neq><ci id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.2.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.2">𝑥</ci><ci id="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.3.cmml" xref="A4.T6.24.24.24.24.24.m4.1.1.1.1.2.2.2.2.3">𝑦</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.24.24.24.24.24.m4.1c">(1\leq x,y\leq n,x\neq y)</annotation></semantics></math> denoting an edge connecting vertex <math id="A4.T6.25.25.25.25.25.m5.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T6.25.25.25.25.25.m5.1a"><mi id="A4.T6.25.25.25.25.25.m5.1.1" xref="A4.T6.25.25.25.25.25.m5.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T6.25.25.25.25.25.m5.1b"><ci id="A4.T6.25.25.25.25.25.m5.1.1.cmml" xref="A4.T6.25.25.25.25.25.m5.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.25.25.25.25.25.m5.1c">x</annotation></semantics></math> with vertex <math id="A4.T6.26.26.26.26.26.m6.1" class="ltx_Math" alttext="y" display="inline"><semantics id="A4.T6.26.26.26.26.26.m6.1a"><mi id="A4.T6.26.26.26.26.26.m6.1.1" xref="A4.T6.26.26.26.26.26.m6.1.1.cmml">y</mi><annotation-xml encoding="MathML-Content" id="A4.T6.26.26.26.26.26.m6.1b"><ci id="A4.T6.26.26.26.26.26.m6.1.1.cmml" xref="A4.T6.26.26.26.26.26.m6.1.1">𝑦</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.26.26.26.26.26.m6.1c">y</annotation></semantics></math>. It is guaranteed that these edges form a tree.</span>
<span id="A4.T6.30.30.30.30.32" class="ltx_p">—–Output—–</span>
<span id="A4.T6.30.30.30.30.30" class="ltx_p">If there is no pair of vertices <math id="A4.T6.27.27.27.27.27.m1.2" class="ltx_Math" alttext="x,y" display="inline"><semantics id="A4.T6.27.27.27.27.27.m1.2a"><mrow id="A4.T6.27.27.27.27.27.m1.2.3.2" xref="A4.T6.27.27.27.27.27.m1.2.3.1.cmml"><mi id="A4.T6.27.27.27.27.27.m1.1.1" xref="A4.T6.27.27.27.27.27.m1.1.1.cmml">x</mi><mo id="A4.T6.27.27.27.27.27.m1.2.3.2.1" xref="A4.T6.27.27.27.27.27.m1.2.3.1.cmml">,</mo><mi id="A4.T6.27.27.27.27.27.m1.2.2" xref="A4.T6.27.27.27.27.27.m1.2.2.cmml">y</mi></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.27.27.27.27.27.m1.2b"><list id="A4.T6.27.27.27.27.27.m1.2.3.1.cmml" xref="A4.T6.27.27.27.27.27.m1.2.3.2"><ci id="A4.T6.27.27.27.27.27.m1.1.1.cmml" xref="A4.T6.27.27.27.27.27.m1.1.1">𝑥</ci><ci id="A4.T6.27.27.27.27.27.m1.2.2.cmml" xref="A4.T6.27.27.27.27.27.m1.2.2">𝑦</ci></list></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.27.27.27.27.27.m1.2c">x,y</annotation></semantics></math> such that <math id="A4.T6.28.28.28.28.28.m2.2" class="ltx_Math" alttext="g(x,y)&gt;1" display="inline"><semantics id="A4.T6.28.28.28.28.28.m2.2a"><mrow id="A4.T6.28.28.28.28.28.m2.2.3" xref="A4.T6.28.28.28.28.28.m2.2.3.cmml"><mrow id="A4.T6.28.28.28.28.28.m2.2.3.2" xref="A4.T6.28.28.28.28.28.m2.2.3.2.cmml"><mi id="A4.T6.28.28.28.28.28.m2.2.3.2.2" xref="A4.T6.28.28.28.28.28.m2.2.3.2.2.cmml">g</mi><mo lspace="0em" rspace="0em" id="A4.T6.28.28.28.28.28.m2.2.3.2.1" xref="A4.T6.28.28.28.28.28.m2.2.3.2.1.cmml">​</mo><mrow id="A4.T6.28.28.28.28.28.m2.2.3.2.3.2" xref="A4.T6.28.28.28.28.28.m2.2.3.2.3.1.cmml"><mo stretchy="false" id="A4.T6.28.28.28.28.28.m2.2.3.2.3.2.1" xref="A4.T6.28.28.28.28.28.m2.2.3.2.3.1.cmml">(</mo><mi id="A4.T6.28.28.28.28.28.m2.1.1" xref="A4.T6.28.28.28.28.28.m2.1.1.cmml">x</mi><mo id="A4.T6.28.28.28.28.28.m2.2.3.2.3.2.2" xref="A4.T6.28.28.28.28.28.m2.2.3.2.3.1.cmml">,</mo><mi id="A4.T6.28.28.28.28.28.m2.2.2" xref="A4.T6.28.28.28.28.28.m2.2.2.cmml">y</mi><mo stretchy="false" id="A4.T6.28.28.28.28.28.m2.2.3.2.3.2.3" xref="A4.T6.28.28.28.28.28.m2.2.3.2.3.1.cmml">)</mo></mrow></mrow><mo id="A4.T6.28.28.28.28.28.m2.2.3.1" xref="A4.T6.28.28.28.28.28.m2.2.3.1.cmml">&gt;</mo><mn id="A4.T6.28.28.28.28.28.m2.2.3.3" xref="A4.T6.28.28.28.28.28.m2.2.3.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.28.28.28.28.28.m2.2b"><apply id="A4.T6.28.28.28.28.28.m2.2.3.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3"><gt id="A4.T6.28.28.28.28.28.m2.2.3.1.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3.1"></gt><apply id="A4.T6.28.28.28.28.28.m2.2.3.2.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3.2"><times id="A4.T6.28.28.28.28.28.m2.2.3.2.1.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3.2.1"></times><ci id="A4.T6.28.28.28.28.28.m2.2.3.2.2.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3.2.2">𝑔</ci><interval closure="open" id="A4.T6.28.28.28.28.28.m2.2.3.2.3.1.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3.2.3.2"><ci id="A4.T6.28.28.28.28.28.m2.1.1.cmml" xref="A4.T6.28.28.28.28.28.m2.1.1">𝑥</ci><ci id="A4.T6.28.28.28.28.28.m2.2.2.cmml" xref="A4.T6.28.28.28.28.28.m2.2.2">𝑦</ci></interval></apply><cn type="integer" id="A4.T6.28.28.28.28.28.m2.2.3.3.cmml" xref="A4.T6.28.28.28.28.28.m2.2.3.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.28.28.28.28.28.m2.2c">g(x,y)&gt;1</annotation></semantics></math>, print <math id="A4.T6.29.29.29.29.29.m3.1" class="ltx_Math" alttext="0" display="inline"><semantics id="A4.T6.29.29.29.29.29.m3.1a"><mn id="A4.T6.29.29.29.29.29.m3.1.1" xref="A4.T6.29.29.29.29.29.m3.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="A4.T6.29.29.29.29.29.m3.1b"><cn type="integer" id="A4.T6.29.29.29.29.29.m3.1.1.cmml" xref="A4.T6.29.29.29.29.29.m3.1.1">0</cn></annotation-xml></semantics></math>. Otherwise print the maximum value of <math id="A4.T6.30.30.30.30.30.m4.2" class="ltx_Math" alttext="dist(x,y)" display="inline"><semantics id="A4.T6.30.30.30.30.30.m4.2a"><mrow id="A4.T6.30.30.30.30.30.m4.2.3" xref="A4.T6.30.30.30.30.30.m4.2.3.cmml"><mi id="A4.T6.30.30.30.30.30.m4.2.3.2" xref="A4.T6.30.30.30.30.30.m4.2.3.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T6.30.30.30.30.30.m4.2.3.1" xref="A4.T6.30.30.30.30.30.m4.2.3.1.cmml">​</mo><mi id="A4.T6.30.30.30.30.30.m4.2.3.3" xref="A4.T6.30.30.30.30.30.m4.2.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T6.30.30.30.30.30.m4.2.3.1a" xref="A4.T6.30.30.30.30.30.m4.2.3.1.cmml">​</mo><mi id="A4.T6.30.30.30.30.30.m4.2.3.4" xref="A4.T6.30.30.30.30.30.m4.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T6.30.30.30.30.30.m4.2.3.1b" xref="A4.T6.30.30.30.30.30.m4.2.3.1.cmml">​</mo><mi id="A4.T6.30.30.30.30.30.m4.2.3.5" xref="A4.T6.30.30.30.30.30.m4.2.3.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T6.30.30.30.30.30.m4.2.3.1c" xref="A4.T6.30.30.30.30.30.m4.2.3.1.cmml">​</mo><mrow id="A4.T6.30.30.30.30.30.m4.2.3.6.2" xref="A4.T6.30.30.30.30.30.m4.2.3.6.1.cmml"><mo stretchy="false" id="A4.T6.30.30.30.30.30.m4.2.3.6.2.1" xref="A4.T6.30.30.30.30.30.m4.2.3.6.1.cmml">(</mo><mi id="A4.T6.30.30.30.30.30.m4.1.1" xref="A4.T6.30.30.30.30.30.m4.1.1.cmml">x</mi><mo id="A4.T6.30.30.30.30.30.m4.2.3.6.2.2" xref="A4.T6.30.30.30.30.30.m4.2.3.6.1.cmml">,</mo><mi id="A4.T6.30.30.30.30.30.m4.2.2" xref="A4.T6.30.30.30.30.30.m4.2.2.cmml">y</mi><mo stretchy="false" id="A4.T6.30.30.30.30.30.m4.2.3.6.2.3" xref="A4.T6.30.30.30.30.30.m4.2.3.6.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T6.30.30.30.30.30.m4.2b"><apply id="A4.T6.30.30.30.30.30.m4.2.3.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3"><times id="A4.T6.30.30.30.30.30.m4.2.3.1.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3.1"></times><ci id="A4.T6.30.30.30.30.30.m4.2.3.2.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3.2">𝑑</ci><ci id="A4.T6.30.30.30.30.30.m4.2.3.3.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3.3">𝑖</ci><ci id="A4.T6.30.30.30.30.30.m4.2.3.4.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3.4">𝑠</ci><ci id="A4.T6.30.30.30.30.30.m4.2.3.5.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3.5">𝑡</ci><interval closure="open" id="A4.T6.30.30.30.30.30.m4.2.3.6.1.cmml" xref="A4.T6.30.30.30.30.30.m4.2.3.6.2"><ci id="A4.T6.30.30.30.30.30.m4.1.1.cmml" xref="A4.T6.30.30.30.30.30.m4.1.1">𝑥</ci><ci id="A4.T6.30.30.30.30.30.m4.2.2.cmml" xref="A4.T6.30.30.30.30.30.m4.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T6.30.30.30.30.30.m4.2c">dist(x,y)</annotation></semantics></math> among such pairs…[TRUNCATED]</span>
</span>
</td>
</tr>
<tr id="A4.T6.30.32" class="ltx_tr">
<td id="A4.T6.30.32.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T6.30.32.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T6.30.32.1.1.1" class="ltx_p" style="width:411.9pt;"><span id="A4.T6.30.32.1.1.1.1" class="ltx_text ltx_font_bold">Solution</span>

<span id="A4.T6.30.32.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,aW1wb3J0IHN5cwoKYW5zd2VyID0gMQp6ID0gVHJ1ZQpwcmltZXMgPSBbXQpmb3IgaSBpbiByYW5nZSAoMiwgNSAqIDEwICoqIDIpOgogICAgdiA9IFRydWUKICAgIGZvciBwIGluIHByaW1lczoKICAgICAgICBpZiBpIFwlIHAgPT0gMDoKICAgICAgICAgICAgdiA9IEZhbHNlCiAgICBpZiB2ID09IFRydWU6CiAgICAgICAgcHJpbWVzLmFwcGVuZChpKQpuID0gaW50KHN5cy5zdGRpbi5yZWFkbGluZSgpLnN0cmlwKCkpCmEgPSBsaXN0KG1hcChpbnQsIHN5cy5zdGRpbi5yZWFkbGluZSgpLnN0cmlwKCkuc3BsaXQoKSkpCmlmIHN1bShhKSA9PSBuOgogICAgeiA9IEZhbHNlCmZvciBpIGluIHJhbmdlICgwLCBuKToKICAgIHggPSBhW2ldCiAgICBhW2ldID0gW10KICAgIGZvciBwIGluIHByaW1lczoKICAgICAgICBpZiB4IFwlIHAgPT0gMDoKICAgICAgICAgICAgYVtpXS5hcHBlbmQoW3AsIDFdKSBcXAogICAgICAgICAgICB4ID0geCAvLyBwCiAgICAgICAgICAgIHdoaWxlIHggXCUgcCA9PSAwOgogICAgICAgICAgICAgICAgeCA9IHggLy8gcAogICAgaWYgeCAhPSAxOgogICAgICAgIGFbaV0uYXBwZW5kKFt4LCAxXSkKbmVpZ2hib3VycyA9IFtbXSBmb3IgaSBpbiByYW5nZSAoMCwgbildCmZvciBpIGluIHJhbmdlICgwLCBuIC0gMSk6CiAgICBsaW5lID0gc3lzLnN0ZGluLnJlYWRsaW5lKCkuc3RyaXAoKS5zcGxpdCgpCiAgICBuZWlnaGJvdXJzW2ludChsaW5lWzBdKSAtIDFdLmFwcGVuZChpbnQobGluZVsxXSkgLSAxKQogICAgbmVpZ2hib3Vyc1tpbnQobGluZVsxXSkgLSAxXS5hcHBlbmQoaW50KGxpbmVbMF0pIC0gMSkKbGVhdmVzID0gW10KZm9yIGkgaW4gcmFuZ2UgKDAsIG4pOgogICAgaWYgbGVuKG5laWdoYm91cnNbaV0pID09IDE6CiAgICAgICAgbGVhdmVzLmFwcGVuZChpKQp3aGlsZSBsZW4obGVhdmVzKSA+IDE6CiAgICB4ID0gbGVhdmVzLnBvcCgpCiAgICB5ID0gbmVpZ2hib3Vyc1t4XVswXQogICAgbmVpZ2hib3Vyc1t5XS5yZW1vdmUoeCkKICAgIGlmIGxlbihuZWlnaGJvdXJzW3ldKSA9PSAxOgogICAgICAgIGxlYXZlcy5hcHBlbmQoeSkKICAgIGZvciBwIGluIGFbeF06CiAgICAgICAgZm9yIHEgaW4gYVt5XToKICAgICAgICAgICAgaWYgcFswXSA9PSBxWzBdOgogICAgICAgICAgICAgICAgYW5zd2VyID0gbWF4KFthbnN3ZXIsIHBbMV0gKyBxWzFdXSkKICAgICAgICAgICAgICAgIHFbMV0gPSBtYXgoW3FbMV0scFsxXSsxXSkKaWYgeiA9PSBGYWxzZToKICAgIHByaW50KDApCmVsc2U6CiAgICBwcmludChhbnN3ZXIp" download="">⬇</a></span>
<span id="lstnumberx1" class="ltx_listingline"><span id="lstnumberx1.1" class="ltx_text ltx_lst_identifier">import</span><span id="lstnumberx1.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx1.3" class="ltx_text ltx_lst_identifier">sys</span>
</span>
<span id="lstnumberx2" class="ltx_listingline">
</span>
<span id="lstnumberx3" class="ltx_listingline"><span id="lstnumberx3.1" class="ltx_text ltx_lst_identifier">answer</span><span id="lstnumberx3.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx3.3" class="ltx_text ltx_lst_space"> </span>1
</span>
<span id="lstnumberx4" class="ltx_listingline"><span id="lstnumberx4.1" class="ltx_text ltx_lst_identifier">z</span><span id="lstnumberx4.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx4.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.4" class="ltx_text ltx_lst_identifier">True</span>
</span>
<span id="lstnumberx5" class="ltx_listingline"><span id="lstnumberx5.1" class="ltx_text ltx_lst_identifier">primes</span><span id="lstnumberx5.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx5.3" class="ltx_text ltx_lst_space"> </span>[]
</span>
<span id="lstnumberx6" class="ltx_listingline"><span id="lstnumberx6.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx6.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.3" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx6.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.5" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx6.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.7" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx6.8" class="ltx_text ltx_lst_space"> </span>(2,<span id="lstnumberx6.9" class="ltx_text ltx_lst_space"> </span>5<span id="lstnumberx6.10" class="ltx_text ltx_lst_space"> </span>*<span id="lstnumberx6.11" class="ltx_text ltx_lst_space"> </span>10<span id="lstnumberx6.12" class="ltx_text ltx_lst_space"> </span>**<span id="lstnumberx6.13" class="ltx_text ltx_lst_space"> </span>2):
</span>
<span id="lstnumberx7" class="ltx_listingline"><span id="lstnumberx7.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx7.2" class="ltx_text ltx_lst_identifier">v</span><span id="lstnumberx7.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx7.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx7.5" class="ltx_text ltx_lst_identifier">True</span>
</span>
<span id="lstnumberx8" class="ltx_listingline"><span id="lstnumberx8.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx8.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx8.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx8.4" class="ltx_text ltx_lst_identifier">p</span><span id="lstnumberx8.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx8.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx8.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx8.8" class="ltx_text ltx_lst_identifier">primes</span>:
</span>
<span id="lstnumberx9" class="ltx_listingline"><span id="lstnumberx9.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx9.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx9.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.4" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx9.5" class="ltx_text ltx_lst_space"> </span>\<span id="lstnumberx9.6" class="ltx_text ltx_lst_comment" style="color:#009900;">%<span id="lstnumberx9.6.1" class="ltx_text ltx_lst_space"> </span>p<span id="lstnumberx9.6.2" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx9.6.3" class="ltx_text ltx_lst_space"> </span>0:</span>
</span>
<span id="lstnumberx10" class="ltx_listingline"><span id="lstnumberx10.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx10.2" class="ltx_text ltx_lst_identifier">v</span><span id="lstnumberx10.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx10.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.5" class="ltx_text ltx_lst_identifier">False</span>
</span>
<span id="lstnumberx11" class="ltx_listingline"><span id="lstnumberx11.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx11.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx11.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.4" class="ltx_text ltx_lst_identifier">v</span><span id="lstnumberx11.5" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx11.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.7" class="ltx_text ltx_lst_identifier">True</span>:
</span>
<span id="lstnumberx12" class="ltx_listingline"><span id="lstnumberx12.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx12.2" class="ltx_text ltx_lst_identifier">primes</span>.<span id="lstnumberx12.3" class="ltx_text ltx_lst_identifier">append</span>(<span id="lstnumberx12.4" class="ltx_text ltx_lst_identifier">i</span>)
</span>
<span id="lstnumberx13" class="ltx_listingline"><span id="lstnumberx13.1" class="ltx_text ltx_lst_identifier">n</span><span id="lstnumberx13.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx13.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.4" class="ltx_text ltx_lst_identifier">int</span>(<span id="lstnumberx13.5" class="ltx_text ltx_lst_identifier">sys</span>.<span id="lstnumberx13.6" class="ltx_text ltx_lst_identifier">stdin</span>.<span id="lstnumberx13.7" class="ltx_text ltx_lst_identifier">readline</span>().<span id="lstnumberx13.8" class="ltx_text ltx_lst_identifier">strip</span>())
</span>
<span id="lstnumberx14" class="ltx_listingline"><span id="lstnumberx14.1" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx14.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx14.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.4" class="ltx_text ltx_lst_identifier">list</span>(<span id="lstnumberx14.5" class="ltx_text ltx_lst_identifier">map</span>(<span id="lstnumberx14.6" class="ltx_text ltx_lst_identifier">int</span>,<span id="lstnumberx14.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.8" class="ltx_text ltx_lst_identifier">sys</span>.<span id="lstnumberx14.9" class="ltx_text ltx_lst_identifier">stdin</span>.<span id="lstnumberx14.10" class="ltx_text ltx_lst_identifier">readline</span>().<span id="lstnumberx14.11" class="ltx_text ltx_lst_identifier">strip</span>().<span id="lstnumberx14.12" class="ltx_text ltx_lst_identifier">split</span>()))
</span>
<span id="lstnumberx15" class="ltx_listingline"><span id="lstnumberx15.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx15.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.3" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">sum</span>(<span id="lstnumberx15.4" class="ltx_text ltx_lst_identifier">a</span>)<span id="lstnumberx15.5" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx15.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.7" class="ltx_text ltx_lst_identifier">n</span>:
</span>
<span id="lstnumberx16" class="ltx_listingline"><span id="lstnumberx16.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx16.2" class="ltx_text ltx_lst_identifier">z</span><span id="lstnumberx16.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx16.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx16.5" class="ltx_text ltx_lst_identifier">False</span>
</span>
<span id="lstnumberx17" class="ltx_listingline"><span id="lstnumberx17.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx17.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.3" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx17.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.5" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx17.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.7" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx17.8" class="ltx_text ltx_lst_space"> </span>(0,<span id="lstnumberx17.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.10" class="ltx_text ltx_lst_identifier">n</span>):
</span>
<span id="lstnumberx18" class="ltx_listingline"><span id="lstnumberx18.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx18.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx18.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx18.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx18.5" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx18.6" class="ltx_text ltx_lst_identifier">i</span>]
</span>
<span id="lstnumberx19" class="ltx_listingline"><span id="lstnumberx19.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx19.2" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx19.3" class="ltx_text ltx_lst_identifier">i</span>]<span id="lstnumberx19.4" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx19.5" class="ltx_text ltx_lst_space"> </span>[]
</span>
<span id="lstnumberx20" class="ltx_listingline"><span id="lstnumberx20.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx20.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx20.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.4" class="ltx_text ltx_lst_identifier">p</span><span id="lstnumberx20.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx20.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.8" class="ltx_text ltx_lst_identifier">primes</span>:
</span>
<span id="lstnumberx21" class="ltx_listingline"><span id="lstnumberx21.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx21.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx21.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.4" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx21.5" class="ltx_text ltx_lst_space"> </span>\<span id="lstnumberx21.6" class="ltx_text ltx_lst_comment" style="color:#009900;">%<span id="lstnumberx21.6.1" class="ltx_text ltx_lst_space"> </span>p<span id="lstnumberx21.6.2" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx21.6.3" class="ltx_text ltx_lst_space"> </span>0:</span>
</span>
<span id="lstnumberx22" class="ltx_listingline"><span id="lstnumberx22.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx22.2" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx22.3" class="ltx_text ltx_lst_identifier">i</span>].<span id="lstnumberx22.4" class="ltx_text ltx_lst_identifier">append</span>([<span id="lstnumberx22.5" class="ltx_text ltx_lst_identifier">p</span>,<span id="lstnumberx22.6" class="ltx_text ltx_lst_space"> </span>1])<span id="lstnumberx22.7" class="ltx_text ltx_lst_space"> </span>\\
</span>
<span id="lstnumberx23" class="ltx_listingline"><span id="lstnumberx23.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx23.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx23.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx23.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.5" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx23.6" class="ltx_text ltx_lst_space"> </span>//<span id="lstnumberx23.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.8" class="ltx_text ltx_lst_identifier">p</span>
</span>
<span id="lstnumberx24" class="ltx_listingline"><span id="lstnumberx24.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx24.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">while</span><span id="lstnumberx24.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.4" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx24.5" class="ltx_text ltx_lst_space"> </span>\<span id="lstnumberx24.6" class="ltx_text ltx_lst_comment" style="color:#009900;">%<span id="lstnumberx24.6.1" class="ltx_text ltx_lst_space"> </span>p<span id="lstnumberx24.6.2" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx24.6.3" class="ltx_text ltx_lst_space"> </span>0:</span>
</span>
<span id="lstnumberx25" class="ltx_listingline"><span id="lstnumberx25.1" class="ltx_text ltx_lst_space">                </span><span id="lstnumberx25.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx25.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx25.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.5" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx25.6" class="ltx_text ltx_lst_space"> </span>//<span id="lstnumberx25.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.8" class="ltx_text ltx_lst_identifier">p</span>
</span>
<span id="lstnumberx26" class="ltx_listingline"><span id="lstnumberx26.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx26.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx26.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.4" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx26.5" class="ltx_text ltx_lst_space"> </span>!=<span id="lstnumberx26.6" class="ltx_text ltx_lst_space"> </span>1:
</span>
<span id="lstnumberx27" class="ltx_listingline"><span id="lstnumberx27.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx27.2" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx27.3" class="ltx_text ltx_lst_identifier">i</span>].<span id="lstnumberx27.4" class="ltx_text ltx_lst_identifier">append</span>([<span id="lstnumberx27.5" class="ltx_text ltx_lst_identifier">x</span>,<span id="lstnumberx27.6" class="ltx_text ltx_lst_space"> </span>1])
</span>
<span id="lstnumberx28" class="ltx_listingline"><span id="lstnumberx28.1" class="ltx_text ltx_lst_identifier">neighbours</span><span id="lstnumberx28.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx28.3" class="ltx_text ltx_lst_space"> </span>[[]<span id="lstnumberx28.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.5" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx28.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.7" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx28.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.9" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx28.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.11" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx28.12" class="ltx_text ltx_lst_space"> </span>(0,<span id="lstnumberx28.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.14" class="ltx_text ltx_lst_identifier">n</span>)]
</span>
<span id="lstnumberx29" class="ltx_listingline"><span id="lstnumberx29.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx29.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.3" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx29.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.5" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx29.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.7" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx29.8" class="ltx_text ltx_lst_space"> </span>(0,<span id="lstnumberx29.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.10" class="ltx_text ltx_lst_identifier">n</span><span id="lstnumberx29.11" class="ltx_text ltx_lst_space"> </span>-<span id="lstnumberx29.12" class="ltx_text ltx_lst_space"> </span>1):
</span>
<span id="lstnumberx30" class="ltx_listingline"><span id="lstnumberx30.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx30.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">line</span><span id="lstnumberx30.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx30.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx30.5" class="ltx_text ltx_lst_identifier">sys</span>.<span id="lstnumberx30.6" class="ltx_text ltx_lst_identifier">stdin</span>.<span id="lstnumberx30.7" class="ltx_text ltx_lst_identifier">readline</span>().<span id="lstnumberx30.8" class="ltx_text ltx_lst_identifier">strip</span>().<span id="lstnumberx30.9" class="ltx_text ltx_lst_identifier">split</span>()
</span>
<span id="lstnumberx31" class="ltx_listingline"><span id="lstnumberx31.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx31.2" class="ltx_text ltx_lst_identifier">neighbours</span>[<span id="lstnumberx31.3" class="ltx_text ltx_lst_identifier">int</span>(<span id="lstnumberx31.4" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">line</span>[0])<span id="lstnumberx31.5" class="ltx_text ltx_lst_space"> </span>-<span id="lstnumberx31.6" class="ltx_text ltx_lst_space"> </span>1].<span id="lstnumberx31.7" class="ltx_text ltx_lst_identifier">append</span>(<span id="lstnumberx31.8" class="ltx_text ltx_lst_identifier">int</span>(<span id="lstnumberx31.9" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">line</span>[1])<span id="lstnumberx31.10" class="ltx_text ltx_lst_space"> </span>-<span id="lstnumberx31.11" class="ltx_text ltx_lst_space"> </span>1)
</span>
<span id="lstnumberx32" class="ltx_listingline"><span id="lstnumberx32.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx32.2" class="ltx_text ltx_lst_identifier">neighbours</span>[<span id="lstnumberx32.3" class="ltx_text ltx_lst_identifier">int</span>(<span id="lstnumberx32.4" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">line</span>[1])<span id="lstnumberx32.5" class="ltx_text ltx_lst_space"> </span>-<span id="lstnumberx32.6" class="ltx_text ltx_lst_space"> </span>1].<span id="lstnumberx32.7" class="ltx_text ltx_lst_identifier">append</span>(<span id="lstnumberx32.8" class="ltx_text ltx_lst_identifier">int</span>(<span id="lstnumberx32.9" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">line</span>[0])<span id="lstnumberx32.10" class="ltx_text ltx_lst_space"> </span>-<span id="lstnumberx32.11" class="ltx_text ltx_lst_space"> </span>1)
</span>
<span id="lstnumberx33" class="ltx_listingline"><span id="lstnumberx33.1" class="ltx_text ltx_lst_identifier">leaves</span><span id="lstnumberx33.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx33.3" class="ltx_text ltx_lst_space"> </span>[]
</span>
<span id="lstnumberx34" class="ltx_listingline"><span id="lstnumberx34.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx34.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx34.3" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx34.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx34.5" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx34.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx34.7" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx34.8" class="ltx_text ltx_lst_space"> </span>(0,<span id="lstnumberx34.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx34.10" class="ltx_text ltx_lst_identifier">n</span>):
</span>
<span id="lstnumberx35" class="ltx_listingline"><span id="lstnumberx35.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx35.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx35.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx35.4" class="ltx_text ltx_lst_identifier">len</span>(<span id="lstnumberx35.5" class="ltx_text ltx_lst_identifier">neighbours</span>[<span id="lstnumberx35.6" class="ltx_text ltx_lst_identifier">i</span>])<span id="lstnumberx35.7" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx35.8" class="ltx_text ltx_lst_space"> </span>1:
</span>
<span id="lstnumberx36" class="ltx_listingline"><span id="lstnumberx36.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx36.2" class="ltx_text ltx_lst_identifier">leaves</span>.<span id="lstnumberx36.3" class="ltx_text ltx_lst_identifier">append</span>(<span id="lstnumberx36.4" class="ltx_text ltx_lst_identifier">i</span>)
</span>
<span id="lstnumberx37" class="ltx_listingline"><span id="lstnumberx37.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">while</span><span id="lstnumberx37.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.3" class="ltx_text ltx_lst_identifier">len</span>(<span id="lstnumberx37.4" class="ltx_text ltx_lst_identifier">leaves</span>)<span id="lstnumberx37.5" class="ltx_text ltx_lst_space"> </span>&gt;<span id="lstnumberx37.6" class="ltx_text ltx_lst_space"> </span>1:
</span>
<span id="lstnumberx38" class="ltx_listingline"><span id="lstnumberx38.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx38.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx38.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx38.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx38.5" class="ltx_text ltx_lst_identifier">leaves</span>.<span id="lstnumberx38.6" class="ltx_text ltx_lst_identifier">pop</span>()
</span>
<span id="lstnumberx39" class="ltx_listingline"><span id="lstnumberx39.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx39.2" class="ltx_text ltx_lst_identifier">y</span><span id="lstnumberx39.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx39.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx39.5" class="ltx_text ltx_lst_identifier">neighbours</span>[<span id="lstnumberx39.6" class="ltx_text ltx_lst_identifier">x</span>][0]
</span>
<span id="lstnumberx40" class="ltx_listingline"><span id="lstnumberx40.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx40.2" class="ltx_text ltx_lst_identifier">neighbours</span>[<span id="lstnumberx40.3" class="ltx_text ltx_lst_identifier">y</span>].<span id="lstnumberx40.4" class="ltx_text ltx_lst_identifier">remove</span>(<span id="lstnumberx40.5" class="ltx_text ltx_lst_identifier">x</span>)
</span>
<span id="lstnumberx41" class="ltx_listingline"><span id="lstnumberx41.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx41.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx41.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.4" class="ltx_text ltx_lst_identifier">len</span>(<span id="lstnumberx41.5" class="ltx_text ltx_lst_identifier">neighbours</span>[<span id="lstnumberx41.6" class="ltx_text ltx_lst_identifier">y</span>])<span id="lstnumberx41.7" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx41.8" class="ltx_text ltx_lst_space"> </span>1:
</span>
<span id="lstnumberx42" class="ltx_listingline"><span id="lstnumberx42.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx42.2" class="ltx_text ltx_lst_identifier">leaves</span>.<span id="lstnumberx42.3" class="ltx_text ltx_lst_identifier">append</span>(<span id="lstnumberx42.4" class="ltx_text ltx_lst_identifier">y</span>)
</span>
<span id="lstnumberx43" class="ltx_listingline"><span id="lstnumberx43.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx43.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx43.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.4" class="ltx_text ltx_lst_identifier">p</span><span id="lstnumberx43.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx43.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.8" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx43.9" class="ltx_text ltx_lst_identifier">x</span>]:
</span>
<span id="lstnumberx44" class="ltx_listingline"><span id="lstnumberx44.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx44.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx44.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.4" class="ltx_text ltx_lst_identifier">q</span><span id="lstnumberx44.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx44.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.8" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx44.9" class="ltx_text ltx_lst_identifier">y</span>]:
</span>
<span id="lstnumberx45" class="ltx_listingline"><span id="lstnumberx45.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx45.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx45.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.4" class="ltx_text ltx_lst_identifier">p</span>[0]<span id="lstnumberx45.5" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx45.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.7" class="ltx_text ltx_lst_identifier">q</span>[0]:
</span>
<span id="lstnumberx46" class="ltx_listingline"><span id="lstnumberx46.1" class="ltx_text ltx_lst_space">                </span><span id="lstnumberx46.2" class="ltx_text ltx_lst_identifier">answer</span><span id="lstnumberx46.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx46.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.5" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">max</span>([<span id="lstnumberx46.6" class="ltx_text ltx_lst_identifier">answer</span>,<span id="lstnumberx46.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.8" class="ltx_text ltx_lst_identifier">p</span>[1]<span id="lstnumberx46.9" class="ltx_text ltx_lst_space"> </span>+<span id="lstnumberx46.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.11" class="ltx_text ltx_lst_identifier">q</span>[1]])
</span>
<span id="lstnumberx47" class="ltx_listingline"><span id="lstnumberx47.1" class="ltx_text ltx_lst_space">                </span><span id="lstnumberx47.2" class="ltx_text ltx_lst_identifier">q</span>[1]<span id="lstnumberx47.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx47.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx47.5" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">max</span>([<span id="lstnumberx47.6" class="ltx_text ltx_lst_identifier">q</span>[1],<span id="lstnumberx47.7" class="ltx_text ltx_lst_identifier">p</span>[1]+1])
</span>
<span id="lstnumberx48" class="ltx_listingline"><span id="lstnumberx48.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx48.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.3" class="ltx_text ltx_lst_identifier">z</span><span id="lstnumberx48.4" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx48.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.6" class="ltx_text ltx_lst_identifier">False</span>:
</span>
<span id="lstnumberx49" class="ltx_listingline"><span id="lstnumberx49.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx49.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">print</span>(0)
</span>
<span id="lstnumberx50" class="ltx_listingline"><span id="lstnumberx50.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">else</span>:
</span>
<span id="lstnumberx51" class="ltx_listingline"><span id="lstnumberx51.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx51.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">print</span>(<span id="lstnumberx51.3" class="ltx_text ltx_lst_identifier">answer</span>)
</span>
</span>
<span id="A4.T6.30.32.1.1.1.3" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T6.30.33" class="ltx_tr">
<td id="A4.T6.30.33.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_b ltx_border_l ltx_border_r">
<span id="A4.T6.30.33.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T6.30.33.1.1.1" class="ltx_p" style="width:411.9pt;">{{Problem}}</span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 6: </span>Excerpt of the input prompt for generating initial solutions.</figcaption>
</figure>
</section>
<section id="A4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">D.2 </span>Vanilla Decomposition</h3>

<div id="A4.SS2.p1" class="ltx_para">
<p id="A4.SS2.p1.1" class="ltx_p">For vanilla decomposition, we adopt the prompt in Table <a href="#A4.T7" title="Table 7 ‣ D.2 Vanilla Decomposition ‣ Appendix D Example Prompts ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>.</p>
</div>
<figure id="A4.T7" class="ltx_table">
<table id="A4.T7.13" class="ltx_tabular ltx_align_middle">
<tr id="A4.T7.13.14" class="ltx_tr">
<td id="A4.T7.13.14.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r ltx_border_t">
<span id="A4.T7.13.14.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.13.14.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T7.13.14.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T7.13.13" class="ltx_tr">
<td id="A4.T7.13.13.13" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T7.13.13.13.13" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.3.3.3.3.3" class="ltx_p" style="width:433.6pt;">You are given a tree consisting of <math id="A4.T7.1.1.1.1.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A4.T7.1.1.1.1.1.m1.1a"><mi id="A4.T7.1.1.1.1.1.m1.1.1" xref="A4.T7.1.1.1.1.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A4.T7.1.1.1.1.1.m1.1b"><ci id="A4.T7.1.1.1.1.1.m1.1.1.cmml" xref="A4.T7.1.1.1.1.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.1.1.1.1.1.m1.1c">n</annotation></semantics></math> vertices. A number is written on each vertex; the number on vertex <math id="A4.T7.2.2.2.2.2.m2.1" class="ltx_Math" alttext="i" display="inline"><semantics id="A4.T7.2.2.2.2.2.m2.1a"><mi id="A4.T7.2.2.2.2.2.m2.1.1" xref="A4.T7.2.2.2.2.2.m2.1.1.cmml">i</mi><annotation-xml encoding="MathML-Content" id="A4.T7.2.2.2.2.2.m2.1b"><ci id="A4.T7.2.2.2.2.2.m2.1.1.cmml" xref="A4.T7.2.2.2.2.2.m2.1.1">𝑖</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.2.2.2.2.2.m2.1c">i</annotation></semantics></math> is equal to <math id="A4.T7.3.3.3.3.3.m3.1" class="ltx_Math" alttext="a_{i}" display="inline"><semantics id="A4.T7.3.3.3.3.3.m3.1a"><msub id="A4.T7.3.3.3.3.3.m3.1.1" xref="A4.T7.3.3.3.3.3.m3.1.1.cmml"><mi id="A4.T7.3.3.3.3.3.m3.1.1.2" xref="A4.T7.3.3.3.3.3.m3.1.1.2.cmml">a</mi><mi id="A4.T7.3.3.3.3.3.m3.1.1.3" xref="A4.T7.3.3.3.3.3.m3.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A4.T7.3.3.3.3.3.m3.1b"><apply id="A4.T7.3.3.3.3.3.m3.1.1.cmml" xref="A4.T7.3.3.3.3.3.m3.1.1"><csymbol cd="ambiguous" id="A4.T7.3.3.3.3.3.m3.1.1.1.cmml" xref="A4.T7.3.3.3.3.3.m3.1.1">subscript</csymbol><ci id="A4.T7.3.3.3.3.3.m3.1.1.2.cmml" xref="A4.T7.3.3.3.3.3.m3.1.1.2">𝑎</ci><ci id="A4.T7.3.3.3.3.3.m3.1.1.3.cmml" xref="A4.T7.3.3.3.3.3.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.3.3.3.3.3.m3.1c">a_{i}</annotation></semantics></math>.</span>
<span id="A4.T7.11.11.11.11.11" class="ltx_p">Let’s denote the function <math id="A4.T7.4.4.4.4.4.m1.2" class="ltx_Math" alttext="g(x,y)" display="inline"><semantics id="A4.T7.4.4.4.4.4.m1.2a"><mrow id="A4.T7.4.4.4.4.4.m1.2.3" xref="A4.T7.4.4.4.4.4.m1.2.3.cmml"><mi id="A4.T7.4.4.4.4.4.m1.2.3.2" xref="A4.T7.4.4.4.4.4.m1.2.3.2.cmml">g</mi><mo lspace="0em" rspace="0em" id="A4.T7.4.4.4.4.4.m1.2.3.1" xref="A4.T7.4.4.4.4.4.m1.2.3.1.cmml">​</mo><mrow id="A4.T7.4.4.4.4.4.m1.2.3.3.2" xref="A4.T7.4.4.4.4.4.m1.2.3.3.1.cmml"><mo stretchy="false" id="A4.T7.4.4.4.4.4.m1.2.3.3.2.1" xref="A4.T7.4.4.4.4.4.m1.2.3.3.1.cmml">(</mo><mi id="A4.T7.4.4.4.4.4.m1.1.1" xref="A4.T7.4.4.4.4.4.m1.1.1.cmml">x</mi><mo id="A4.T7.4.4.4.4.4.m1.2.3.3.2.2" xref="A4.T7.4.4.4.4.4.m1.2.3.3.1.cmml">,</mo><mi id="A4.T7.4.4.4.4.4.m1.2.2" xref="A4.T7.4.4.4.4.4.m1.2.2.cmml">y</mi><mo stretchy="false" id="A4.T7.4.4.4.4.4.m1.2.3.3.2.3" xref="A4.T7.4.4.4.4.4.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T7.4.4.4.4.4.m1.2b"><apply id="A4.T7.4.4.4.4.4.m1.2.3.cmml" xref="A4.T7.4.4.4.4.4.m1.2.3"><times id="A4.T7.4.4.4.4.4.m1.2.3.1.cmml" xref="A4.T7.4.4.4.4.4.m1.2.3.1"></times><ci id="A4.T7.4.4.4.4.4.m1.2.3.2.cmml" xref="A4.T7.4.4.4.4.4.m1.2.3.2">𝑔</ci><interval closure="open" id="A4.T7.4.4.4.4.4.m1.2.3.3.1.cmml" xref="A4.T7.4.4.4.4.4.m1.2.3.3.2"><ci id="A4.T7.4.4.4.4.4.m1.1.1.cmml" xref="A4.T7.4.4.4.4.4.m1.1.1">𝑥</ci><ci id="A4.T7.4.4.4.4.4.m1.2.2.cmml" xref="A4.T7.4.4.4.4.4.m1.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.4.4.4.4.4.m1.2c">g(x,y)</annotation></semantics></math> as the greatest common divisor of the numbers written on the vertices belonging to the simple path from vertex <math id="A4.T7.5.5.5.5.5.m2.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T7.5.5.5.5.5.m2.1a"><mi id="A4.T7.5.5.5.5.5.m2.1.1" xref="A4.T7.5.5.5.5.5.m2.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T7.5.5.5.5.5.m2.1b"><ci id="A4.T7.5.5.5.5.5.m2.1.1.cmml" xref="A4.T7.5.5.5.5.5.m2.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.5.5.5.5.5.m2.1c">x</annotation></semantics></math> to vertex <math id="A4.T7.6.6.6.6.6.m3.1" class="ltx_Math" alttext="y" display="inline"><semantics id="A4.T7.6.6.6.6.6.m3.1a"><mi id="A4.T7.6.6.6.6.6.m3.1.1" xref="A4.T7.6.6.6.6.6.m3.1.1.cmml">y</mi><annotation-xml encoding="MathML-Content" id="A4.T7.6.6.6.6.6.m3.1b"><ci id="A4.T7.6.6.6.6.6.m3.1.1.cmml" xref="A4.T7.6.6.6.6.6.m3.1.1">𝑦</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.6.6.6.6.6.m3.1c">y</annotation></semantics></math> (including these two vertices). Also let’s denote <math id="A4.T7.7.7.7.7.7.m4.2" class="ltx_Math" alttext="dist(x,y)" display="inline"><semantics id="A4.T7.7.7.7.7.7.m4.2a"><mrow id="A4.T7.7.7.7.7.7.m4.2.3" xref="A4.T7.7.7.7.7.7.m4.2.3.cmml"><mi id="A4.T7.7.7.7.7.7.m4.2.3.2" xref="A4.T7.7.7.7.7.7.m4.2.3.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T7.7.7.7.7.7.m4.2.3.1" xref="A4.T7.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mi id="A4.T7.7.7.7.7.7.m4.2.3.3" xref="A4.T7.7.7.7.7.7.m4.2.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T7.7.7.7.7.7.m4.2.3.1a" xref="A4.T7.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mi id="A4.T7.7.7.7.7.7.m4.2.3.4" xref="A4.T7.7.7.7.7.7.m4.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T7.7.7.7.7.7.m4.2.3.1b" xref="A4.T7.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mi id="A4.T7.7.7.7.7.7.m4.2.3.5" xref="A4.T7.7.7.7.7.7.m4.2.3.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T7.7.7.7.7.7.m4.2.3.1c" xref="A4.T7.7.7.7.7.7.m4.2.3.1.cmml">​</mo><mrow id="A4.T7.7.7.7.7.7.m4.2.3.6.2" xref="A4.T7.7.7.7.7.7.m4.2.3.6.1.cmml"><mo stretchy="false" id="A4.T7.7.7.7.7.7.m4.2.3.6.2.1" xref="A4.T7.7.7.7.7.7.m4.2.3.6.1.cmml">(</mo><mi id="A4.T7.7.7.7.7.7.m4.1.1" xref="A4.T7.7.7.7.7.7.m4.1.1.cmml">x</mi><mo id="A4.T7.7.7.7.7.7.m4.2.3.6.2.2" xref="A4.T7.7.7.7.7.7.m4.2.3.6.1.cmml">,</mo><mi id="A4.T7.7.7.7.7.7.m4.2.2" xref="A4.T7.7.7.7.7.7.m4.2.2.cmml">y</mi><mo stretchy="false" id="A4.T7.7.7.7.7.7.m4.2.3.6.2.3" xref="A4.T7.7.7.7.7.7.m4.2.3.6.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T7.7.7.7.7.7.m4.2b"><apply id="A4.T7.7.7.7.7.7.m4.2.3.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3"><times id="A4.T7.7.7.7.7.7.m4.2.3.1.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3.1"></times><ci id="A4.T7.7.7.7.7.7.m4.2.3.2.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3.2">𝑑</ci><ci id="A4.T7.7.7.7.7.7.m4.2.3.3.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3.3">𝑖</ci><ci id="A4.T7.7.7.7.7.7.m4.2.3.4.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3.4">𝑠</ci><ci id="A4.T7.7.7.7.7.7.m4.2.3.5.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3.5">𝑡</ci><interval closure="open" id="A4.T7.7.7.7.7.7.m4.2.3.6.1.cmml" xref="A4.T7.7.7.7.7.7.m4.2.3.6.2"><ci id="A4.T7.7.7.7.7.7.m4.1.1.cmml" xref="A4.T7.7.7.7.7.7.m4.1.1">𝑥</ci><ci id="A4.T7.7.7.7.7.7.m4.2.2.cmml" xref="A4.T7.7.7.7.7.7.m4.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.7.7.7.7.7.m4.2c">dist(x,y)</annotation></semantics></math> as the number of vertices on the simple path between vertices <math id="A4.T7.8.8.8.8.8.m5.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T7.8.8.8.8.8.m5.1a"><mi id="A4.T7.8.8.8.8.8.m5.1.1" xref="A4.T7.8.8.8.8.8.m5.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T7.8.8.8.8.8.m5.1b"><ci id="A4.T7.8.8.8.8.8.m5.1.1.cmml" xref="A4.T7.8.8.8.8.8.m5.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.8.8.8.8.8.m5.1c">x</annotation></semantics></math> and <math id="A4.T7.9.9.9.9.9.m6.1" class="ltx_Math" alttext="y" display="inline"><semantics id="A4.T7.9.9.9.9.9.m6.1a"><mi id="A4.T7.9.9.9.9.9.m6.1.1" xref="A4.T7.9.9.9.9.9.m6.1.1.cmml">y</mi><annotation-xml encoding="MathML-Content" id="A4.T7.9.9.9.9.9.m6.1b"><ci id="A4.T7.9.9.9.9.9.m6.1.1.cmml" xref="A4.T7.9.9.9.9.9.m6.1.1">𝑦</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.9.9.9.9.9.m6.1c">y</annotation></semantics></math>, including the endpoints. <math id="A4.T7.10.10.10.10.10.m7.2" class="ltx_Math" alttext="dist(x,x)=1" display="inline"><semantics id="A4.T7.10.10.10.10.10.m7.2a"><mrow id="A4.T7.10.10.10.10.10.m7.2.3" xref="A4.T7.10.10.10.10.10.m7.2.3.cmml"><mrow id="A4.T7.10.10.10.10.10.m7.2.3.2" xref="A4.T7.10.10.10.10.10.m7.2.3.2.cmml"><mi id="A4.T7.10.10.10.10.10.m7.2.3.2.2" xref="A4.T7.10.10.10.10.10.m7.2.3.2.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T7.10.10.10.10.10.m7.2.3.2.1" xref="A4.T7.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mi id="A4.T7.10.10.10.10.10.m7.2.3.2.3" xref="A4.T7.10.10.10.10.10.m7.2.3.2.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T7.10.10.10.10.10.m7.2.3.2.1a" xref="A4.T7.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mi id="A4.T7.10.10.10.10.10.m7.2.3.2.4" xref="A4.T7.10.10.10.10.10.m7.2.3.2.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T7.10.10.10.10.10.m7.2.3.2.1b" xref="A4.T7.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mi id="A4.T7.10.10.10.10.10.m7.2.3.2.5" xref="A4.T7.10.10.10.10.10.m7.2.3.2.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T7.10.10.10.10.10.m7.2.3.2.1c" xref="A4.T7.10.10.10.10.10.m7.2.3.2.1.cmml">​</mo><mrow id="A4.T7.10.10.10.10.10.m7.2.3.2.6.2" xref="A4.T7.10.10.10.10.10.m7.2.3.2.6.1.cmml"><mo stretchy="false" id="A4.T7.10.10.10.10.10.m7.2.3.2.6.2.1" xref="A4.T7.10.10.10.10.10.m7.2.3.2.6.1.cmml">(</mo><mi id="A4.T7.10.10.10.10.10.m7.1.1" xref="A4.T7.10.10.10.10.10.m7.1.1.cmml">x</mi><mo id="A4.T7.10.10.10.10.10.m7.2.3.2.6.2.2" xref="A4.T7.10.10.10.10.10.m7.2.3.2.6.1.cmml">,</mo><mi id="A4.T7.10.10.10.10.10.m7.2.2" xref="A4.T7.10.10.10.10.10.m7.2.2.cmml">x</mi><mo stretchy="false" id="A4.T7.10.10.10.10.10.m7.2.3.2.6.2.3" xref="A4.T7.10.10.10.10.10.m7.2.3.2.6.1.cmml">)</mo></mrow></mrow><mo id="A4.T7.10.10.10.10.10.m7.2.3.1" xref="A4.T7.10.10.10.10.10.m7.2.3.1.cmml">=</mo><mn id="A4.T7.10.10.10.10.10.m7.2.3.3" xref="A4.T7.10.10.10.10.10.m7.2.3.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.T7.10.10.10.10.10.m7.2b"><apply id="A4.T7.10.10.10.10.10.m7.2.3.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3"><eq id="A4.T7.10.10.10.10.10.m7.2.3.1.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.1"></eq><apply id="A4.T7.10.10.10.10.10.m7.2.3.2.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2"><times id="A4.T7.10.10.10.10.10.m7.2.3.2.1.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2.1"></times><ci id="A4.T7.10.10.10.10.10.m7.2.3.2.2.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2.2">𝑑</ci><ci id="A4.T7.10.10.10.10.10.m7.2.3.2.3.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2.3">𝑖</ci><ci id="A4.T7.10.10.10.10.10.m7.2.3.2.4.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2.4">𝑠</ci><ci id="A4.T7.10.10.10.10.10.m7.2.3.2.5.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2.5">𝑡</ci><interval closure="open" id="A4.T7.10.10.10.10.10.m7.2.3.2.6.1.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.2.6.2"><ci id="A4.T7.10.10.10.10.10.m7.1.1.cmml" xref="A4.T7.10.10.10.10.10.m7.1.1">𝑥</ci><ci id="A4.T7.10.10.10.10.10.m7.2.2.cmml" xref="A4.T7.10.10.10.10.10.m7.2.2">𝑥</ci></interval></apply><cn type="integer" id="A4.T7.10.10.10.10.10.m7.2.3.3.cmml" xref="A4.T7.10.10.10.10.10.m7.2.3.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.10.10.10.10.10.m7.2c">dist(x,x)=1</annotation></semantics></math> for every vertex <math id="A4.T7.11.11.11.11.11.m8.1" class="ltx_Math" alttext="x" display="inline"><semantics id="A4.T7.11.11.11.11.11.m8.1a"><mi id="A4.T7.11.11.11.11.11.m8.1.1" xref="A4.T7.11.11.11.11.11.m8.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="A4.T7.11.11.11.11.11.m8.1b"><ci id="A4.T7.11.11.11.11.11.m8.1.1.cmml" xref="A4.T7.11.11.11.11.11.m8.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.11.11.11.11.11.m8.1c">x</annotation></semantics></math>.</span>
<span id="A4.T7.13.13.13.13.13" class="ltx_p">Your task is to calculate the maximum value of <math id="A4.T7.12.12.12.12.12.m1.2" class="ltx_Math" alttext="dist(x,y)" display="inline"><semantics id="A4.T7.12.12.12.12.12.m1.2a"><mrow id="A4.T7.12.12.12.12.12.m1.2.3" xref="A4.T7.12.12.12.12.12.m1.2.3.cmml"><mi id="A4.T7.12.12.12.12.12.m1.2.3.2" xref="A4.T7.12.12.12.12.12.m1.2.3.2.cmml">d</mi><mo lspace="0em" rspace="0em" id="A4.T7.12.12.12.12.12.m1.2.3.1" xref="A4.T7.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mi id="A4.T7.12.12.12.12.12.m1.2.3.3" xref="A4.T7.12.12.12.12.12.m1.2.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="A4.T7.12.12.12.12.12.m1.2.3.1a" xref="A4.T7.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mi id="A4.T7.12.12.12.12.12.m1.2.3.4" xref="A4.T7.12.12.12.12.12.m1.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="A4.T7.12.12.12.12.12.m1.2.3.1b" xref="A4.T7.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mi id="A4.T7.12.12.12.12.12.m1.2.3.5" xref="A4.T7.12.12.12.12.12.m1.2.3.5.cmml">t</mi><mo lspace="0em" rspace="0em" id="A4.T7.12.12.12.12.12.m1.2.3.1c" xref="A4.T7.12.12.12.12.12.m1.2.3.1.cmml">​</mo><mrow id="A4.T7.12.12.12.12.12.m1.2.3.6.2" xref="A4.T7.12.12.12.12.12.m1.2.3.6.1.cmml"><mo stretchy="false" id="A4.T7.12.12.12.12.12.m1.2.3.6.2.1" xref="A4.T7.12.12.12.12.12.m1.2.3.6.1.cmml">(</mo><mi id="A4.T7.12.12.12.12.12.m1.1.1" xref="A4.T7.12.12.12.12.12.m1.1.1.cmml">x</mi><mo id="A4.T7.12.12.12.12.12.m1.2.3.6.2.2" xref="A4.T7.12.12.12.12.12.m1.2.3.6.1.cmml">,</mo><mi id="A4.T7.12.12.12.12.12.m1.2.2" xref="A4.T7.12.12.12.12.12.m1.2.2.cmml">y</mi><mo stretchy="false" id="A4.T7.12.12.12.12.12.m1.2.3.6.2.3" xref="A4.T7.12.12.12.12.12.m1.2.3.6.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T7.12.12.12.12.12.m1.2b"><apply id="A4.T7.12.12.12.12.12.m1.2.3.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3"><times id="A4.T7.12.12.12.12.12.m1.2.3.1.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3.1"></times><ci id="A4.T7.12.12.12.12.12.m1.2.3.2.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3.2">𝑑</ci><ci id="A4.T7.12.12.12.12.12.m1.2.3.3.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3.3">𝑖</ci><ci id="A4.T7.12.12.12.12.12.m1.2.3.4.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3.4">𝑠</ci><ci id="A4.T7.12.12.12.12.12.m1.2.3.5.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3.5">𝑡</ci><interval closure="open" id="A4.T7.12.12.12.12.12.m1.2.3.6.1.cmml" xref="A4.T7.12.12.12.12.12.m1.2.3.6.2"><ci id="A4.T7.12.12.12.12.12.m1.1.1.cmml" xref="A4.T7.12.12.12.12.12.m1.1.1">𝑥</ci><ci id="A4.T7.12.12.12.12.12.m1.2.2.cmml" xref="A4.T7.12.12.12.12.12.m1.2.2">𝑦</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.12.12.12.12.12.m1.2c">dist(x,y)</annotation></semantics></math> among such pairs of vertices that <math id="A4.T7.13.13.13.13.13.m2.2" class="ltx_Math" alttext="g(x,y)&gt;1" display="inline"><semantics id="A4.T7.13.13.13.13.13.m2.2a"><mrow id="A4.T7.13.13.13.13.13.m2.2.3" xref="A4.T7.13.13.13.13.13.m2.2.3.cmml"><mrow id="A4.T7.13.13.13.13.13.m2.2.3.2" xref="A4.T7.13.13.13.13.13.m2.2.3.2.cmml"><mi id="A4.T7.13.13.13.13.13.m2.2.3.2.2" xref="A4.T7.13.13.13.13.13.m2.2.3.2.2.cmml">g</mi><mo lspace="0em" rspace="0em" id="A4.T7.13.13.13.13.13.m2.2.3.2.1" xref="A4.T7.13.13.13.13.13.m2.2.3.2.1.cmml">​</mo><mrow id="A4.T7.13.13.13.13.13.m2.2.3.2.3.2" xref="A4.T7.13.13.13.13.13.m2.2.3.2.3.1.cmml"><mo stretchy="false" id="A4.T7.13.13.13.13.13.m2.2.3.2.3.2.1" xref="A4.T7.13.13.13.13.13.m2.2.3.2.3.1.cmml">(</mo><mi id="A4.T7.13.13.13.13.13.m2.1.1" xref="A4.T7.13.13.13.13.13.m2.1.1.cmml">x</mi><mo id="A4.T7.13.13.13.13.13.m2.2.3.2.3.2.2" xref="A4.T7.13.13.13.13.13.m2.2.3.2.3.1.cmml">,</mo><mi id="A4.T7.13.13.13.13.13.m2.2.2" xref="A4.T7.13.13.13.13.13.m2.2.2.cmml">y</mi><mo stretchy="false" id="A4.T7.13.13.13.13.13.m2.2.3.2.3.2.3" xref="A4.T7.13.13.13.13.13.m2.2.3.2.3.1.cmml">)</mo></mrow></mrow><mo id="A4.T7.13.13.13.13.13.m2.2.3.1" xref="A4.T7.13.13.13.13.13.m2.2.3.1.cmml">&gt;</mo><mn id="A4.T7.13.13.13.13.13.m2.2.3.3" xref="A4.T7.13.13.13.13.13.m2.2.3.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A4.T7.13.13.13.13.13.m2.2b"><apply id="A4.T7.13.13.13.13.13.m2.2.3.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3"><gt id="A4.T7.13.13.13.13.13.m2.2.3.1.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3.1"></gt><apply id="A4.T7.13.13.13.13.13.m2.2.3.2.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3.2"><times id="A4.T7.13.13.13.13.13.m2.2.3.2.1.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3.2.1"></times><ci id="A4.T7.13.13.13.13.13.m2.2.3.2.2.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3.2.2">𝑔</ci><interval closure="open" id="A4.T7.13.13.13.13.13.m2.2.3.2.3.1.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3.2.3.2"><ci id="A4.T7.13.13.13.13.13.m2.1.1.cmml" xref="A4.T7.13.13.13.13.13.m2.1.1">𝑥</ci><ci id="A4.T7.13.13.13.13.13.m2.2.2.cmml" xref="A4.T7.13.13.13.13.13.m2.2.2">𝑦</ci></interval></apply><cn type="integer" id="A4.T7.13.13.13.13.13.m2.2.3.3.cmml" xref="A4.T7.13.13.13.13.13.m2.2.3.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T7.13.13.13.13.13.m2.2c">g(x,y)&gt;1</annotation></semantics></math>…[TRUNCATED]</span>
</span>
</td>
</tr>
<tr id="A4.T7.13.15" class="ltx_tr">
<td id="A4.T7.13.15.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T7.13.15.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.13.15.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T7.13.15.1.1.1.1" class="ltx_text ltx_font_bold">Initial Solution</span>

<span id="A4.T7.13.15.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,Iy4uLltUUlVOQ0FURURdCmZvciBpIGluIHJhbmdlICgyLCA1ICogMTAgKiogMik6CiAgICB2ID0gVHJ1ZQogICAgZm9yIHAgaW4gcHJpbWVzOgogICAgICAgIGlmIGkgXCUgcCA9PSAwOgogICAgICAgICAgICB2ID0gRmFsc2UKICAgIGlmIHYgPT0gVHJ1ZToKICAgICAgICBwcmltZXMuYXBwZW5kKGkpCm4gPSBpbnQoc3lzLnN0ZGluLnJlYWRsaW5lKCkuc3RyaXAoKSkKYSA9IGxpc3QobWFwKGludCwgc3lzLnN0ZGluLnJlYWRsaW5lKCkuc3RyaXAoKS5zcGxpdCgpKSkKaWYgc3VtKGEpID09IG46CiAgICB6ID0gRmFsc2UKZm9yIGkgaW4gcmFuZ2UgKDAsIG4pOgogICAgeCA9IGFbaV0KICAgIGFbaV0gPSBbXQogICAgZm9yIHAgaW4gcHJpbWVzOgogICAgICAgIGlmIHggXCUgcCA9PSAwOgogICAgICAgICAgICBhW2ldLmFwcGVuZChbcCwgMV0pCiAgICAgICAgICAgIHggPSB4IC8vIHAKICAgICAgICAgICAgd2hpbGUgeCBcJSBwID09IDA6CiAgICAgICAgICAgICAgICB4ID0geCAvLyBwCiAgICBpZiB4ICE9IDE6CiAgICAgICAgYVtpXS5hcHBlbmQoW3gsIDFdKQojLi4uW1RSVU5DQVRFRF0=" download="">⬇</a></span>
<span id="lstnumberx52" class="ltx_listingline"><span id="lstnumberx52.1" class="ltx_text ltx_lst_comment" style="color:#009900;">#…[TRUNCATED]</span>
</span>
<span id="lstnumberx53" class="ltx_listingline"><span id="lstnumberx53.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx53.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx53.3" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx53.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx53.5" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx53.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx53.7" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx53.8" class="ltx_text ltx_lst_space"> </span>(2,<span id="lstnumberx53.9" class="ltx_text ltx_lst_space"> </span>5<span id="lstnumberx53.10" class="ltx_text ltx_lst_space"> </span>*<span id="lstnumberx53.11" class="ltx_text ltx_lst_space"> </span>10<span id="lstnumberx53.12" class="ltx_text ltx_lst_space"> </span>**<span id="lstnumberx53.13" class="ltx_text ltx_lst_space"> </span>2):
</span>
<span id="lstnumberx54" class="ltx_listingline"><span id="lstnumberx54.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx54.2" class="ltx_text ltx_lst_identifier">v</span><span id="lstnumberx54.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx54.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.5" class="ltx_text ltx_lst_identifier">True</span>
</span>
<span id="lstnumberx55" class="ltx_listingline"><span id="lstnumberx55.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx55.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx55.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx55.4" class="ltx_text ltx_lst_identifier">p</span><span id="lstnumberx55.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx55.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx55.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx55.8" class="ltx_text ltx_lst_identifier">primes</span>:
</span>
<span id="lstnumberx56" class="ltx_listingline"><span id="lstnumberx56.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx56.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx56.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx56.4" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx56.5" class="ltx_text ltx_lst_space"> </span>\<span id="lstnumberx56.6" class="ltx_text ltx_lst_comment" style="color:#009900;">%<span id="lstnumberx56.6.1" class="ltx_text ltx_lst_space"> </span>p<span id="lstnumberx56.6.2" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx56.6.3" class="ltx_text ltx_lst_space"> </span>0:</span>
</span>
<span id="lstnumberx57" class="ltx_listingline"><span id="lstnumberx57.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx57.2" class="ltx_text ltx_lst_identifier">v</span><span id="lstnumberx57.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx57.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.5" class="ltx_text ltx_lst_identifier">False</span>
</span>
<span id="lstnumberx58" class="ltx_listingline"><span id="lstnumberx58.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx58.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx58.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx58.4" class="ltx_text ltx_lst_identifier">v</span><span id="lstnumberx58.5" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx58.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx58.7" class="ltx_text ltx_lst_identifier">True</span>:
</span>
<span id="lstnumberx59" class="ltx_listingline"><span id="lstnumberx59.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx59.2" class="ltx_text ltx_lst_identifier">primes</span>.<span id="lstnumberx59.3" class="ltx_text ltx_lst_identifier">append</span>(<span id="lstnumberx59.4" class="ltx_text ltx_lst_identifier">i</span>)
</span>
<span id="lstnumberx60" class="ltx_listingline"><span id="lstnumberx60.1" class="ltx_text ltx_lst_identifier">n</span><span id="lstnumberx60.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx60.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.4" class="ltx_text ltx_lst_identifier">int</span>(<span id="lstnumberx60.5" class="ltx_text ltx_lst_identifier">sys</span>.<span id="lstnumberx60.6" class="ltx_text ltx_lst_identifier">stdin</span>.<span id="lstnumberx60.7" class="ltx_text ltx_lst_identifier">readline</span>().<span id="lstnumberx60.8" class="ltx_text ltx_lst_identifier">strip</span>())
</span>
<span id="lstnumberx61" class="ltx_listingline"><span id="lstnumberx61.1" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx61.2" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx61.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.4" class="ltx_text ltx_lst_identifier">list</span>(<span id="lstnumberx61.5" class="ltx_text ltx_lst_identifier">map</span>(<span id="lstnumberx61.6" class="ltx_text ltx_lst_identifier">int</span>,<span id="lstnumberx61.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.8" class="ltx_text ltx_lst_identifier">sys</span>.<span id="lstnumberx61.9" class="ltx_text ltx_lst_identifier">stdin</span>.<span id="lstnumberx61.10" class="ltx_text ltx_lst_identifier">readline</span>().<span id="lstnumberx61.11" class="ltx_text ltx_lst_identifier">strip</span>().<span id="lstnumberx61.12" class="ltx_text ltx_lst_identifier">split</span>()))
</span>
<span id="lstnumberx62" class="ltx_listingline"><span id="lstnumberx62.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx62.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.3" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">sum</span>(<span id="lstnumberx62.4" class="ltx_text ltx_lst_identifier">a</span>)<span id="lstnumberx62.5" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx62.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.7" class="ltx_text ltx_lst_identifier">n</span>:
</span>
<span id="lstnumberx63" class="ltx_listingline"><span id="lstnumberx63.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx63.2" class="ltx_text ltx_lst_identifier">z</span><span id="lstnumberx63.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx63.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.5" class="ltx_text ltx_lst_identifier">False</span>
</span>
<span id="lstnumberx64" class="ltx_listingline"><span id="lstnumberx64.1" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx64.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.3" class="ltx_text ltx_lst_identifier">i</span><span id="lstnumberx64.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.5" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx64.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.7" class="ltx_text ltx_lst_identifier">range</span><span id="lstnumberx64.8" class="ltx_text ltx_lst_space"> </span>(0,<span id="lstnumberx64.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.10" class="ltx_text ltx_lst_identifier">n</span>):
</span>
<span id="lstnumberx65" class="ltx_listingline"><span id="lstnumberx65.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx65.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx65.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx65.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.5" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx65.6" class="ltx_text ltx_lst_identifier">i</span>]
</span>
<span id="lstnumberx66" class="ltx_listingline"><span id="lstnumberx66.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx66.2" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx66.3" class="ltx_text ltx_lst_identifier">i</span>]<span id="lstnumberx66.4" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx66.5" class="ltx_text ltx_lst_space"> </span>[]
</span>
<span id="lstnumberx67" class="ltx_listingline"><span id="lstnumberx67.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx67.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">for</span><span id="lstnumberx67.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.4" class="ltx_text ltx_lst_identifier">p</span><span id="lstnumberx67.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx67.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.8" class="ltx_text ltx_lst_identifier">primes</span>:
</span>
<span id="lstnumberx68" class="ltx_listingline"><span id="lstnumberx68.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx68.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx68.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx68.4" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx68.5" class="ltx_text ltx_lst_space"> </span>\<span id="lstnumberx68.6" class="ltx_text ltx_lst_comment" style="color:#009900;">%<span id="lstnumberx68.6.1" class="ltx_text ltx_lst_space"> </span>p<span id="lstnumberx68.6.2" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx68.6.3" class="ltx_text ltx_lst_space"> </span>0:</span>
</span>
<span id="lstnumberx69" class="ltx_listingline"><span id="lstnumberx69.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx69.2" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx69.3" class="ltx_text ltx_lst_identifier">i</span>].<span id="lstnumberx69.4" class="ltx_text ltx_lst_identifier">append</span>([<span id="lstnumberx69.5" class="ltx_text ltx_lst_identifier">p</span>,<span id="lstnumberx69.6" class="ltx_text ltx_lst_space"> </span>1])
</span>
<span id="lstnumberx70" class="ltx_listingline"><span id="lstnumberx70.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx70.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx70.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx70.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx70.5" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx70.6" class="ltx_text ltx_lst_space"> </span>//<span id="lstnumberx70.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx70.8" class="ltx_text ltx_lst_identifier">p</span>
</span>
<span id="lstnumberx71" class="ltx_listingline"><span id="lstnumberx71.1" class="ltx_text ltx_lst_space">            </span><span id="lstnumberx71.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">while</span><span id="lstnumberx71.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx71.4" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx71.5" class="ltx_text ltx_lst_space"> </span>\<span id="lstnumberx71.6" class="ltx_text ltx_lst_comment" style="color:#009900;">%<span id="lstnumberx71.6.1" class="ltx_text ltx_lst_space"> </span>p<span id="lstnumberx71.6.2" class="ltx_text ltx_lst_space"> </span>==<span id="lstnumberx71.6.3" class="ltx_text ltx_lst_space"> </span>0:</span>
</span>
<span id="lstnumberx72" class="ltx_listingline"><span id="lstnumberx72.1" class="ltx_text ltx_lst_space">                </span><span id="lstnumberx72.2" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx72.3" class="ltx_text ltx_lst_space"> </span>=<span id="lstnumberx72.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx72.5" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx72.6" class="ltx_text ltx_lst_space"> </span>//<span id="lstnumberx72.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx72.8" class="ltx_text ltx_lst_identifier">p</span>
</span>
<span id="lstnumberx73" class="ltx_listingline"><span id="lstnumberx73.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx73.2" class="ltx_text ltx_lst_keyword" style="color:#FF00FF;">if</span><span id="lstnumberx73.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx73.4" class="ltx_text ltx_lst_identifier">x</span><span id="lstnumberx73.5" class="ltx_text ltx_lst_space"> </span>!=<span id="lstnumberx73.6" class="ltx_text ltx_lst_space"> </span>1:
</span>
<span id="lstnumberx74" class="ltx_listingline"><span id="lstnumberx74.1" class="ltx_text ltx_lst_space">        </span><span id="lstnumberx74.2" class="ltx_text ltx_lst_identifier">a</span>[<span id="lstnumberx74.3" class="ltx_text ltx_lst_identifier">i</span>].<span id="lstnumberx74.4" class="ltx_text ltx_lst_identifier">append</span>([<span id="lstnumberx74.5" class="ltx_text ltx_lst_identifier">x</span>,<span id="lstnumberx74.6" class="ltx_text ltx_lst_space"> </span>1])
</span>
<span id="lstnumberx75" class="ltx_listingline"><span id="lstnumberx75.1" class="ltx_text ltx_lst_comment" style="color:#009900;">#…[TRUNCATED]</span>
</span>
</span>
Your task is to decompose the initial solution into separate functions that handle specific tasks. This will make the code more modular and easier to understand and debug.</span>
</span>
</td>
</tr>
<tr id="A4.T7.13.16" class="ltx_tr">
<td id="A4.T7.13.16.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T7.13.16.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.13.16.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T7.13.16.1.1.1.1" class="ltx_text ltx_font_bold">Decomposed Solution</span>

<span id="A4.T7.13.16.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,ZGVmIHNpZXZlX29mX2VyYXRvc3RoZW5lcyhsaW1pdCk6CiAgICAiIiIKICAgIEdlbmVyYXRlIGEgbGlzdCBvZiBwcmltZSBudW1iZXJzIHVwIHRvIHRoZSBzcGVjaWZpZWQgbGltaXQuCiAgICBQYXJhbWV0ZXJzOgogICAgLSBsaW1pdDogVGhlIHVwcGVyIGJvdW5kIHVwIHRvIHdoaWNoIHByaW1lIG51bWJlcnMgYXJlIHRvIGJlIGZvdW5kLgogICAgUmV0dXJuczoKICAgIC0gcHJpbWVzOiBBIGxpc3Qgb2YgcHJpbWUgbnVtYmVycyB1cCB0byB0aGUgc3BlY2lmaWVkIGxpbWl0LgogICAgIiIiCiAgICBwcmltZXMgPSBbXQogICAgaXNfcHJpbWUgPSBbVHJ1ZV0gKiAobGltaXQgKyAxKQogICAgZm9yIHAgaW4gcmFuZ2UoMiwgbGltaXQgKyAxKToKICAgICAgICBpZiBpc19wcmltZVtwXToKICAgICAgICAgICAgcHJpbWVzLmFwcGVuZChwKQogICAgICAgICAgICBmb3IgaSBpbiByYW5nZShwICogcCwgbGltaXQgKyAxLCBwKToKICAgICAgICAgICAgICAgIGlzX3ByaW1lW2ldID0gRmFsc2UKICAgIHJldHVybiBwcmltZXMKCmRlZiBmYWN0b3JpemVfbnVtYmVycyhhLCBwcmltZXMpOgogICAgIiIiCiAgICBGYWN0b3JpemUgZWFjaCBudW1iZXIgaW4gdGhlIGxpc3QgJ2EnIHVzaW5nIHRoZSBsaXN0IG9mIGtub3duIHByaW1lcy4KICAgIFBhcmFtZXRlcnM6CiAgICAtIGE6IFRoZSBsaXN0IG9mIG51bWJlcnMgdG8gYmUgZmFjdG9yaXplZC4KICAgIC0gcHJpbWVzOiBUaGUgbGlzdCBvZiBrbm93biBwcmltZSBudW1iZXJzLgogICAgUmV0dXJuczoKICAgIC0gZmFjdG9yaXplZDogQSBsaXN0IG9mIGZhY3Rvcml6ZWQgbnVtYmVycywgd2hlcmUgZWFjaCBudW1iZXIgaXMgcmVwcmVzZW50ZWQgYXMgYSBsaXN0IG9mIHByaW1lIGZhY3RvcnMgYW5kIHRoZWlyIGV4cG9uZW50cy4KICAgICIiIgogICAgZmFjdG9yaXplZCA9IFtdCiAgICBmb3IgeCBpbiBhOgogICAgICAgIGZhY3RvcnMgPSBbXQogICAgICAgIGZvciBwIGluIHByaW1lczoKICAgICAgICAgICAgaWYgeCBcJSBwID09IDA6CiAgICAgICAgICAgICAgICB3aGlsZSB4IFwlIHAgPT0gMDoKICAgICAgICAgICAgICAgICAgICB4IC8vPSBwCiAgICAgICAgICAgICAgICBmYWN0b3JzLmFwcGVuZCgocCwgMSkpCiAgICAgICAgaWYgeCAhPSAxOgogICAgICAgICAgICBmYWN0b3JzLmFwcGVuZCgoeCwgMSkpCiAgICAgICAgZmFjdG9yaXplZC5hcHBlbmQoZmFjdG9ycykKICAgIHJldHVybiBmYWN0b3JpemVkCgpkZWYgZmluZF9sZWF2ZXMobmVpZ2hib3Vycyk6CiAgICAiIiIKICAgIEZpbmQgdGhlIGxlYWYgbm9kZXMgKHZlcnRpY2VzIHdpdGggb25seSBvbmUgYWRqYWNlbnQgdmVydGV4KSBvZiBhIHRyZWUuCiAgICBQYXJhbWV0ZXJzOgogICAgLSBuZWlnaGJvdXJzOiBUaGUgYWRqYWNlbmN5IGxpc3QgcmVwcmVzZW50aW5nIHRoZSB0cmVlLgogICAgUmV0dXJuczoKICAgIC0gbGVhdmVzOiBBIGxpc3Qgb2YgbGVhZiBub2RlcyBpbiB0aGUgdHJlZS4KICAgICIiIgogICAgbGVhdmVzID0gW10KICAgIGZvciBpLCBuZWlnaGJvdXJzX2xpc3QgaW4gZW51bWVyYXRlKG5laWdoYm91cnMpOgogICAgICAgIGlmIGxlbihuZWlnaGJvdXJzX2xpc3QpID09IDE6CiAgICAgICAgICAgIGxlYXZlcy5hcHBlbmQoaSkKICAgIHJldHVybiBsZWF2ZXMKIy4uLltUUlVOQ0FURURd" download="">⬇</a></span>
<span id="lstnumberx76" class="ltx_listingline"><span id="lstnumberx76.1" class="ltx_text ltx_lst_identifier">def</span><span id="lstnumberx76.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx76.3" class="ltx_text ltx_lst_identifier">sieve_of_eratosthenes</span>(<span id="lstnumberx76.4" class="ltx_text ltx_lst_identifier">limit</span>):
</span>
<span id="lstnumberx77" class="ltx_listingline"><span id="lstnumberx77.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx77.2" class="ltx_text ltx_lst_string" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx78" class="ltx_listingline"><span id="lstnumberx78.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx78.2" class="ltx_text" style="color:#9400D1;">Generate</span><span id="lstnumberx78.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.4" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx78.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.6" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx78.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.8" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx78.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.10" class="ltx_text" style="color:#9400D1;">prime</span><span id="lstnumberx78.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.12" class="ltx_text" style="color:#9400D1;">numbers</span><span id="lstnumberx78.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.14" class="ltx_text" style="color:#9400D1;">up</span><span id="lstnumberx78.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx78.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.18" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx78.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.20" class="ltx_text" style="color:#9400D1;">specified</span><span id="lstnumberx78.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx78.22" class="ltx_text" style="color:#9400D1;">limit.</span>
</span>
<span id="lstnumberx79" class="ltx_listingline"><span id="lstnumberx79.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx79.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx80" class="ltx_listingline"><span id="lstnumberx80.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx80.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx80.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.4" class="ltx_text" style="color:#9400D1;">limit:</span><span id="lstnumberx80.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx80.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.8" class="ltx_text" style="color:#9400D1;">upper</span><span id="lstnumberx80.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.10" class="ltx_text" style="color:#9400D1;">bound</span><span id="lstnumberx80.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.12" class="ltx_text" style="color:#9400D1;">up</span><span id="lstnumberx80.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.14" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx80.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.16" class="ltx_text" style="color:#9400D1;">which</span><span id="lstnumberx80.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.18" class="ltx_text" style="color:#9400D1;">prime</span><span id="lstnumberx80.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.20" class="ltx_text" style="color:#9400D1;">numbers</span><span id="lstnumberx80.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.22" class="ltx_text" style="color:#9400D1;">are</span><span id="lstnumberx80.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.24" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx80.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.26" class="ltx_text" style="color:#9400D1;">be</span><span id="lstnumberx80.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx80.28" class="ltx_text" style="color:#9400D1;">found.</span>
</span>
<span id="lstnumberx81" class="ltx_listingline"><span id="lstnumberx81.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx81.2" class="ltx_text" style="color:#9400D1;">Returns:</span>
</span>
<span id="lstnumberx82" class="ltx_listingline"><span id="lstnumberx82.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx82.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx82.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.4" class="ltx_text" style="color:#9400D1;">primes:</span><span id="lstnumberx82.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.6" class="ltx_text" style="color:#9400D1;">A</span><span id="lstnumberx82.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx82.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx82.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.12" class="ltx_text" style="color:#9400D1;">prime</span><span id="lstnumberx82.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.14" class="ltx_text" style="color:#9400D1;">numbers</span><span id="lstnumberx82.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.16" class="ltx_text" style="color:#9400D1;">up</span><span id="lstnumberx82.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.18" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx82.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx82.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.22" class="ltx_text" style="color:#9400D1;">specified</span><span id="lstnumberx82.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx82.24" class="ltx_text" style="color:#9400D1;">limit.</span>
</span>
<span id="lstnumberx83" class="ltx_listingline"><span id="lstnumberx83.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx83.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx84" class="ltx_listingline"><span id="lstnumberx84.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx84.2" class="ltx_text" style="color:#9400D1;">primes</span><span id="lstnumberx84.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx84.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx84.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx84.6" class="ltx_text" style="color:#9400D1;">[]</span>
</span>
<span id="lstnumberx85" class="ltx_listingline"><span id="lstnumberx85.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx85.2" class="ltx_text" style="color:#9400D1;">is_prime</span><span id="lstnumberx85.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx85.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx85.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx85.6" class="ltx_text" style="color:#9400D1;">[True]</span><span id="lstnumberx85.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx85.8" class="ltx_text" style="color:#9400D1;">*</span><span id="lstnumberx85.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx85.10" class="ltx_text" style="color:#9400D1;">(limit</span><span id="lstnumberx85.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx85.12" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx85.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx85.14" class="ltx_text" style="color:#9400D1;">1)</span>
</span>
<span id="lstnumberx86" class="ltx_listingline"><span id="lstnumberx86.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx86.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx86.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx86.4" class="ltx_text" style="color:#9400D1;">p</span><span id="lstnumberx86.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx86.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx86.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx86.8" class="ltx_text" style="color:#9400D1;">range(2,</span><span id="lstnumberx86.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx86.10" class="ltx_text" style="color:#9400D1;">limit</span><span id="lstnumberx86.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx86.12" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx86.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx86.14" class="ltx_text" style="color:#9400D1;">1):</span>
</span>
<span id="lstnumberx87" class="ltx_listingline"><span id="lstnumberx87.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx87.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx87.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx87.4" class="ltx_text" style="color:#9400D1;">is_prime[p]:</span>
</span>
<span id="lstnumberx88" class="ltx_listingline"><span id="lstnumberx88.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">            </span><span id="lstnumberx88.2" class="ltx_text" style="color:#9400D1;">primes.append(p)</span>
</span>
<span id="lstnumberx89" class="ltx_listingline"><span id="lstnumberx89.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">            </span><span id="lstnumberx89.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx89.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.4" class="ltx_text" style="color:#9400D1;">i</span><span id="lstnumberx89.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx89.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.8" class="ltx_text" style="color:#9400D1;">range(p</span><span id="lstnumberx89.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.10" class="ltx_text" style="color:#9400D1;">*</span><span id="lstnumberx89.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.12" class="ltx_text" style="color:#9400D1;">p,</span><span id="lstnumberx89.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.14" class="ltx_text" style="color:#9400D1;">limit</span><span id="lstnumberx89.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.16" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx89.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.18" class="ltx_text" style="color:#9400D1;">1,</span><span id="lstnumberx89.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx89.20" class="ltx_text" style="color:#9400D1;">p):</span>
</span>
<span id="lstnumberx90" class="ltx_listingline"><span id="lstnumberx90.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">                </span><span id="lstnumberx90.2" class="ltx_text" style="color:#9400D1;">is_prime[i]</span><span id="lstnumberx90.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx90.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx90.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx90.6" class="ltx_text" style="color:#9400D1;">False</span>
</span>
<span id="lstnumberx91" class="ltx_listingline"><span id="lstnumberx91.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx91.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx91.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx91.4" class="ltx_text" style="color:#9400D1;">primes</span>
</span>
<span id="lstnumberx92" class="ltx_listingline">
</span>
<span id="lstnumberx93" class="ltx_listingline"><span id="lstnumberx93.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx93.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx93.3" class="ltx_text" style="color:#9400D1;">factorize_numbers(a,</span><span id="lstnumberx93.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx93.5" class="ltx_text" style="color:#9400D1;">primes):</span>
</span>
<span id="lstnumberx94" class="ltx_listingline"><span id="lstnumberx94.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx94.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx95" class="ltx_listingline"><span id="lstnumberx95.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx95.2" class="ltx_text" style="color:#9400D1;">Factorize</span><span id="lstnumberx95.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.4" class="ltx_text" style="color:#9400D1;">each</span><span id="lstnumberx95.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.6" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx95.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.8" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx95.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.10" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx95.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.12" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx95.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.14" class="ltx_text" style="color:#9400D1;">’a’</span><span id="lstnumberx95.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.16" class="ltx_text" style="color:#9400D1;">using</span><span id="lstnumberx95.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.18" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx95.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.20" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx95.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.22" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx95.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.24" class="ltx_text" style="color:#9400D1;">known</span><span id="lstnumberx95.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx95.26" class="ltx_text" style="color:#9400D1;">primes.</span>
</span>
<span id="lstnumberx96" class="ltx_listingline"><span id="lstnumberx96.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx96.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx97" class="ltx_listingline"><span id="lstnumberx97.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx97.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx97.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.4" class="ltx_text" style="color:#9400D1;">a:</span><span id="lstnumberx97.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx97.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx97.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx97.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.12" class="ltx_text" style="color:#9400D1;">numbers</span><span id="lstnumberx97.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.14" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx97.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.16" class="ltx_text" style="color:#9400D1;">be</span><span id="lstnumberx97.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx97.18" class="ltx_text" style="color:#9400D1;">factorized.</span>
</span>
<span id="lstnumberx98" class="ltx_listingline"><span id="lstnumberx98.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx98.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx98.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.4" class="ltx_text" style="color:#9400D1;">primes:</span><span id="lstnumberx98.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx98.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx98.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx98.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.12" class="ltx_text" style="color:#9400D1;">known</span><span id="lstnumberx98.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.14" class="ltx_text" style="color:#9400D1;">prime</span><span id="lstnumberx98.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx98.16" class="ltx_text" style="color:#9400D1;">numbers.</span>
</span>
<span id="lstnumberx99" class="ltx_listingline"><span id="lstnumberx99.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx99.2" class="ltx_text" style="color:#9400D1;">Returns:</span>
</span>
<span id="lstnumberx100" class="ltx_listingline"><span id="lstnumberx100.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx100.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx100.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.4" class="ltx_text" style="color:#9400D1;">factorized:</span><span id="lstnumberx100.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.6" class="ltx_text" style="color:#9400D1;">A</span><span id="lstnumberx100.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx100.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx100.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.12" class="ltx_text" style="color:#9400D1;">factorized</span><span id="lstnumberx100.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.14" class="ltx_text" style="color:#9400D1;">numbers,</span><span id="lstnumberx100.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.16" class="ltx_text" style="color:#9400D1;">where</span><span id="lstnumberx100.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.18" class="ltx_text" style="color:#9400D1;">each</span><span id="lstnumberx100.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.20" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx100.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.22" class="ltx_text" style="color:#9400D1;">is</span><span id="lstnumberx100.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.24" class="ltx_text" style="color:#9400D1;">represented</span><span id="lstnumberx100.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.26" class="ltx_text" style="color:#9400D1;">as</span><span id="lstnumberx100.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.28" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx100.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.30" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx100.31" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.32" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx100.33" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.34" class="ltx_text" style="color:#9400D1;">prime</span><span id="lstnumberx100.35" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.36" class="ltx_text" style="color:#9400D1;">factors</span><span id="lstnumberx100.37" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.38" class="ltx_text" style="color:#9400D1;">and</span><span id="lstnumberx100.39" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.40" class="ltx_text" style="color:#9400D1;">their</span><span id="lstnumberx100.41" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx100.42" class="ltx_text" style="color:#9400D1;">exponents.</span>
</span>
<span id="lstnumberx101" class="ltx_listingline"><span id="lstnumberx101.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx101.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx102" class="ltx_listingline"><span id="lstnumberx102.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx102.2" class="ltx_text" style="color:#9400D1;">factorized</span><span id="lstnumberx102.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx102.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx102.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx102.6" class="ltx_text" style="color:#9400D1;">[]</span>
</span>
<span id="lstnumberx103" class="ltx_listingline"><span id="lstnumberx103.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx103.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx103.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx103.4" class="ltx_text" style="color:#9400D1;">x</span><span id="lstnumberx103.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx103.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx103.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx103.8" class="ltx_text" style="color:#9400D1;">a:</span>
</span>
<span id="lstnumberx104" class="ltx_listingline"><span id="lstnumberx104.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx104.2" class="ltx_text" style="color:#9400D1;">factors</span><span id="lstnumberx104.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx104.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx104.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx104.6" class="ltx_text" style="color:#9400D1;">[]</span>
</span>
<span id="lstnumberx105" class="ltx_listingline"><span id="lstnumberx105.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx105.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx105.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx105.4" class="ltx_text" style="color:#9400D1;">p</span><span id="lstnumberx105.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx105.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx105.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx105.8" class="ltx_text" style="color:#9400D1;">primes:</span>
</span>
<span id="lstnumberx106" class="ltx_listingline"><span id="lstnumberx106.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">            </span><span id="lstnumberx106.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx106.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx106.4" class="ltx_text" style="color:#9400D1;">x</span><span id="lstnumberx106.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx106.6" class="ltx_text" style="color:#9400D1;">\%</span><span id="lstnumberx106.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx106.8" class="ltx_text" style="color:#9400D1;">p</span><span id="lstnumberx106.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx106.10" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx106.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx106.12" class="ltx_text" style="color:#9400D1;">0:</span>
</span>
<span id="lstnumberx107" class="ltx_listingline"><span id="lstnumberx107.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">                </span><span id="lstnumberx107.2" class="ltx_text" style="color:#9400D1;">while</span><span id="lstnumberx107.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx107.4" class="ltx_text" style="color:#9400D1;">x</span><span id="lstnumberx107.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx107.6" class="ltx_text" style="color:#9400D1;">\%</span><span id="lstnumberx107.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx107.8" class="ltx_text" style="color:#9400D1;">p</span><span id="lstnumberx107.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx107.10" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx107.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx107.12" class="ltx_text" style="color:#9400D1;">0:</span>
</span>
<span id="lstnumberx108" class="ltx_listingline"><span id="lstnumberx108.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">                    </span><span id="lstnumberx108.2" class="ltx_text" style="color:#9400D1;">x</span><span id="lstnumberx108.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx108.4" class="ltx_text" style="color:#9400D1;">//=</span><span id="lstnumberx108.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx108.6" class="ltx_text" style="color:#9400D1;">p</span>
</span>
<span id="lstnumberx109" class="ltx_listingline"><span id="lstnumberx109.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">                </span><span id="lstnumberx109.2" class="ltx_text" style="color:#9400D1;">factors.append((p,</span><span id="lstnumberx109.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx109.4" class="ltx_text" style="color:#9400D1;">1))</span>
</span>
<span id="lstnumberx110" class="ltx_listingline"><span id="lstnumberx110.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx110.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx110.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx110.4" class="ltx_text" style="color:#9400D1;">x</span><span id="lstnumberx110.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx110.6" class="ltx_text" style="color:#9400D1;">!=</span><span id="lstnumberx110.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx110.8" class="ltx_text" style="color:#9400D1;">1:</span>
</span>
<span id="lstnumberx111" class="ltx_listingline"><span id="lstnumberx111.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">            </span><span id="lstnumberx111.2" class="ltx_text" style="color:#9400D1;">factors.append((x,</span><span id="lstnumberx111.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx111.4" class="ltx_text" style="color:#9400D1;">1))</span>
</span>
<span id="lstnumberx112" class="ltx_listingline"><span id="lstnumberx112.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx112.2" class="ltx_text" style="color:#9400D1;">factorized.append(factors)</span>
</span>
<span id="lstnumberx113" class="ltx_listingline"><span id="lstnumberx113.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx113.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx113.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx113.4" class="ltx_text" style="color:#9400D1;">factorized</span>
</span>
<span id="lstnumberx114" class="ltx_listingline">
</span>
<span id="lstnumberx115" class="ltx_listingline"><span id="lstnumberx115.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx115.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx115.3" class="ltx_text" style="color:#9400D1;">find_leaves(neighbours):</span>
</span>
<span id="lstnumberx116" class="ltx_listingline"><span id="lstnumberx116.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx116.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx117" class="ltx_listingline"><span id="lstnumberx117.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx117.2" class="ltx_text" style="color:#9400D1;">Find</span><span id="lstnumberx117.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx117.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.6" class="ltx_text" style="color:#9400D1;">leaf</span><span id="lstnumberx117.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.8" class="ltx_text" style="color:#9400D1;">nodes</span><span id="lstnumberx117.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.10" class="ltx_text" style="color:#9400D1;">(vertices</span><span id="lstnumberx117.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.12" class="ltx_text" style="color:#9400D1;">with</span><span id="lstnumberx117.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.14" class="ltx_text" style="color:#9400D1;">only</span><span id="lstnumberx117.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.16" class="ltx_text" style="color:#9400D1;">one</span><span id="lstnumberx117.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.18" class="ltx_text" style="color:#9400D1;">adjacent</span><span id="lstnumberx117.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.20" class="ltx_text" style="color:#9400D1;">vertex)</span><span id="lstnumberx117.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.22" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx117.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.24" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx117.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx117.26" class="ltx_text" style="color:#9400D1;">tree.</span>
</span>
<span id="lstnumberx118" class="ltx_listingline"><span id="lstnumberx118.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx118.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx119" class="ltx_listingline"><span id="lstnumberx119.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx119.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx119.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.4" class="ltx_text" style="color:#9400D1;">neighbours:</span><span id="lstnumberx119.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx119.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.8" class="ltx_text" style="color:#9400D1;">adjacency</span><span id="lstnumberx119.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.10" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx119.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.12" class="ltx_text" style="color:#9400D1;">representing</span><span id="lstnumberx119.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.14" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx119.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx119.16" class="ltx_text" style="color:#9400D1;">tree.</span>
</span>
<span id="lstnumberx120" class="ltx_listingline"><span id="lstnumberx120.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx120.2" class="ltx_text" style="color:#9400D1;">Returns:</span>
</span>
<span id="lstnumberx121" class="ltx_listingline"><span id="lstnumberx121.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx121.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx121.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.4" class="ltx_text" style="color:#9400D1;">leaves:</span><span id="lstnumberx121.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.6" class="ltx_text" style="color:#9400D1;">A</span><span id="lstnumberx121.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx121.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx121.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.12" class="ltx_text" style="color:#9400D1;">leaf</span><span id="lstnumberx121.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.14" class="ltx_text" style="color:#9400D1;">nodes</span><span id="lstnumberx121.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.16" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx121.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.18" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx121.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx121.20" class="ltx_text" style="color:#9400D1;">tree.</span>
</span>
<span id="lstnumberx122" class="ltx_listingline"><span id="lstnumberx122.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx122.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx123" class="ltx_listingline"><span id="lstnumberx123.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx123.2" class="ltx_text" style="color:#9400D1;">leaves</span><span id="lstnumberx123.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx123.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx123.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx123.6" class="ltx_text" style="color:#9400D1;">[]</span>
</span>
<span id="lstnumberx124" class="ltx_listingline"><span id="lstnumberx124.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx124.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx124.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx124.4" class="ltx_text" style="color:#9400D1;">i,</span><span id="lstnumberx124.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx124.6" class="ltx_text" style="color:#9400D1;">neighbours_list</span><span id="lstnumberx124.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx124.8" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx124.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx124.10" class="ltx_text" style="color:#9400D1;">enumerate(neighbours):</span>
</span>
<span id="lstnumberx125" class="ltx_listingline"><span id="lstnumberx125.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx125.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx125.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx125.4" class="ltx_text" style="color:#9400D1;">len(neighbours_list)</span><span id="lstnumberx125.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx125.6" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx125.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx125.8" class="ltx_text" style="color:#9400D1;">1:</span>
</span>
<span id="lstnumberx126" class="ltx_listingline"><span id="lstnumberx126.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">            </span><span id="lstnumberx126.2" class="ltx_text" style="color:#9400D1;">leaves.append(i)</span>
</span>
<span id="lstnumberx127" class="ltx_listingline"><span id="lstnumberx127.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx127.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx127.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx127.4" class="ltx_text" style="color:#9400D1;">leaves</span>
</span>
<span id="lstnumberx128" class="ltx_listingline"><span id="lstnumberx128.1" class="ltx_text" style="color:#9400D1;">#…[TRUNCATED]</span>
</span>
</span>
<span id="A4.T7.13.16.1.1.1.3" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T7.13.17" class="ltx_tr">
<td id="A4.T7.13.17.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T7.13.17.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.13.17.1.1.1" class="ltx_p" style="width:433.6pt;">{{Problem}}</span>
</span>
</td>
</tr>
<tr id="A4.T7.13.18" class="ltx_tr">
<td id="A4.T7.13.18.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T7.13.18.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.13.18.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T7.13.18.1.1.1.1" class="ltx_text ltx_font_bold">Initial Solution</span></span>
</span>
</td>
</tr>
<tr id="A4.T7.13.19" class="ltx_tr">
<td id="A4.T7.13.19.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_b ltx_border_l ltx_border_r">
<span id="A4.T7.13.19.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T7.13.19.1.1.1" class="ltx_p" style="width:433.6pt;">{{Initial Solution}}</span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 7: </span>Excerpt of the input prompt for generating vanilla decomposition.</figcaption>
</figure>
</section>
<section id="A4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">D.3 </span>Critique</h3>

<div id="A4.SS3.p1" class="ltx_para">
<p id="A4.SS3.p1.1" class="ltx_p">For generating critique on decomposition, we adopt the prompt in Table <a href="#A4.T8" title="Table 8 ‣ D.3 Critique ‣ Appendix D Example Prompts ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>.</p>
</div>
<figure id="A4.T8" class="ltx_table">
<table id="A4.T8.7" class="ltx_tabular ltx_align_middle">
<tr id="A4.T8.7.8" class="ltx_tr">
<td id="A4.T8.7.8.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r ltx_border_t">
<span id="A4.T8.7.8.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.8.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T8.7.8.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T8.3.3" class="ltx_tr">
<td id="A4.T8.3.3.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.3.3.3.3" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.1.1.1.1.1" class="ltx_p" style="width:433.6pt;">Two pirates Polycarpus and Vasily play a very interesting game. They have n chests with coins, the chests are numbered with integers from 1 to n. Chest number i has <math id="A4.T8.1.1.1.1.1.m1.1" class="ltx_Math" alttext="a_{i}" display="inline"><semantics id="A4.T8.1.1.1.1.1.m1.1a"><msub id="A4.T8.1.1.1.1.1.m1.1.1" xref="A4.T8.1.1.1.1.1.m1.1.1.cmml"><mi id="A4.T8.1.1.1.1.1.m1.1.1.2" xref="A4.T8.1.1.1.1.1.m1.1.1.2.cmml">a</mi><mi id="A4.T8.1.1.1.1.1.m1.1.1.3" xref="A4.T8.1.1.1.1.1.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A4.T8.1.1.1.1.1.m1.1b"><apply id="A4.T8.1.1.1.1.1.m1.1.1.cmml" xref="A4.T8.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T8.1.1.1.1.1.m1.1.1.1.cmml" xref="A4.T8.1.1.1.1.1.m1.1.1">subscript</csymbol><ci id="A4.T8.1.1.1.1.1.m1.1.1.2.cmml" xref="A4.T8.1.1.1.1.1.m1.1.1.2">𝑎</ci><ci id="A4.T8.1.1.1.1.1.m1.1.1.3.cmml" xref="A4.T8.1.1.1.1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T8.1.1.1.1.1.m1.1c">a_{i}</annotation></semantics></math> coins.</span>
<span id="A4.T8.3.3.3.3.3" class="ltx_p">Polycarpus and Vasily move in turns. Polycarpus moves first. During a move a player is allowed to choose a positive integer <math id="A4.T8.2.2.2.2.2.m1.1" class="ltx_Math" alttext="x(2·x+1\leq n)" display="inline"><semantics id="A4.T8.2.2.2.2.2.m1.1a"><mrow id="A4.T8.2.2.2.2.2.m1.1.1" xref="A4.T8.2.2.2.2.2.m1.1.1.cmml"><mi id="A4.T8.2.2.2.2.2.m1.1.1.3" xref="A4.T8.2.2.2.2.2.m1.1.1.3.cmml">x</mi><mo lspace="0em" rspace="0em" id="A4.T8.2.2.2.2.2.m1.1.1.2" xref="A4.T8.2.2.2.2.2.m1.1.1.2.cmml">​</mo><mrow id="A4.T8.2.2.2.2.2.m1.1.1.1.1" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.cmml"><mo stretchy="false" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.2" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.cmml">(</mo><mrow id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.cmml"><mrow id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.cmml"><mrow id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.cmml"><mn id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.2" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.1" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml">​</mo><mi mathvariant="normal" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.3" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.1a" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml">​</mo><mi id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.4" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.4.cmml">x</mi></mrow><mo id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.1" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.1.cmml">+</mo><mn id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.3" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.3.cmml">1</mn></mrow><mo id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.1" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.1.cmml">≤</mo><mi id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.3" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.3.cmml">n</mi></mrow><mo stretchy="false" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.3" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T8.2.2.2.2.2.m1.1b"><apply id="A4.T8.2.2.2.2.2.m1.1.1.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1"><times id="A4.T8.2.2.2.2.2.m1.1.1.2.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.2"></times><ci id="A4.T8.2.2.2.2.2.m1.1.1.3.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.3">𝑥</ci><apply id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1"><leq id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.1.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.1"></leq><apply id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2"><plus id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.1.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.1"></plus><apply id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2"><times id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.1"></times><cn type="integer" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.2.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.2">2</cn><ci id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.3.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.3">·</ci><ci id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.4.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.2.4">𝑥</ci></apply><cn type="integer" id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.3.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.2.3">1</cn></apply><ci id="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.3.cmml" xref="A4.T8.2.2.2.2.2.m1.1.1.1.1.1.3">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T8.2.2.2.2.2.m1.1c">x(2·x+1\leq n)</annotation></semantics></math> and take a coin from each chest with numbers <math id="A4.T8.3.3.3.3.3.m2.3" class="ltx_Math" alttext="x,2·x,2·x+1" display="inline"><semantics id="A4.T8.3.3.3.3.3.m2.3a"><mrow id="A4.T8.3.3.3.3.3.m2.3.3.2" xref="A4.T8.3.3.3.3.3.m2.3.3.3.cmml"><mi id="A4.T8.3.3.3.3.3.m2.1.1" xref="A4.T8.3.3.3.3.3.m2.1.1.cmml">x</mi><mo id="A4.T8.3.3.3.3.3.m2.3.3.2.3" xref="A4.T8.3.3.3.3.3.m2.3.3.3.cmml">,</mo><mrow id="A4.T8.3.3.3.3.3.m2.2.2.1.1" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.cmml"><mn id="A4.T8.3.3.3.3.3.m2.2.2.1.1.2" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T8.3.3.3.3.3.m2.2.2.1.1.1" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.1.cmml">​</mo><mi mathvariant="normal" id="A4.T8.3.3.3.3.3.m2.2.2.1.1.3" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T8.3.3.3.3.3.m2.2.2.1.1.1a" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.1.cmml">​</mo><mi id="A4.T8.3.3.3.3.3.m2.2.2.1.1.4" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.4.cmml">x</mi></mrow><mo id="A4.T8.3.3.3.3.3.m2.3.3.2.4" xref="A4.T8.3.3.3.3.3.m2.3.3.3.cmml">,</mo><mrow id="A4.T8.3.3.3.3.3.m2.3.3.2.2" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.cmml"><mrow id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.cmml"><mn id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.2" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.1" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.1.cmml">​</mo><mi mathvariant="normal" id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.3" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.1a" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.1.cmml">​</mo><mi id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.4" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.4.cmml">x</mi></mrow><mo id="A4.T8.3.3.3.3.3.m2.3.3.2.2.1" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.1.cmml">+</mo><mn id="A4.T8.3.3.3.3.3.m2.3.3.2.2.3" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.3.cmml">1</mn></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T8.3.3.3.3.3.m2.3b"><list id="A4.T8.3.3.3.3.3.m2.3.3.3.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2"><ci id="A4.T8.3.3.3.3.3.m2.1.1.cmml" xref="A4.T8.3.3.3.3.3.m2.1.1">𝑥</ci><apply id="A4.T8.3.3.3.3.3.m2.2.2.1.1.cmml" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1"><times id="A4.T8.3.3.3.3.3.m2.2.2.1.1.1.cmml" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.1"></times><cn type="integer" id="A4.T8.3.3.3.3.3.m2.2.2.1.1.2.cmml" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.2">2</cn><ci id="A4.T8.3.3.3.3.3.m2.2.2.1.1.3.cmml" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.3">·</ci><ci id="A4.T8.3.3.3.3.3.m2.2.2.1.1.4.cmml" xref="A4.T8.3.3.3.3.3.m2.2.2.1.1.4">𝑥</ci></apply><apply id="A4.T8.3.3.3.3.3.m2.3.3.2.2.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2"><plus id="A4.T8.3.3.3.3.3.m2.3.3.2.2.1.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.1"></plus><apply id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2"><times id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.1.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.1"></times><cn type="integer" id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.2.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.2">2</cn><ci id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.3.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.3">·</ci><ci id="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.4.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.2.4">𝑥</ci></apply><cn type="integer" id="A4.T8.3.3.3.3.3.m2.3.3.2.2.3.cmml" xref="A4.T8.3.3.3.3.3.m2.3.3.2.2.3">1</cn></apply></list></annotation-xml><annotation encoding="application/x-tex" id="A4.T8.3.3.3.3.3.m2.3c">x,2·x,2·x+1</annotation></semantics></math>. It may turn out that some chest has no coins, in this case the player doesn’t take a coin from this chest. The game finishes when all chests get emptied…[TRUNCATED]</span>
</span>
</td>
</tr>
<tr id="A4.T8.7.9" class="ltx_tr">
<td id="A4.T8.7.9.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.9.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.9.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T8.7.9.1.1.1.1" class="ltx_text ltx_font_bold">Initial Solution</span>

<span id="A4.T8.7.9.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,ZGVmIGlzX3ZhbGlkX2lucHV0KG4sIGEpOgogICAgIiIiCiAgICBDaGVjayBpZiB0aGUgaW5wdXQgaXMgdmFsaWQgZm9yIHRoZSBnaXZlbiBjb25zdHJhaW50cy4KCiAgICBQYXJhbWV0ZXJzOgogICAgLSBuOiBUaGUgbnVtYmVyIG9mIGNoZXN0cyB3aXRoIGNvaW5zLgogICAgLSBhOiBUaGUgbGlzdCBvZiBudW1iZXIgb2YgY29pbnMgaW4gZWFjaCBjaGVzdC4KCiAgICBSZXR1cm5zOgogICAgLSBUcnVlIGlmIHRoZSBpbnB1dCBpcyB2YWxpZCwgRmFsc2Ugb3RoZXJ3aXNlLgogICAgIiIiCiAgICByZXR1cm4gbiAlIDIgPT0gMQoKZGVmIGluaXRpYWxpemVfY2hlc3RzKGEpOgogICAgIiIiCiAgICBJbml0aWFsaXplIHRoZSBsaXN0IG9mIGNoZXN0cyB3aXRoIGNvaW5zLCBhZGRpbmcgZXh0cmEgZWxlbWVudHMgYXQgdGhlIGJlZ2lubmluZyBhbmQgZW5kLgoKICAgIFBhcmFtZXRlcnM6CiAgICAtIGE6IFRoZSBsaXN0IG9mIG51bWJlciBvZiBjb2lucyBpbiBlYWNoIGNoZXN0LgoKICAgIFJldHVybnM6CiAgICAtIFRoZSBtb2RpZmllZCBsaXN0IG9mIGNoZXN0cy4KICAgICIiIgogICAgcmV0dXJuIFswXSArIGEgKyBbMCwgMF0KCmRlZiBtYWtlX21vdmUoYSwgaSk6CiAgICAiIiIKICAgIFBlcmZvcm0gYSBtb3ZlIGluIHRoZSBnYW1lLCB1cGRhdGluZyB0aGUgbnVtYmVyIG9mIGNvaW5zIGluIHRoZSBjaGVzdHMgYWNjb3JkaW5nbHkuCgogICAgUGFyYW1ldGVyczoKICAgIC0gYTogVGhlIGxpc3Qgb2YgbnVtYmVyIG9mIGNvaW5zIGluIGVhY2ggY2hlc3QuCiAgICAtIGk6IFRoZSBpbmRleCBvZiB0aGUgY2hlc3QgdG8gcGVyZm9ybSB0aGUgbW92ZSBvbi4KICAgICIiIgogICAgbSA9IG1pbihhW2ldLCBhW2kgLSAxXSkKICAgIGFbaSAvLyAyXSA9IG1heCgwLCBhW2kgLy8gMl0gLSBtKQogICAgYVtpXSAtPSBtCiAgICBhW2kgLSAxXSAtPSBtCgpkZWYgY2FsY3VsYXRlX21pbmltdW1fbW92ZXMobiwgYSk6CiAgICAiIiIKICAgIENhbGN1bGF0ZSB0aGUgbWluaW11bSBudW1iZXIgb2YgbW92ZXMgbmVlZGVkIHRvIGZpbmlzaCB0aGUgZ2FtZS4KCiAgICBQYXJhbWV0ZXJzOgogICAgLSBuOiBUaGUgbnVtYmVyIG9mIGNoZXN0cyB3aXRoIGNvaW5zLgogICAgLSBhOiBUaGUgbGlzdCBvZiBudW1iZXIgb2YgY29pbnMgaW4gZWFjaCBjaGVzdC4KCiAgICBSZXR1cm5zOgogICAgLSBUaGUgbWluaW11bSBudW1iZXIgb2YgbW92ZXMgbmVlZGVkIHRvIGZpbmlzaCB0aGUgZ2FtZS4KICAgICIiIgogICAgaWYgbiA9PSAxIG9yIG5vdCBpc192YWxpZF9pbnB1dChuLCBhKToKICAgICAgICByZXR1cm4gLTEKICAgIGEgPSBpbml0aWFsaXplX2NoZXN0cyhhKQogICAgZm9yIGkgaW4gcmFuZ2UobiwgMCwgLTIpOgogICAgICAgIG1ha2VfbW92ZShhLCBpKQogICAgcmV0dXJuIHN1bShhKQojLi4uW1RSVU5DQVRFRF0=" download="">⬇</a></span>
<span id="lstnumberx129" class="ltx_listingline"><span id="lstnumberx129.1" class="ltx_text ltx_lst_identifier">def</span><span id="lstnumberx129.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.3" class="ltx_text ltx_lst_identifier">is_valid_input</span>(<span id="lstnumberx129.4" class="ltx_text ltx_lst_identifier">n</span>,<span id="lstnumberx129.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.6" class="ltx_text ltx_lst_identifier">a</span>):
</span>
<span id="lstnumberx130" class="ltx_listingline"><span id="lstnumberx130.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx130.2" class="ltx_text ltx_lst_string" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx131" class="ltx_listingline"><span id="lstnumberx131.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx131.2" class="ltx_text" style="color:#9400D1;">Check</span><span id="lstnumberx131.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.4" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx131.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.6" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx131.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.8" class="ltx_text" style="color:#9400D1;">input</span><span id="lstnumberx131.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.10" class="ltx_text" style="color:#9400D1;">is</span><span id="lstnumberx131.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.12" class="ltx_text" style="color:#9400D1;">valid</span><span id="lstnumberx131.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.14" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx131.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.16" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx131.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.18" class="ltx_text" style="color:#9400D1;">given</span><span id="lstnumberx131.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx131.20" class="ltx_text" style="color:#9400D1;">constraints.</span>
</span>
<span id="lstnumberx132" class="ltx_listingline">
</span>
<span id="lstnumberx133" class="ltx_listingline"><span id="lstnumberx133.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx133.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx134" class="ltx_listingline"><span id="lstnumberx134.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx134.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx134.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.4" class="ltx_text" style="color:#9400D1;">n:</span><span id="lstnumberx134.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx134.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx134.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx134.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.12" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx134.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.14" class="ltx_text" style="color:#9400D1;">with</span><span id="lstnumberx134.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx134.16" class="ltx_text" style="color:#9400D1;">coins.</span>
</span>
<span id="lstnumberx135" class="ltx_listingline"><span id="lstnumberx135.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx135.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx135.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.4" class="ltx_text" style="color:#9400D1;">a:</span><span id="lstnumberx135.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx135.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx135.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx135.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.12" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx135.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.14" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx135.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.16" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx135.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.18" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx135.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.20" class="ltx_text" style="color:#9400D1;">each</span><span id="lstnumberx135.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx135.22" class="ltx_text" style="color:#9400D1;">chest.</span>
</span>
<span id="lstnumberx136" class="ltx_listingline">
</span>
<span id="lstnumberx137" class="ltx_listingline"><span id="lstnumberx137.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx137.2" class="ltx_text" style="color:#9400D1;">Returns:</span>
</span>
<span id="lstnumberx138" class="ltx_listingline"><span id="lstnumberx138.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx138.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx138.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.4" class="ltx_text" style="color:#9400D1;">True</span><span id="lstnumberx138.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.6" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx138.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.8" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx138.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.10" class="ltx_text" style="color:#9400D1;">input</span><span id="lstnumberx138.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.12" class="ltx_text" style="color:#9400D1;">is</span><span id="lstnumberx138.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.14" class="ltx_text" style="color:#9400D1;">valid,</span><span id="lstnumberx138.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.16" class="ltx_text" style="color:#9400D1;">False</span><span id="lstnumberx138.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx138.18" class="ltx_text" style="color:#9400D1;">otherwise.</span>
</span>
<span id="lstnumberx139" class="ltx_listingline"><span id="lstnumberx139.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx139.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx140" class="ltx_listingline"><span id="lstnumberx140.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx140.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx140.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx140.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx140.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx140.6" class="ltx_text" style="color:#9400D1;">%</span><span id="lstnumberx140.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx140.8" class="ltx_text" style="color:#9400D1;">2</span><span id="lstnumberx140.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx140.10" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx140.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx140.12" class="ltx_text" style="color:#9400D1;">1</span>
</span>
<span id="lstnumberx141" class="ltx_listingline">
</span>
<span id="lstnumberx142" class="ltx_listingline"><span id="lstnumberx142.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx142.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx142.3" class="ltx_text" style="color:#9400D1;">initialize_chests(a):</span>
</span>
<span id="lstnumberx143" class="ltx_listingline"><span id="lstnumberx143.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx143.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx144" class="ltx_listingline"><span id="lstnumberx144.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx144.2" class="ltx_text" style="color:#9400D1;">Initialize</span><span id="lstnumberx144.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx144.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.6" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx144.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.8" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx144.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.10" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx144.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.12" class="ltx_text" style="color:#9400D1;">with</span><span id="lstnumberx144.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.14" class="ltx_text" style="color:#9400D1;">coins,</span><span id="lstnumberx144.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.16" class="ltx_text" style="color:#9400D1;">adding</span><span id="lstnumberx144.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.18" class="ltx_text" style="color:#9400D1;">extra</span><span id="lstnumberx144.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.20" class="ltx_text" style="color:#9400D1;">elements</span><span id="lstnumberx144.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.22" class="ltx_text" style="color:#9400D1;">at</span><span id="lstnumberx144.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.24" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx144.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.26" class="ltx_text" style="color:#9400D1;">beginning</span><span id="lstnumberx144.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.28" class="ltx_text" style="color:#9400D1;">and</span><span id="lstnumberx144.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx144.30" class="ltx_text" style="color:#9400D1;">end.</span>
</span>
<span id="lstnumberx145" class="ltx_listingline">
</span>
<span id="lstnumberx146" class="ltx_listingline"><span id="lstnumberx146.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx146.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx147" class="ltx_listingline"><span id="lstnumberx147.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx147.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx147.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.4" class="ltx_text" style="color:#9400D1;">a:</span><span id="lstnumberx147.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx147.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx147.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx147.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.12" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx147.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.14" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx147.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.16" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx147.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.18" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx147.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.20" class="ltx_text" style="color:#9400D1;">each</span><span id="lstnumberx147.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx147.22" class="ltx_text" style="color:#9400D1;">chest.</span>
</span>
<span id="lstnumberx148" class="ltx_listingline">
</span>
<span id="lstnumberx149" class="ltx_listingline"><span id="lstnumberx149.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx149.2" class="ltx_text" style="color:#9400D1;">Returns:</span>
</span>
<span id="lstnumberx150" class="ltx_listingline"><span id="lstnumberx150.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx150.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx150.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx150.4" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx150.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx150.6" class="ltx_text" style="color:#9400D1;">modified</span><span id="lstnumberx150.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx150.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx150.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx150.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx150.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx150.12" class="ltx_text" style="color:#9400D1;">chests.</span>
</span>
<span id="lstnumberx151" class="ltx_listingline"><span id="lstnumberx151.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx151.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx152" class="ltx_listingline"><span id="lstnumberx152.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx152.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx152.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx152.4" class="ltx_text" style="color:#9400D1;">[0]</span><span id="lstnumberx152.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx152.6" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx152.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx152.8" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx152.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx152.10" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx152.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx152.12" class="ltx_text" style="color:#9400D1;">[0,</span><span id="lstnumberx152.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx152.14" class="ltx_text" style="color:#9400D1;">0]</span>
</span>
<span id="lstnumberx153" class="ltx_listingline">
</span>
<span id="lstnumberx154" class="ltx_listingline"><span id="lstnumberx154.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx154.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx154.3" class="ltx_text" style="color:#9400D1;">make_move(a,</span><span id="lstnumberx154.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx154.5" class="ltx_text" style="color:#9400D1;">i):</span>
</span>
<span id="lstnumberx155" class="ltx_listingline"><span id="lstnumberx155.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx155.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx156" class="ltx_listingline"><span id="lstnumberx156.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx156.2" class="ltx_text" style="color:#9400D1;">Perform</span><span id="lstnumberx156.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.4" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx156.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.6" class="ltx_text" style="color:#9400D1;">move</span><span id="lstnumberx156.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.8" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx156.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.10" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx156.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.12" class="ltx_text" style="color:#9400D1;">game,</span><span id="lstnumberx156.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.14" class="ltx_text" style="color:#9400D1;">updating</span><span id="lstnumberx156.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.16" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx156.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.18" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx156.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.20" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx156.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.22" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx156.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.24" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx156.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.26" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx156.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.28" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx156.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx156.30" class="ltx_text" style="color:#9400D1;">accordingly.</span>
</span>
<span id="lstnumberx157" class="ltx_listingline">
</span>
<span id="lstnumberx158" class="ltx_listingline"><span id="lstnumberx158.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx158.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx159" class="ltx_listingline"><span id="lstnumberx159.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx159.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx159.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.4" class="ltx_text" style="color:#9400D1;">a:</span><span id="lstnumberx159.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx159.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx159.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx159.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.12" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx159.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.14" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx159.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.16" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx159.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.18" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx159.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.20" class="ltx_text" style="color:#9400D1;">each</span><span id="lstnumberx159.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx159.22" class="ltx_text" style="color:#9400D1;">chest.</span>
</span>
<span id="lstnumberx160" class="ltx_listingline"><span id="lstnumberx160.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx160.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx160.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.4" class="ltx_text" style="color:#9400D1;">i:</span><span id="lstnumberx160.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx160.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.8" class="ltx_text" style="color:#9400D1;">index</span><span id="lstnumberx160.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx160.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.12" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx160.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.14" class="ltx_text" style="color:#9400D1;">chest</span><span id="lstnumberx160.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx160.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.18" class="ltx_text" style="color:#9400D1;">perform</span><span id="lstnumberx160.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx160.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.22" class="ltx_text" style="color:#9400D1;">move</span><span id="lstnumberx160.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx160.24" class="ltx_text" style="color:#9400D1;">on.</span>
</span>
<span id="lstnumberx161" class="ltx_listingline"><span id="lstnumberx161.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx161.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx162" class="ltx_listingline"><span id="lstnumberx162.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx162.2" class="ltx_text" style="color:#9400D1;">m</span><span id="lstnumberx162.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx162.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx162.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx162.6" class="ltx_text" style="color:#9400D1;">min(a[i],</span><span id="lstnumberx162.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx162.8" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx162.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx162.10" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx162.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx162.12" class="ltx_text" style="color:#9400D1;">1])</span>
</span>
<span id="lstnumberx163" class="ltx_listingline"><span id="lstnumberx163.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx163.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx163.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.4" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx163.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.6" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx163.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.8" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx163.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.10" class="ltx_text" style="color:#9400D1;">max(0,</span><span id="lstnumberx163.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.12" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx163.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.14" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx163.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.16" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx163.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.18" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx163.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx163.20" class="ltx_text" style="color:#9400D1;">m)</span>
</span>
<span id="lstnumberx164" class="ltx_listingline"><span id="lstnumberx164.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx164.2" class="ltx_text" style="color:#9400D1;">a[i]</span><span id="lstnumberx164.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx164.4" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx164.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx164.6" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx165" class="ltx_listingline"><span id="lstnumberx165.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx165.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx165.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx165.4" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx165.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx165.6" class="ltx_text" style="color:#9400D1;">1]</span><span id="lstnumberx165.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx165.8" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx165.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx165.10" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx166" class="ltx_listingline">
</span>
<span id="lstnumberx167" class="ltx_listingline"><span id="lstnumberx167.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx167.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx167.3" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx167.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx167.5" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx168" class="ltx_listingline"><span id="lstnumberx168.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx168.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx169" class="ltx_listingline"><span id="lstnumberx169.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx169.2" class="ltx_text" style="color:#9400D1;">Calculate</span><span id="lstnumberx169.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx169.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.6" class="ltx_text" style="color:#9400D1;">minimum</span><span id="lstnumberx169.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx169.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx169.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.12" class="ltx_text" style="color:#9400D1;">moves</span><span id="lstnumberx169.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.14" class="ltx_text" style="color:#9400D1;">needed</span><span id="lstnumberx169.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx169.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.18" class="ltx_text" style="color:#9400D1;">finish</span><span id="lstnumberx169.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx169.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx169.22" class="ltx_text" style="color:#9400D1;">game.</span>
</span>
<span id="lstnumberx170" class="ltx_listingline">
</span>
<span id="lstnumberx171" class="ltx_listingline"><span id="lstnumberx171.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx171.2" class="ltx_text" style="color:#9400D1;">Parameters:</span>
</span>
<span id="lstnumberx172" class="ltx_listingline"><span id="lstnumberx172.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx172.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx172.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.4" class="ltx_text" style="color:#9400D1;">n:</span><span id="lstnumberx172.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx172.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx172.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx172.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.12" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx172.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.14" class="ltx_text" style="color:#9400D1;">with</span><span id="lstnumberx172.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx172.16" class="ltx_text" style="color:#9400D1;">coins.</span>
</span>
<span id="lstnumberx173" class="ltx_listingline"><span id="lstnumberx173.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx173.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx173.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.4" class="ltx_text" style="color:#9400D1;">a:</span><span id="lstnumberx173.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.6" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx173.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.8" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx173.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx173.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.12" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx173.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.14" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx173.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.16" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx173.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.18" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx173.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.20" class="ltx_text" style="color:#9400D1;">each</span><span id="lstnumberx173.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx173.22" class="ltx_text" style="color:#9400D1;">chest.</span>
</span>
<span id="lstnumberx174" class="ltx_listingline">
</span>
<span id="lstnumberx175" class="ltx_listingline"><span id="lstnumberx175.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx175.2" class="ltx_text" style="color:#9400D1;">Returns:</span>
</span>
<span id="lstnumberx176" class="ltx_listingline"><span id="lstnumberx176.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx176.2" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx176.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.4" class="ltx_text" style="color:#9400D1;">The</span><span id="lstnumberx176.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.6" class="ltx_text" style="color:#9400D1;">minimum</span><span id="lstnumberx176.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx176.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx176.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.12" class="ltx_text" style="color:#9400D1;">moves</span><span id="lstnumberx176.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.14" class="ltx_text" style="color:#9400D1;">needed</span><span id="lstnumberx176.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx176.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.18" class="ltx_text" style="color:#9400D1;">finish</span><span id="lstnumberx176.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx176.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx176.22" class="ltx_text" style="color:#9400D1;">game.</span>
</span>
<span id="lstnumberx177" class="ltx_listingline"><span id="lstnumberx177.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx177.2" class="ltx_text" style="color:#9400D1;">"""</span>
</span>
<span id="lstnumberx178" class="ltx_listingline"><span id="lstnumberx178.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx178.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx178.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx178.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.6" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx178.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.8" class="ltx_text" style="color:#9400D1;">1</span><span id="lstnumberx178.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.10" class="ltx_text" style="color:#9400D1;">or</span><span id="lstnumberx178.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.12" class="ltx_text" style="color:#9400D1;">not</span><span id="lstnumberx178.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.14" class="ltx_text" style="color:#9400D1;">is_valid_input(n,</span><span id="lstnumberx178.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx178.16" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx179" class="ltx_listingline"><span id="lstnumberx179.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx179.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx179.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx179.4" class="ltx_text" style="color:#9400D1;">-1</span>
</span>
<span id="lstnumberx180" class="ltx_listingline"><span id="lstnumberx180.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx180.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx180.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx180.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx180.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx180.6" class="ltx_text" style="color:#9400D1;">initialize_chests(a)</span>
</span>
<span id="lstnumberx181" class="ltx_listingline"><span id="lstnumberx181.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx181.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx181.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx181.4" class="ltx_text" style="color:#9400D1;">i</span><span id="lstnumberx181.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx181.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx181.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx181.8" class="ltx_text" style="color:#9400D1;">range(n,</span><span id="lstnumberx181.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx181.10" class="ltx_text" style="color:#9400D1;">0,</span><span id="lstnumberx181.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx181.12" class="ltx_text" style="color:#9400D1;">-2):</span>
</span>
<span id="lstnumberx182" class="ltx_listingline"><span id="lstnumberx182.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx182.2" class="ltx_text" style="color:#9400D1;">make_move(a,</span><span id="lstnumberx182.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx182.4" class="ltx_text" style="color:#9400D1;">i)</span>
</span>
<span id="lstnumberx183" class="ltx_listingline"><span id="lstnumberx183.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx183.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx183.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx183.4" class="ltx_text" style="color:#9400D1;">sum(a)</span>
</span>
<span id="lstnumberx184" class="ltx_listingline"><span id="lstnumberx184.1" class="ltx_text" style="color:#9400D1;">#…[TRUNCATED]</span>
</span>
</span></span>
<span id="A4.T8.7.9.1.1.2" class="ltx_p"><span id="A4.T8.7.9.1.1.2.1" class="ltx_text ltx_font_bold">Critique Request</span></span>
</span>
</td>
</tr>
<tr id="A4.T8.7.10" class="ltx_tr">
<td id="A4.T8.7.10.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.10.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.10.1.1.1" class="ltx_p" style="width:433.6pt;">Identify specific ways in which the decomposition strategy of the initial solution is less effective in assisting humans to debug the solution.</span>
</span>
</td>
</tr>
<tr id="A4.T8.7.11" class="ltx_tr">
<td id="A4.T8.7.11.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.11.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.11.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T8.7.11.1.1.1.1" class="ltx_text ltx_font_bold">Critique</span></span>
</span>
</td>
</tr>
<tr id="A4.T8.5.5" class="ltx_tr">
<td id="A4.T8.5.5.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.5.5.2.2" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.5.5.2.2.2" class="ltx_p" style="width:433.6pt;">1. Decomposition should be around logical breakpoints, but not syntax. However,  <svg id="A4.T8.4.4.1.1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="82.81"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(41.4,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 38.36 9.69 L -38.36 9.69 C -39.89 9.69 -41.13 8.45 -41.13 6.92 L -41.13 -1.4 C -41.13 -2.93 -39.89 -4.17 -38.36 -4.17 L 38.36 -4.17 C 39.89 -4.17 41.13 -2.93 41.13 -1.4 L 41.13 6.92 C 41.13 8.45 39.89 9.69 38.36 9.69 Z M -41.13 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -36.51 0)" fill="#000000" stroke="#000000"><foreignObject width="73.8" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T8.4.4.1.1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">make_move</span></foreignObject></g></g></svg> and  <svg id="A4.T8.5.5.2.2.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="107.56"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(53.78,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 50.74 9.69 L -50.74 9.69 C -52.26 9.69 -53.5 8.45 -53.5 6.92 L -53.5 -1.4 C -53.5 -2.93 -52.26 -4.17 -50.74 -4.17 L 50.74 -4.17 C 52.26 -4.17 53.5 -2.93 53.5 -1.4 L 53.5 6.92 C 53.5 8.45 52.26 9.69 50.74 9.69 Z M -53.5 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -48.89 0)" fill="#000000" stroke="#000000"><foreignObject width="97.78" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T8.5.5.2.2.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">initialize_chests</span></foreignObject></g></g></svg> are not the main logic steps of this problem. We should present a clear high-level logic that aligns with how humans reason about programs conceptually. In this problem, the main logic steps are input reading, checking game finishing conditions, calculating minimum steps, and output printing.</span>
</span>
</td>
</tr>
<tr id="A4.T8.7.7" class="ltx_tr">
<td id="A4.T8.7.7.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.7.2.2" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.7.2.2.2" class="ltx_p" style="width:433.6pt;">2. Boundary conditions (i.e.,  <svg id="A4.T8.6.6.1.1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="172.36"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(86.18,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 83.14 9.69 L -83.14 9.69 C -84.67 9.69 -85.9 8.45 -85.9 6.92 L -85.9 -1.4 C -85.9 -2.93 -84.67 -4.17 -83.14 -4.17 L 83.14 -4.17 C 84.67 -4.17 85.9 -2.93 85.9 -1.4 L 85.9 6.92 C 85.9 8.45 84.67 9.69 83.14 9.69 Z M -85.9 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -81.29 0)" fill="#000000" stroke="#000000"><foreignObject width="162.59" height="11.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T8.6.6.1.1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">if n == 1 and n % 2 == 1</span></foreignObject></g></g></svg> should be explicitly highlighted as individual functions in the main logic, instead of in the function  <svg id="A4.T8.7.7.2.2.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="177.44"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(88.72,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 85.67 9.69 L -85.67 9.69 C -87.2 9.69 -88.44 8.45 -88.44 6.92 L -88.44 -1.4 C -88.44 -2.93 -87.2 -4.17 -85.67 -4.17 L 85.67 -4.17 C 87.2 -4.17 88.44 -2.93 88.44 -1.4 L 88.44 6.92 C 88.44 8.45 87.2 9.69 85.67 9.69 Z M -88.44 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -83.83 0)" fill="#000000" stroke="#000000"><foreignObject width="168.43" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T8.7.7.2.2.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">calculate_minimum_moves</span></foreignObject></g></g></svg>.</span>
</span>
</td>
</tr>
<tr id="A4.T8.7.12" class="ltx_tr">
<td id="A4.T8.7.12.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.12.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.12.1.1.1" class="ltx_p" style="width:433.6pt;">(…TRUNCATED)</span>
</span>
</td>
</tr>
<tr id="A4.T8.7.13" class="ltx_tr">
<td id="A4.T8.7.13.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.13.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.13.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T8.7.13.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T8.7.14" class="ltx_tr">
<td id="A4.T8.7.14.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.14.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.14.1.1.1" class="ltx_p" style="width:433.6pt;">{{Problem}}</span>
</span>
</td>
</tr>
<tr id="A4.T8.7.15" class="ltx_tr">
<td id="A4.T8.7.15.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T8.7.15.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.15.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T8.7.15.1.1.1.1" class="ltx_text ltx_font_bold">Initial Solution</span></span>
</span>
</td>
</tr>
<tr id="A4.T8.7.16" class="ltx_tr">
<td id="A4.T8.7.16.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_b ltx_border_l ltx_border_r">
<span id="A4.T8.7.16.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T8.7.16.1.1.1" class="ltx_p" style="width:433.6pt;">{{Initial Solution}}</span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 8: </span>Excerpt of the input prompt for generating critique on decomposition</figcaption>
</figure>
</section>
<section id="A4.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">D.4 </span>Refine</h3>

<div id="A4.SS4.p1" class="ltx_para">
<p id="A4.SS4.p1.1" class="ltx_p">For generating the refined decomposed solution, we adopt the prompt in Table <a href="#A4.T9" title="Table 9 ‣ D.4 Refine ‣ Appendix D Example Prompts ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a>.</p>
</div>
<figure id="A4.T9" class="ltx_table">
<table id="A4.T9.7" class="ltx_tabular ltx_align_middle">
<tr id="A4.T9.7.8" class="ltx_tr">
<td id="A4.T9.7.8.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r ltx_border_t">
<span id="A4.T9.7.8.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.8.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T9.7.8.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T9.3.3" class="ltx_tr">
<td id="A4.T9.3.3.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.3.3.3.3" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.1.1.1.1.1" class="ltx_p" style="width:433.6pt;">Two pirates Polycarpus and Vasily play a very interesting game. They have n chests with coins, the chests are numbered with integers from 1 to n. Chest number i has <math id="A4.T9.1.1.1.1.1.m1.1" class="ltx_Math" alttext="a_{i}" display="inline"><semantics id="A4.T9.1.1.1.1.1.m1.1a"><msub id="A4.T9.1.1.1.1.1.m1.1.1" xref="A4.T9.1.1.1.1.1.m1.1.1.cmml"><mi id="A4.T9.1.1.1.1.1.m1.1.1.2" xref="A4.T9.1.1.1.1.1.m1.1.1.2.cmml">a</mi><mi id="A4.T9.1.1.1.1.1.m1.1.1.3" xref="A4.T9.1.1.1.1.1.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A4.T9.1.1.1.1.1.m1.1b"><apply id="A4.T9.1.1.1.1.1.m1.1.1.cmml" xref="A4.T9.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T9.1.1.1.1.1.m1.1.1.1.cmml" xref="A4.T9.1.1.1.1.1.m1.1.1">subscript</csymbol><ci id="A4.T9.1.1.1.1.1.m1.1.1.2.cmml" xref="A4.T9.1.1.1.1.1.m1.1.1.2">𝑎</ci><ci id="A4.T9.1.1.1.1.1.m1.1.1.3.cmml" xref="A4.T9.1.1.1.1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T9.1.1.1.1.1.m1.1c">a_{i}</annotation></semantics></math> coins.</span>
<span id="A4.T9.3.3.3.3.3" class="ltx_p">Polycarpus and Vasily move in turns. Polycarpus moves first. During a move a player is allowed to choose a positive integer <math id="A4.T9.2.2.2.2.2.m1.1" class="ltx_Math" alttext="x(2·x+1\leq n)" display="inline"><semantics id="A4.T9.2.2.2.2.2.m1.1a"><mrow id="A4.T9.2.2.2.2.2.m1.1.1" xref="A4.T9.2.2.2.2.2.m1.1.1.cmml"><mi id="A4.T9.2.2.2.2.2.m1.1.1.3" xref="A4.T9.2.2.2.2.2.m1.1.1.3.cmml">x</mi><mo lspace="0em" rspace="0em" id="A4.T9.2.2.2.2.2.m1.1.1.2" xref="A4.T9.2.2.2.2.2.m1.1.1.2.cmml">​</mo><mrow id="A4.T9.2.2.2.2.2.m1.1.1.1.1" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.cmml"><mo stretchy="false" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.2" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.cmml">(</mo><mrow id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.cmml"><mrow id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.cmml"><mrow id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.cmml"><mn id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.2" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.1" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml">​</mo><mi mathvariant="normal" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.3" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.1a" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml">​</mo><mi id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.4" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.4.cmml">x</mi></mrow><mo id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.1" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.1.cmml">+</mo><mn id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.3" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.3.cmml">1</mn></mrow><mo id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.1" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.1.cmml">≤</mo><mi id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.3" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.3.cmml">n</mi></mrow><mo stretchy="false" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.3" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T9.2.2.2.2.2.m1.1b"><apply id="A4.T9.2.2.2.2.2.m1.1.1.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1"><times id="A4.T9.2.2.2.2.2.m1.1.1.2.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.2"></times><ci id="A4.T9.2.2.2.2.2.m1.1.1.3.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.3">𝑥</ci><apply id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1"><leq id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.1.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.1"></leq><apply id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2"><plus id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.1.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.1"></plus><apply id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2"><times id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.1"></times><cn type="integer" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.2.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.2">2</cn><ci id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.3.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.3">·</ci><ci id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.4.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.2.4">𝑥</ci></apply><cn type="integer" id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.3.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.2.3">1</cn></apply><ci id="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.3.cmml" xref="A4.T9.2.2.2.2.2.m1.1.1.1.1.1.3">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T9.2.2.2.2.2.m1.1c">x(2·x+1\leq n)</annotation></semantics></math> and take a coin from each chest with numbers <math id="A4.T9.3.3.3.3.3.m2.3" class="ltx_Math" alttext="x,2·x,2·x+1" display="inline"><semantics id="A4.T9.3.3.3.3.3.m2.3a"><mrow id="A4.T9.3.3.3.3.3.m2.3.3.2" xref="A4.T9.3.3.3.3.3.m2.3.3.3.cmml"><mi id="A4.T9.3.3.3.3.3.m2.1.1" xref="A4.T9.3.3.3.3.3.m2.1.1.cmml">x</mi><mo id="A4.T9.3.3.3.3.3.m2.3.3.2.3" xref="A4.T9.3.3.3.3.3.m2.3.3.3.cmml">,</mo><mrow id="A4.T9.3.3.3.3.3.m2.2.2.1.1" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.cmml"><mn id="A4.T9.3.3.3.3.3.m2.2.2.1.1.2" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T9.3.3.3.3.3.m2.2.2.1.1.1" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.1.cmml">​</mo><mi mathvariant="normal" id="A4.T9.3.3.3.3.3.m2.2.2.1.1.3" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T9.3.3.3.3.3.m2.2.2.1.1.1a" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.1.cmml">​</mo><mi id="A4.T9.3.3.3.3.3.m2.2.2.1.1.4" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.4.cmml">x</mi></mrow><mo id="A4.T9.3.3.3.3.3.m2.3.3.2.4" xref="A4.T9.3.3.3.3.3.m2.3.3.3.cmml">,</mo><mrow id="A4.T9.3.3.3.3.3.m2.3.3.2.2" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.cmml"><mrow id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.cmml"><mn id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.2" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.1" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.1.cmml">​</mo><mi mathvariant="normal" id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.3" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.1a" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.1.cmml">​</mo><mi id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.4" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.4.cmml">x</mi></mrow><mo id="A4.T9.3.3.3.3.3.m2.3.3.2.2.1" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.1.cmml">+</mo><mn id="A4.T9.3.3.3.3.3.m2.3.3.2.2.3" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.3.cmml">1</mn></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T9.3.3.3.3.3.m2.3b"><list id="A4.T9.3.3.3.3.3.m2.3.3.3.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2"><ci id="A4.T9.3.3.3.3.3.m2.1.1.cmml" xref="A4.T9.3.3.3.3.3.m2.1.1">𝑥</ci><apply id="A4.T9.3.3.3.3.3.m2.2.2.1.1.cmml" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1"><times id="A4.T9.3.3.3.3.3.m2.2.2.1.1.1.cmml" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.1"></times><cn type="integer" id="A4.T9.3.3.3.3.3.m2.2.2.1.1.2.cmml" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.2">2</cn><ci id="A4.T9.3.3.3.3.3.m2.2.2.1.1.3.cmml" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.3">·</ci><ci id="A4.T9.3.3.3.3.3.m2.2.2.1.1.4.cmml" xref="A4.T9.3.3.3.3.3.m2.2.2.1.1.4">𝑥</ci></apply><apply id="A4.T9.3.3.3.3.3.m2.3.3.2.2.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2"><plus id="A4.T9.3.3.3.3.3.m2.3.3.2.2.1.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.1"></plus><apply id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2"><times id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.1.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.1"></times><cn type="integer" id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.2.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.2">2</cn><ci id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.3.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.3">·</ci><ci id="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.4.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.2.4">𝑥</ci></apply><cn type="integer" id="A4.T9.3.3.3.3.3.m2.3.3.2.2.3.cmml" xref="A4.T9.3.3.3.3.3.m2.3.3.2.2.3">1</cn></apply></list></annotation-xml><annotation encoding="application/x-tex" id="A4.T9.3.3.3.3.3.m2.3c">x,2·x,2·x+1</annotation></semantics></math>. It may turn out that some chest has no coins, in this case the player doesn’t take a coin from this chest. The game finishes when all chests get emptied…[TRUNCATED]</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.9" class="ltx_tr">
<td id="A4.T9.7.9.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.9.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.9.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T9.7.9.1.1.1.1" class="ltx_text ltx_font_bold">Initial Solution</span>

<span id="A4.T9.7.9.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,ZGVmIGlzX3ZhbGlkX2lucHV0KG4sIGEpOgogICAgIiIiQ2hlY2sgaWYgdGhlIGlucHV0IGlzIHZhbGlkIGZvciB0aGUgZ2l2ZW4gY29uc3RyYWludHMuLi4oVFJVTkNBVEVEKSIiIgogICAgcmV0dXJuIG4gJSAyID09IDEKCmRlZiBpbml0aWFsaXplX2NoZXN0cyhhKToKICAgICIiIkluaXRpYWxpemUgdGhlIGxpc3Qgb2YgY2hlc3RzIHdpdGggY29pbnMsIGFkZGluZyBleHRyYSBlbGVtZW50cyBhdCB0aGUgYmVnaW5uaW5nIGFuZCBlbmQuLi4oVFJVTkNBVEVEKSIiIgogICAgcmV0dXJuIFswXSArIGEgKyBbMCwgMF0KCmRlZiBtYWtlX21vdmUoYSwgaSk6CiAgICAiIiJQZXJmb3JtIGEgbW92ZSBpbiB0aGUgZ2FtZSwgdXBkYXRpbmcgdGhlIG51bWJlciBvZiBjb2lucyBpbiB0aGUgY2hlc3RzIGFjY29yZGluZ2x5Li4uKFRSVU5DQVRFRCkiIiIKICAgIG0gPSBtaW4oYVtpXSwgYVtpIC0gMV0pCiAgICBhW2kgLy8gMl0gPSBtYXgoMCwgYVtpIC8vIDJdIC0gbSkKICAgIGFbaV0gLT0gbQogICAgYVtpIC0gMV0gLT0gbQoKZGVmIGNhbGN1bGF0ZV9taW5pbXVtX21vdmVzKG4sIGEpOgogICAgIiIiQ2FsY3VsYXRlIHRoZSBtaW5pbXVtIG51bWJlciBvZiBtb3ZlcyBuZWVkZWQgdG8gZmluaXNoIHRoZSBnYW1lLi4uKFRSVU5DQVRFRCkiIiIKICAgIGlmIG4gPT0gMSBvciBub3QgaXNfdmFsaWRfaW5wdXQobiwgYSk6CiAgICAgICAgcmV0dXJuIC0xCiAgICBhID0gaW5pdGlhbGl6ZV9jaGVzdHMoYSkKICAgIGZvciBpIGluIHJhbmdlKG4sIDAsIC0yKToKICAgICAgICBtYWtlX21vdmUoYSwgaSkKICAgIHJldHVybiBzdW0oYSkKCmRlZiBtYWluKCk6CiAgICBuID0gaW50KGlucHV0KCkpCiAgICBhID0gbGlzdChtYXAoaW50LCBpbnB1dCgpLnNwbGl0KCkpKQogICAgcmVzdWx0ID0gY2FsY3VsYXRlX21pbmltdW1fbW92ZXMobiwgYSkKICAgIHByaW50KHJlc3VsdCkKIy4uLltUUlVOQ0FURURd" download="">⬇</a></span>
<span id="lstnumberx185" class="ltx_listingline"><span id="lstnumberx185.1" class="ltx_text ltx_lst_identifier">def</span><span id="lstnumberx185.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx185.3" class="ltx_text ltx_lst_identifier">is_valid_input</span>(<span id="lstnumberx185.4" class="ltx_text ltx_lst_identifier">n</span>,<span id="lstnumberx185.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx185.6" class="ltx_text ltx_lst_identifier">a</span>):
</span>
<span id="lstnumberx186" class="ltx_listingline"><span id="lstnumberx186.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx186.2" class="ltx_text ltx_lst_string" style="color:#9400D1;">"""Check<span id="lstnumberx186.2.1" class="ltx_text ltx_lst_space"> </span>if<span id="lstnumberx186.2.2" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx186.2.3" class="ltx_text ltx_lst_space"> </span>input<span id="lstnumberx186.2.4" class="ltx_text ltx_lst_space"> </span>is<span id="lstnumberx186.2.5" class="ltx_text ltx_lst_space"> </span>valid<span id="lstnumberx186.2.6" class="ltx_text ltx_lst_space"> </span>for<span id="lstnumberx186.2.7" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx186.2.8" class="ltx_text ltx_lst_space"> </span>given<span id="lstnumberx186.2.9" class="ltx_text ltx_lst_space"> </span>constraints…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx187" class="ltx_listingline"><span id="lstnumberx187.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx187.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx187.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx187.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx187.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx187.6" class="ltx_text" style="color:#9400D1;">%</span><span id="lstnumberx187.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx187.8" class="ltx_text" style="color:#9400D1;">2</span><span id="lstnumberx187.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx187.10" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx187.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx187.12" class="ltx_text" style="color:#9400D1;">1</span>
</span>
<span id="lstnumberx188" class="ltx_listingline">
</span>
<span id="lstnumberx189" class="ltx_listingline"><span id="lstnumberx189.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx189.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx189.3" class="ltx_text" style="color:#9400D1;">initialize_chests(a):</span>
</span>
<span id="lstnumberx190" class="ltx_listingline"><span id="lstnumberx190.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx190.2" class="ltx_text" style="color:#9400D1;">"""Initialize</span><span id="lstnumberx190.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx190.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.6" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx190.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.8" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx190.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.10" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx190.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.12" class="ltx_text" style="color:#9400D1;">with</span><span id="lstnumberx190.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.14" class="ltx_text" style="color:#9400D1;">coins,</span><span id="lstnumberx190.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.16" class="ltx_text" style="color:#9400D1;">adding</span><span id="lstnumberx190.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.18" class="ltx_text" style="color:#9400D1;">extra</span><span id="lstnumberx190.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.20" class="ltx_text" style="color:#9400D1;">elements</span><span id="lstnumberx190.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.22" class="ltx_text" style="color:#9400D1;">at</span><span id="lstnumberx190.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.24" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx190.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.26" class="ltx_text" style="color:#9400D1;">beginning</span><span id="lstnumberx190.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.28" class="ltx_text" style="color:#9400D1;">and</span><span id="lstnumberx190.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx190.30" class="ltx_text" style="color:#9400D1;">end…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx191" class="ltx_listingline"><span id="lstnumberx191.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx191.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx191.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx191.4" class="ltx_text" style="color:#9400D1;">[0]</span><span id="lstnumberx191.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx191.6" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx191.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx191.8" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx191.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx191.10" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx191.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx191.12" class="ltx_text" style="color:#9400D1;">[0,</span><span id="lstnumberx191.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx191.14" class="ltx_text" style="color:#9400D1;">0]</span>
</span>
<span id="lstnumberx192" class="ltx_listingline">
</span>
<span id="lstnumberx193" class="ltx_listingline"><span id="lstnumberx193.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx193.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx193.3" class="ltx_text" style="color:#9400D1;">make_move(a,</span><span id="lstnumberx193.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx193.5" class="ltx_text" style="color:#9400D1;">i):</span>
</span>
<span id="lstnumberx194" class="ltx_listingline"><span id="lstnumberx194.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx194.2" class="ltx_text" style="color:#9400D1;">"""Perform</span><span id="lstnumberx194.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.4" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx194.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.6" class="ltx_text" style="color:#9400D1;">move</span><span id="lstnumberx194.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.8" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx194.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.10" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx194.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.12" class="ltx_text" style="color:#9400D1;">game,</span><span id="lstnumberx194.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.14" class="ltx_text" style="color:#9400D1;">updating</span><span id="lstnumberx194.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.16" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx194.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.18" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx194.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.20" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx194.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.22" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx194.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.24" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx194.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.26" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx194.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.28" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx194.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx194.30" class="ltx_text" style="color:#9400D1;">accordingly…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx195" class="ltx_listingline"><span id="lstnumberx195.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx195.2" class="ltx_text" style="color:#9400D1;">m</span><span id="lstnumberx195.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx195.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx195.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx195.6" class="ltx_text" style="color:#9400D1;">min(a[i],</span><span id="lstnumberx195.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx195.8" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx195.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx195.10" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx195.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx195.12" class="ltx_text" style="color:#9400D1;">1])</span>
</span>
<span id="lstnumberx196" class="ltx_listingline"><span id="lstnumberx196.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx196.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx196.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.4" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx196.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.6" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx196.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.8" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx196.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.10" class="ltx_text" style="color:#9400D1;">max(0,</span><span id="lstnumberx196.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.12" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx196.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.14" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx196.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.16" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx196.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.18" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx196.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx196.20" class="ltx_text" style="color:#9400D1;">m)</span>
</span>
<span id="lstnumberx197" class="ltx_listingline"><span id="lstnumberx197.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx197.2" class="ltx_text" style="color:#9400D1;">a[i]</span><span id="lstnumberx197.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx197.4" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx197.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx197.6" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx198" class="ltx_listingline"><span id="lstnumberx198.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx198.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx198.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx198.4" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx198.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx198.6" class="ltx_text" style="color:#9400D1;">1]</span><span id="lstnumberx198.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx198.8" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx198.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx198.10" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx199" class="ltx_listingline">
</span>
<span id="lstnumberx200" class="ltx_listingline"><span id="lstnumberx200.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx200.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx200.3" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx200.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx200.5" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx201" class="ltx_listingline"><span id="lstnumberx201.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx201.2" class="ltx_text" style="color:#9400D1;">"""Calculate</span><span id="lstnumberx201.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx201.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.6" class="ltx_text" style="color:#9400D1;">minimum</span><span id="lstnumberx201.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx201.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx201.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.12" class="ltx_text" style="color:#9400D1;">moves</span><span id="lstnumberx201.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.14" class="ltx_text" style="color:#9400D1;">needed</span><span id="lstnumberx201.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx201.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.18" class="ltx_text" style="color:#9400D1;">finish</span><span id="lstnumberx201.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx201.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx201.22" class="ltx_text" style="color:#9400D1;">game…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx202" class="ltx_listingline"><span id="lstnumberx202.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx202.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx202.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx202.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.6" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx202.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.8" class="ltx_text" style="color:#9400D1;">1</span><span id="lstnumberx202.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.10" class="ltx_text" style="color:#9400D1;">or</span><span id="lstnumberx202.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.12" class="ltx_text" style="color:#9400D1;">not</span><span id="lstnumberx202.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.14" class="ltx_text" style="color:#9400D1;">is_valid_input(n,</span><span id="lstnumberx202.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx202.16" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx203" class="ltx_listingline"><span id="lstnumberx203.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx203.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx203.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx203.4" class="ltx_text" style="color:#9400D1;">-1</span>
</span>
<span id="lstnumberx204" class="ltx_listingline"><span id="lstnumberx204.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx204.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx204.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx204.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx204.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx204.6" class="ltx_text" style="color:#9400D1;">initialize_chests(a)</span>
</span>
<span id="lstnumberx205" class="ltx_listingline"><span id="lstnumberx205.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx205.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx205.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx205.4" class="ltx_text" style="color:#9400D1;">i</span><span id="lstnumberx205.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx205.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx205.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx205.8" class="ltx_text" style="color:#9400D1;">range(n,</span><span id="lstnumberx205.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx205.10" class="ltx_text" style="color:#9400D1;">0,</span><span id="lstnumberx205.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx205.12" class="ltx_text" style="color:#9400D1;">-2):</span>
</span>
<span id="lstnumberx206" class="ltx_listingline"><span id="lstnumberx206.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx206.2" class="ltx_text" style="color:#9400D1;">make_move(a,</span><span id="lstnumberx206.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx206.4" class="ltx_text" style="color:#9400D1;">i)</span>
</span>
<span id="lstnumberx207" class="ltx_listingline"><span id="lstnumberx207.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx207.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx207.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx207.4" class="ltx_text" style="color:#9400D1;">sum(a)</span>
</span>
<span id="lstnumberx208" class="ltx_listingline">
</span>
<span id="lstnumberx209" class="ltx_listingline"><span id="lstnumberx209.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx209.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx209.3" class="ltx_text" style="color:#9400D1;">main():</span>
</span>
<span id="lstnumberx210" class="ltx_listingline"><span id="lstnumberx210.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx210.2" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx210.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx210.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx210.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx210.6" class="ltx_text" style="color:#9400D1;">int(input())</span>
</span>
<span id="lstnumberx211" class="ltx_listingline"><span id="lstnumberx211.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx211.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx211.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx211.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx211.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx211.6" class="ltx_text" style="color:#9400D1;">list(map(int,</span><span id="lstnumberx211.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx211.8" class="ltx_text" style="color:#9400D1;">input().split()))</span>
</span>
<span id="lstnumberx212" class="ltx_listingline"><span id="lstnumberx212.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx212.2" class="ltx_text" style="color:#9400D1;">result</span><span id="lstnumberx212.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx212.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx212.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx212.6" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx212.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx212.8" class="ltx_text" style="color:#9400D1;">a)</span>
</span>
<span id="lstnumberx213" class="ltx_listingline"><span id="lstnumberx213.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx213.2" class="ltx_text" style="color:#9400D1;">print(result)</span>
</span>
<span id="lstnumberx214" class="ltx_listingline"><span id="lstnumberx214.1" class="ltx_text" style="color:#9400D1;">#…[TRUNCATED]</span>
</span>
</span></span>
<span id="A4.T9.7.9.1.1.2" class="ltx_p"><span id="A4.T9.7.9.1.1.2.1" class="ltx_text ltx_font_bold">Critique</span></span>
</span>
</td>
</tr>
<tr id="A4.T9.5.5" class="ltx_tr">
<td id="A4.T9.5.5.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.5.5.2.2" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.5.5.2.2.2" class="ltx_p" style="width:433.6pt;">1. Decomposition should be around logical breakpoints, but not syntax. However,  <svg id="A4.T9.4.4.1.1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="82.81"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(41.4,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 38.36 9.69 L -38.36 9.69 C -39.89 9.69 -41.13 8.45 -41.13 6.92 L -41.13 -1.4 C -41.13 -2.93 -39.89 -4.17 -38.36 -4.17 L 38.36 -4.17 C 39.89 -4.17 41.13 -2.93 41.13 -1.4 L 41.13 6.92 C 41.13 8.45 39.89 9.69 38.36 9.69 Z M -41.13 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -36.51 0)" fill="#000000" stroke="#000000"><foreignObject width="73.8" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T9.4.4.1.1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">make_move</span></foreignObject></g></g></svg> and  <svg id="A4.T9.5.5.2.2.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="107.56"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(53.78,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 50.74 9.69 L -50.74 9.69 C -52.26 9.69 -53.5 8.45 -53.5 6.92 L -53.5 -1.4 C -53.5 -2.93 -52.26 -4.17 -50.74 -4.17 L 50.74 -4.17 C 52.26 -4.17 53.5 -2.93 53.5 -1.4 L 53.5 6.92 C 53.5 8.45 52.26 9.69 50.74 9.69 Z M -53.5 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -48.89 0)" fill="#000000" stroke="#000000"><foreignObject width="97.78" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T9.5.5.2.2.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">initialize_chests</span></foreignObject></g></g></svg> are not the main logic steps of this problem. We should present a clear high-level logic that aligns with how humans reason about programs conceptually. In this problem, the main logic steps are input reading, checking game finishing conditions, calculating minimum steps, and output printing.</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.7" class="ltx_tr">
<td id="A4.T9.7.7.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.7.2.2" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.7.2.2.2" class="ltx_p" style="width:433.6pt;">2. Boundary conditions (i.e.,  <svg id="A4.T9.6.6.1.1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="172.36"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(86.18,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 83.14 9.69 L -83.14 9.69 C -84.67 9.69 -85.9 8.45 -85.9 6.92 L -85.9 -1.4 C -85.9 -2.93 -84.67 -4.17 -83.14 -4.17 L 83.14 -4.17 C 84.67 -4.17 85.9 -2.93 85.9 -1.4 L 85.9 6.92 C 85.9 8.45 84.67 9.69 83.14 9.69 Z M -85.9 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -81.29 0)" fill="#000000" stroke="#000000"><foreignObject width="162.59" height="11.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T9.6.6.1.1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">if n == 1 and n % 2 == 1</span></foreignObject></g></g></svg> should be explicitly highlighted as individual functions in the main logic, instead of in the function  <svg id="A4.T9.7.7.2.2.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="177.44"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(88.72,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 85.67 9.69 L -85.67 9.69 C -87.2 9.69 -88.44 8.45 -88.44 6.92 L -88.44 -1.4 C -88.44 -2.93 -87.2 -4.17 -85.67 -4.17 L 85.67 -4.17 C 87.2 -4.17 88.44 -2.93 88.44 -1.4 L 88.44 6.92 C 88.44 8.45 87.2 9.69 85.67 9.69 Z M -88.44 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -83.83 0)" fill="#000000" stroke="#000000"><foreignObject width="168.43" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T9.7.7.2.2.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">calculate_minimum_moves</span></foreignObject></g></g></svg>.</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.10" class="ltx_tr">
<td id="A4.T9.7.10.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.10.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.10.1.1.1" class="ltx_p" style="width:433.6pt;">Refine the initial solution based on the above critique. Do not introduce modification unrelated to the critique.</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.11" class="ltx_tr">
<td id="A4.T9.7.11.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.11.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.11.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T9.7.11.1.1.1.1" class="ltx_text ltx_font_bold">Refine</span>

<span id="A4.T9.7.11.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,Li4uKFRSVU5DQVRFRCkKZGVmIGNoZWNrX3ZhbGlkaXR5KG4pOgogICAgIiIiQ2hlY2sgaWYgdGhlIGdhbWUgY2FuIGJlIGZpbmlzaGVkLi4uKFRSVU5DQVRFRCkiIiIKICAgIHJldHVybiBuICE9IDEgYW5kIG4gJSAyICE9IDAKCmRlZiBjYWxjdWxhdGVfbWluaW11bV9tb3ZlcyhuLCBhKToKICAgICIiIkNhbGN1bGF0ZSB0aGUgbWluaW11bSBudW1iZXIgb2YgbW92ZXMgbmVlZGVkIHRvIGZpbmlzaCB0aGUgZ2FtZS4uLihUUlVOQ0FURUQpIiIiCiAgICBhID0gWzBdICsgYSArIFswLCAwXQogICAgZm9yIGkgaW4gcmFuZ2UobiwgMCwgLTIpOgogICAgICAgIG0gPSBtaW4oYVtpXSwgYVtpIC0gMV0pCiAgICAgICAgYVtpIC8vIDJdID0gbWF4KDAsIGFbaSAvLyAyXSAtIG0pCiAgICAgICAgYVtpXSAtPSBtCiAgICAgICAgYVtpIC0gMV0gLT0gbQogICAgcmV0dXJuIHN1bShhKQoKZGVmIG1haW4oKToKICAgIG4sIGEgPSByZWFkX2lucHV0KCkKICAgIGlmIG5vdCBjaGVja192YWxpZGl0eShuKToKICAgICAgICBwcmludCgtMSkKICAgIGVsc2U6CiAgICAgICAgcHJpbnQoY2FsY3VsYXRlX21pbmltdW1fbW92ZXMobiwgYSkpCiMuLi5bVFJVTkNBVEVEXQ==" download="">⬇</a></span>
<span id="lstnumberx215" class="ltx_listingline">…(<span id="lstnumberx215.1" class="ltx_text ltx_lst_identifier">TRUNCATED</span>)
</span>
<span id="lstnumberx216" class="ltx_listingline"><span id="lstnumberx216.1" class="ltx_text ltx_lst_identifier">def</span><span id="lstnumberx216.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx216.3" class="ltx_text ltx_lst_identifier">check_validity</span>(<span id="lstnumberx216.4" class="ltx_text ltx_lst_identifier">n</span>):
</span>
<span id="lstnumberx217" class="ltx_listingline"><span id="lstnumberx217.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx217.2" class="ltx_text ltx_lst_string" style="color:#9400D1;">"""Check<span id="lstnumberx217.2.1" class="ltx_text ltx_lst_space"> </span>if<span id="lstnumberx217.2.2" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx217.2.3" class="ltx_text ltx_lst_space"> </span>game<span id="lstnumberx217.2.4" class="ltx_text ltx_lst_space"> </span>can<span id="lstnumberx217.2.5" class="ltx_text ltx_lst_space"> </span>be<span id="lstnumberx217.2.6" class="ltx_text ltx_lst_space"> </span>finished…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx218" class="ltx_listingline"><span id="lstnumberx218.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx218.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx218.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx218.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.6" class="ltx_text" style="color:#9400D1;">!=</span><span id="lstnumberx218.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.8" class="ltx_text" style="color:#9400D1;">1</span><span id="lstnumberx218.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.10" class="ltx_text" style="color:#9400D1;">and</span><span id="lstnumberx218.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.12" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx218.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.14" class="ltx_text" style="color:#9400D1;">%</span><span id="lstnumberx218.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.16" class="ltx_text" style="color:#9400D1;">2</span><span id="lstnumberx218.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.18" class="ltx_text" style="color:#9400D1;">!=</span><span id="lstnumberx218.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx218.20" class="ltx_text" style="color:#9400D1;">0</span>
</span>
<span id="lstnumberx219" class="ltx_listingline">
</span>
<span id="lstnumberx220" class="ltx_listingline"><span id="lstnumberx220.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx220.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx220.3" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx220.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx220.5" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx221" class="ltx_listingline"><span id="lstnumberx221.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx221.2" class="ltx_text" style="color:#9400D1;">"""Calculate</span><span id="lstnumberx221.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx221.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.6" class="ltx_text" style="color:#9400D1;">minimum</span><span id="lstnumberx221.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx221.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx221.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.12" class="ltx_text" style="color:#9400D1;">moves</span><span id="lstnumberx221.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.14" class="ltx_text" style="color:#9400D1;">needed</span><span id="lstnumberx221.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx221.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.18" class="ltx_text" style="color:#9400D1;">finish</span><span id="lstnumberx221.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx221.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx221.22" class="ltx_text" style="color:#9400D1;">game…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx222" class="ltx_listingline"><span id="lstnumberx222.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx222.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx222.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx222.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.6" class="ltx_text" style="color:#9400D1;">[0]</span><span id="lstnumberx222.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.8" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx222.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.10" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx222.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.12" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx222.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.14" class="ltx_text" style="color:#9400D1;">[0,</span><span id="lstnumberx222.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx222.16" class="ltx_text" style="color:#9400D1;">0]</span>
</span>
<span id="lstnumberx223" class="ltx_listingline"><span id="lstnumberx223.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx223.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx223.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx223.4" class="ltx_text" style="color:#9400D1;">i</span><span id="lstnumberx223.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx223.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx223.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx223.8" class="ltx_text" style="color:#9400D1;">range(n,</span><span id="lstnumberx223.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx223.10" class="ltx_text" style="color:#9400D1;">0,</span><span id="lstnumberx223.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx223.12" class="ltx_text" style="color:#9400D1;">-2):</span>
</span>
<span id="lstnumberx224" class="ltx_listingline"><span id="lstnumberx224.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx224.2" class="ltx_text" style="color:#9400D1;">m</span><span id="lstnumberx224.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx224.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx224.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx224.6" class="ltx_text" style="color:#9400D1;">min(a[i],</span><span id="lstnumberx224.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx224.8" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx224.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx224.10" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx224.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx224.12" class="ltx_text" style="color:#9400D1;">1])</span>
</span>
<span id="lstnumberx225" class="ltx_listingline"><span id="lstnumberx225.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx225.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx225.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.4" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx225.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.6" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx225.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.8" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx225.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.10" class="ltx_text" style="color:#9400D1;">max(0,</span><span id="lstnumberx225.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.12" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx225.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.14" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx225.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.16" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx225.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.18" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx225.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx225.20" class="ltx_text" style="color:#9400D1;">m)</span>
</span>
<span id="lstnumberx226" class="ltx_listingline"><span id="lstnumberx226.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx226.2" class="ltx_text" style="color:#9400D1;">a[i]</span><span id="lstnumberx226.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx226.4" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx226.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx226.6" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx227" class="ltx_listingline"><span id="lstnumberx227.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx227.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx227.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx227.4" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx227.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx227.6" class="ltx_text" style="color:#9400D1;">1]</span><span id="lstnumberx227.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx227.8" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx227.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx227.10" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx228" class="ltx_listingline"><span id="lstnumberx228.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx228.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx228.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx228.4" class="ltx_text" style="color:#9400D1;">sum(a)</span>
</span>
<span id="lstnumberx229" class="ltx_listingline">
</span>
<span id="lstnumberx230" class="ltx_listingline"><span id="lstnumberx230.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx230.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx230.3" class="ltx_text" style="color:#9400D1;">main():</span>
</span>
<span id="lstnumberx231" class="ltx_listingline"><span id="lstnumberx231.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx231.2" class="ltx_text" style="color:#9400D1;">n,</span><span id="lstnumberx231.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx231.4" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx231.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx231.6" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx231.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx231.8" class="ltx_text" style="color:#9400D1;">read_input()</span>
</span>
<span id="lstnumberx232" class="ltx_listingline"><span id="lstnumberx232.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx232.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx232.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx232.4" class="ltx_text" style="color:#9400D1;">not</span><span id="lstnumberx232.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx232.6" class="ltx_text" style="color:#9400D1;">check_validity(n):</span>
</span>
<span id="lstnumberx233" class="ltx_listingline"><span id="lstnumberx233.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx233.2" class="ltx_text" style="color:#9400D1;">print(-1)</span>
</span>
<span id="lstnumberx234" class="ltx_listingline"><span id="lstnumberx234.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx234.2" class="ltx_text" style="color:#9400D1;">else:</span>
</span>
<span id="lstnumberx235" class="ltx_listingline"><span id="lstnumberx235.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx235.2" class="ltx_text" style="color:#9400D1;">print(calculate_minimum_moves(n,</span><span id="lstnumberx235.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx235.4" class="ltx_text" style="color:#9400D1;">a))</span>
</span>
<span id="lstnumberx236" class="ltx_listingline"><span id="lstnumberx236.1" class="ltx_text" style="color:#9400D1;">#…[TRUNCATED]</span>
</span>
</span>
(…TRUNCATED)</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.12" class="ltx_tr">
<td id="A4.T9.7.12.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.12.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.12.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T9.7.12.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T9.7.13" class="ltx_tr">
<td id="A4.T9.7.13.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.13.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.13.1.1.1" class="ltx_p" style="width:433.6pt;">{{Problem}}</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.14" class="ltx_tr">
<td id="A4.T9.7.14.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.14.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.14.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T9.7.14.1.1.1.1" class="ltx_text ltx_font_bold">Initial Solution</span></span>
</span>
</td>
</tr>
<tr id="A4.T9.7.15" class="ltx_tr">
<td id="A4.T9.7.15.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.15.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.15.1.1.1" class="ltx_p" style="width:433.6pt;">{{Initial Solution}}</span>
</span>
</td>
</tr>
<tr id="A4.T9.7.16" class="ltx_tr">
<td id="A4.T9.7.16.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T9.7.16.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.16.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T9.7.16.1.1.1.1" class="ltx_text ltx_font_bold">Critique</span></span>
</span>
</td>
</tr>
<tr id="A4.T9.7.17" class="ltx_tr">
<td id="A4.T9.7.17.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_b ltx_border_l ltx_border_r">
<span id="A4.T9.7.17.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T9.7.17.1.1.1" class="ltx_p" style="width:433.6pt;">{{Critique}}</span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 9: </span>Excerpt of the input prompt for generating refined decomposition</figcaption>
</figure>
</section>
<section id="A4.SS5" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">D.5 </span>Rank</h3>

<div id="A4.SS5.p1" class="ltx_para">
<p id="A4.SS5.p1.1" class="ltx_p">For ranking which decomposition leads to higher human efficiency, we adopt the prompt in Table <a href="#A4.T10" title="Table 10 ‣ D.5 Rank ‣ Appendix D Example Prompts ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>.</p>
</div>
<figure id="A4.T10" class="ltx_table">
<table id="A4.T10.7" class="ltx_tabular ltx_align_middle">
<tr id="A4.T10.7.8" class="ltx_tr">
<td id="A4.T10.7.8.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r ltx_border_t">
<span id="A4.T10.7.8.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.8.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T10.7.8.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T10.3.3" class="ltx_tr">
<td id="A4.T10.3.3.3" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.3.3.3.3" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.1.1.1.1.1" class="ltx_p" style="width:433.6pt;">Two pirates Polycarpus and Vasily play a very interesting game. They have n chests with coins, the chests are numbered with integers from 1 to n. Chest number i has <math id="A4.T10.1.1.1.1.1.m1.1" class="ltx_Math" alttext="a_{i}" display="inline"><semantics id="A4.T10.1.1.1.1.1.m1.1a"><msub id="A4.T10.1.1.1.1.1.m1.1.1" xref="A4.T10.1.1.1.1.1.m1.1.1.cmml"><mi id="A4.T10.1.1.1.1.1.m1.1.1.2" xref="A4.T10.1.1.1.1.1.m1.1.1.2.cmml">a</mi><mi id="A4.T10.1.1.1.1.1.m1.1.1.3" xref="A4.T10.1.1.1.1.1.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A4.T10.1.1.1.1.1.m1.1b"><apply id="A4.T10.1.1.1.1.1.m1.1.1.cmml" xref="A4.T10.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="A4.T10.1.1.1.1.1.m1.1.1.1.cmml" xref="A4.T10.1.1.1.1.1.m1.1.1">subscript</csymbol><ci id="A4.T10.1.1.1.1.1.m1.1.1.2.cmml" xref="A4.T10.1.1.1.1.1.m1.1.1.2">𝑎</ci><ci id="A4.T10.1.1.1.1.1.m1.1.1.3.cmml" xref="A4.T10.1.1.1.1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.1.1.1.1.1.m1.1c">a_{i}</annotation></semantics></math> coins.</span>
<span id="A4.T10.3.3.3.3.3" class="ltx_p">Polycarpus and Vasily move in turns. Polycarpus moves first. During a move a player is allowed to choose a positive integer <math id="A4.T10.2.2.2.2.2.m1.1" class="ltx_Math" alttext="x(2·x+1\leq n)" display="inline"><semantics id="A4.T10.2.2.2.2.2.m1.1a"><mrow id="A4.T10.2.2.2.2.2.m1.1.1" xref="A4.T10.2.2.2.2.2.m1.1.1.cmml"><mi id="A4.T10.2.2.2.2.2.m1.1.1.3" xref="A4.T10.2.2.2.2.2.m1.1.1.3.cmml">x</mi><mo lspace="0em" rspace="0em" id="A4.T10.2.2.2.2.2.m1.1.1.2" xref="A4.T10.2.2.2.2.2.m1.1.1.2.cmml">​</mo><mrow id="A4.T10.2.2.2.2.2.m1.1.1.1.1" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.cmml"><mo stretchy="false" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.2" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.cmml">(</mo><mrow id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.cmml"><mrow id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.cmml"><mrow id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.cmml"><mn id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.2" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.1" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml">​</mo><mi mathvariant="normal" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.3" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.1a" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml">​</mo><mi id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.4" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.4.cmml">x</mi></mrow><mo id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.1" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.1.cmml">+</mo><mn id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.3" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.3.cmml">1</mn></mrow><mo id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.1" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.1.cmml">≤</mo><mi id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.3" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.3.cmml">n</mi></mrow><mo stretchy="false" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.3" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T10.2.2.2.2.2.m1.1b"><apply id="A4.T10.2.2.2.2.2.m1.1.1.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1"><times id="A4.T10.2.2.2.2.2.m1.1.1.2.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.2"></times><ci id="A4.T10.2.2.2.2.2.m1.1.1.3.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.3">𝑥</ci><apply id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1"><leq id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.1.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.1"></leq><apply id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2"><plus id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.1.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.1"></plus><apply id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2"><times id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.1.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.1"></times><cn type="integer" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.2.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.2">2</cn><ci id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.3.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.3">·</ci><ci id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.4.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.2.4">𝑥</ci></apply><cn type="integer" id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.3.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.2.3">1</cn></apply><ci id="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.3.cmml" xref="A4.T10.2.2.2.2.2.m1.1.1.1.1.1.3">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.2.2.2.2.2.m1.1c">x(2·x+1\leq n)</annotation></semantics></math> and take a coin from each chest with numbers <math id="A4.T10.3.3.3.3.3.m2.3" class="ltx_Math" alttext="x,2·x,2·x+1" display="inline"><semantics id="A4.T10.3.3.3.3.3.m2.3a"><mrow id="A4.T10.3.3.3.3.3.m2.3.3.2" xref="A4.T10.3.3.3.3.3.m2.3.3.3.cmml"><mi id="A4.T10.3.3.3.3.3.m2.1.1" xref="A4.T10.3.3.3.3.3.m2.1.1.cmml">x</mi><mo id="A4.T10.3.3.3.3.3.m2.3.3.2.3" xref="A4.T10.3.3.3.3.3.m2.3.3.3.cmml">,</mo><mrow id="A4.T10.3.3.3.3.3.m2.2.2.1.1" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.cmml"><mn id="A4.T10.3.3.3.3.3.m2.2.2.1.1.2" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T10.3.3.3.3.3.m2.2.2.1.1.1" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.1.cmml">​</mo><mi mathvariant="normal" id="A4.T10.3.3.3.3.3.m2.2.2.1.1.3" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T10.3.3.3.3.3.m2.2.2.1.1.1a" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.1.cmml">​</mo><mi id="A4.T10.3.3.3.3.3.m2.2.2.1.1.4" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.4.cmml">x</mi></mrow><mo id="A4.T10.3.3.3.3.3.m2.3.3.2.4" xref="A4.T10.3.3.3.3.3.m2.3.3.3.cmml">,</mo><mrow id="A4.T10.3.3.3.3.3.m2.3.3.2.2" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.cmml"><mrow id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.cmml"><mn id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.2" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.1" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.1.cmml">​</mo><mi mathvariant="normal" id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.3" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.3.cmml">·</mi><mo lspace="0em" rspace="0em" id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.1a" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.1.cmml">​</mo><mi id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.4" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.4.cmml">x</mi></mrow><mo id="A4.T10.3.3.3.3.3.m2.3.3.2.2.1" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.1.cmml">+</mo><mn id="A4.T10.3.3.3.3.3.m2.3.3.2.2.3" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.3.cmml">1</mn></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.T10.3.3.3.3.3.m2.3b"><list id="A4.T10.3.3.3.3.3.m2.3.3.3.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2"><ci id="A4.T10.3.3.3.3.3.m2.1.1.cmml" xref="A4.T10.3.3.3.3.3.m2.1.1">𝑥</ci><apply id="A4.T10.3.3.3.3.3.m2.2.2.1.1.cmml" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1"><times id="A4.T10.3.3.3.3.3.m2.2.2.1.1.1.cmml" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.1"></times><cn type="integer" id="A4.T10.3.3.3.3.3.m2.2.2.1.1.2.cmml" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.2">2</cn><ci id="A4.T10.3.3.3.3.3.m2.2.2.1.1.3.cmml" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.3">·</ci><ci id="A4.T10.3.3.3.3.3.m2.2.2.1.1.4.cmml" xref="A4.T10.3.3.3.3.3.m2.2.2.1.1.4">𝑥</ci></apply><apply id="A4.T10.3.3.3.3.3.m2.3.3.2.2.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2"><plus id="A4.T10.3.3.3.3.3.m2.3.3.2.2.1.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.1"></plus><apply id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2"><times id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.1.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.1"></times><cn type="integer" id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.2.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.2">2</cn><ci id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.3.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.3">·</ci><ci id="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.4.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.2.4">𝑥</ci></apply><cn type="integer" id="A4.T10.3.3.3.3.3.m2.3.3.2.2.3.cmml" xref="A4.T10.3.3.3.3.3.m2.3.3.2.2.3">1</cn></apply></list></annotation-xml><annotation encoding="application/x-tex" id="A4.T10.3.3.3.3.3.m2.3c">x,2·x,2·x+1</annotation></semantics></math>. It may turn out that some chest has no coins, in this case the player doesn’t take a coin from this chest. The game finishes when all chests get emptied…[TRUNCATED]</span>
</span>
</td>
</tr>
<tr id="A4.T10.7.9" class="ltx_tr">
<td id="A4.T10.7.9.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.9.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.9.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T10.7.9.1.1.1.1" class="ltx_text ltx_font_bold">Decomposed Solution A</span>

<span id="A4.T10.7.9.1.1.1.2" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,ZGVmIGlzX3ZhbGlkX2lucHV0KG4sIGEpOgogICAgIiIiQ2hlY2sgaWYgdGhlIGlucHV0IGlzIHZhbGlkIGZvciB0aGUgZ2l2ZW4gY29uc3RyYWludHMuLi4oVFJVTkNBVEVEKSIiIgogICAgcmV0dXJuIG4gJSAyID09IDEKCmRlZiBpbml0aWFsaXplX2NoZXN0cyhhKToKICAgICIiIkluaXRpYWxpemUgdGhlIGxpc3Qgb2YgY2hlc3RzIHdpdGggY29pbnMsIGFkZGluZyBleHRyYSBlbGVtZW50cyBhdCB0aGUgYmVnaW5uaW5nIGFuZCBlbmQuLi4oVFJVTkNBVEVEKSIiIgogICAgcmV0dXJuIFswXSArIGEgKyBbMCwgMF0KCmRlZiBtYWtlX21vdmUoYSwgaSk6CiAgICAiIiJQZXJmb3JtIGEgbW92ZSBpbiB0aGUgZ2FtZSwgdXBkYXRpbmcgdGhlIG51bWJlciBvZiBjb2lucyBpbiB0aGUgY2hlc3RzIGFjY29yZGluZ2x5Li4uKFRSVU5DQVRFRCkiIiIKICAgIG0gPSBtaW4oYVtpXSwgYVtpIC0gMV0pCiAgICBhW2kgLy8gMl0gPSBtYXgoMCwgYVtpIC8vIDJdIC0gbSkKICAgIGFbaV0gLT0gbQogICAgYVtpIC0gMV0gLT0gbQoKZGVmIGNhbGN1bGF0ZV9taW5pbXVtX21vdmVzKG4sIGEpOgogICAgIiIiQ2FsY3VsYXRlIHRoZSBtaW5pbXVtIG51bWJlciBvZiBtb3ZlcyBuZWVkZWQgdG8gZmluaXNoIHRoZSBnYW1lLi4uKFRSVU5DQVRFRCkiIiIKICAgIGlmIG4gPT0gMSBvciBub3QgaXNfdmFsaWRfaW5wdXQobiwgYSk6CiAgICAgICAgcmV0dXJuIC0xCiAgICBhID0gaW5pdGlhbGl6ZV9jaGVzdHMoYSkKICAgIGZvciBpIGluIHJhbmdlKG4sIDAsIC0yKToKICAgICAgICBtYWtlX21vdmUoYSwgaSkKICAgIHJldHVybiBzdW0oYSkKCmRlZiBtYWluKCk6CiAgICBuID0gaW50KGlucHV0KCkpCiAgICBhID0gbGlzdChtYXAoaW50LCBpbnB1dCgpLnNwbGl0KCkpKQogICAgcmVzdWx0ID0gY2FsY3VsYXRlX21pbmltdW1fbW92ZXMobiwgYSkKICAgIHByaW50KHJlc3VsdCkKIy4uLltUUlVOQ0FURURd" download="">⬇</a></span>
<span id="lstnumberx237" class="ltx_listingline"><span id="lstnumberx237.1" class="ltx_text ltx_lst_identifier">def</span><span id="lstnumberx237.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx237.3" class="ltx_text ltx_lst_identifier">is_valid_input</span>(<span id="lstnumberx237.4" class="ltx_text ltx_lst_identifier">n</span>,<span id="lstnumberx237.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx237.6" class="ltx_text ltx_lst_identifier">a</span>):
</span>
<span id="lstnumberx238" class="ltx_listingline"><span id="lstnumberx238.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx238.2" class="ltx_text ltx_lst_string" style="color:#9400D1;">"""Check<span id="lstnumberx238.2.1" class="ltx_text ltx_lst_space"> </span>if<span id="lstnumberx238.2.2" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx238.2.3" class="ltx_text ltx_lst_space"> </span>input<span id="lstnumberx238.2.4" class="ltx_text ltx_lst_space"> </span>is<span id="lstnumberx238.2.5" class="ltx_text ltx_lst_space"> </span>valid<span id="lstnumberx238.2.6" class="ltx_text ltx_lst_space"> </span>for<span id="lstnumberx238.2.7" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx238.2.8" class="ltx_text ltx_lst_space"> </span>given<span id="lstnumberx238.2.9" class="ltx_text ltx_lst_space"> </span>constraints…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx239" class="ltx_listingline"><span id="lstnumberx239.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx239.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx239.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx239.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx239.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx239.6" class="ltx_text" style="color:#9400D1;">%</span><span id="lstnumberx239.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx239.8" class="ltx_text" style="color:#9400D1;">2</span><span id="lstnumberx239.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx239.10" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx239.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx239.12" class="ltx_text" style="color:#9400D1;">1</span>
</span>
<span id="lstnumberx240" class="ltx_listingline">
</span>
<span id="lstnumberx241" class="ltx_listingline"><span id="lstnumberx241.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx241.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx241.3" class="ltx_text" style="color:#9400D1;">initialize_chests(a):</span>
</span>
<span id="lstnumberx242" class="ltx_listingline"><span id="lstnumberx242.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx242.2" class="ltx_text" style="color:#9400D1;">"""Initialize</span><span id="lstnumberx242.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx242.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.6" class="ltx_text" style="color:#9400D1;">list</span><span id="lstnumberx242.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.8" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx242.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.10" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx242.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.12" class="ltx_text" style="color:#9400D1;">with</span><span id="lstnumberx242.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.14" class="ltx_text" style="color:#9400D1;">coins,</span><span id="lstnumberx242.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.16" class="ltx_text" style="color:#9400D1;">adding</span><span id="lstnumberx242.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.18" class="ltx_text" style="color:#9400D1;">extra</span><span id="lstnumberx242.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.20" class="ltx_text" style="color:#9400D1;">elements</span><span id="lstnumberx242.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.22" class="ltx_text" style="color:#9400D1;">at</span><span id="lstnumberx242.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.24" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx242.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.26" class="ltx_text" style="color:#9400D1;">beginning</span><span id="lstnumberx242.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.28" class="ltx_text" style="color:#9400D1;">and</span><span id="lstnumberx242.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx242.30" class="ltx_text" style="color:#9400D1;">end…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx243" class="ltx_listingline"><span id="lstnumberx243.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx243.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx243.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx243.4" class="ltx_text" style="color:#9400D1;">[0]</span><span id="lstnumberx243.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx243.6" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx243.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx243.8" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx243.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx243.10" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx243.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx243.12" class="ltx_text" style="color:#9400D1;">[0,</span><span id="lstnumberx243.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx243.14" class="ltx_text" style="color:#9400D1;">0]</span>
</span>
<span id="lstnumberx244" class="ltx_listingline">
</span>
<span id="lstnumberx245" class="ltx_listingline"><span id="lstnumberx245.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx245.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx245.3" class="ltx_text" style="color:#9400D1;">make_move(a,</span><span id="lstnumberx245.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx245.5" class="ltx_text" style="color:#9400D1;">i):</span>
</span>
<span id="lstnumberx246" class="ltx_listingline"><span id="lstnumberx246.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx246.2" class="ltx_text" style="color:#9400D1;">"""Perform</span><span id="lstnumberx246.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.4" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx246.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.6" class="ltx_text" style="color:#9400D1;">move</span><span id="lstnumberx246.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.8" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx246.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.10" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx246.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.12" class="ltx_text" style="color:#9400D1;">game,</span><span id="lstnumberx246.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.14" class="ltx_text" style="color:#9400D1;">updating</span><span id="lstnumberx246.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.16" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx246.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.18" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx246.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.20" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx246.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.22" class="ltx_text" style="color:#9400D1;">coins</span><span id="lstnumberx246.23" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.24" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx246.25" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.26" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx246.27" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.28" class="ltx_text" style="color:#9400D1;">chests</span><span id="lstnumberx246.29" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx246.30" class="ltx_text" style="color:#9400D1;">accordingly…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx247" class="ltx_listingline"><span id="lstnumberx247.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx247.2" class="ltx_text" style="color:#9400D1;">m</span><span id="lstnumberx247.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx247.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx247.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx247.6" class="ltx_text" style="color:#9400D1;">min(a[i],</span><span id="lstnumberx247.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx247.8" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx247.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx247.10" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx247.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx247.12" class="ltx_text" style="color:#9400D1;">1])</span>
</span>
<span id="lstnumberx248" class="ltx_listingline"><span id="lstnumberx248.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx248.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx248.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.4" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx248.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.6" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx248.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.8" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx248.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.10" class="ltx_text" style="color:#9400D1;">max(0,</span><span id="lstnumberx248.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.12" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx248.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.14" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx248.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.16" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx248.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.18" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx248.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx248.20" class="ltx_text" style="color:#9400D1;">m)</span>
</span>
<span id="lstnumberx249" class="ltx_listingline"><span id="lstnumberx249.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx249.2" class="ltx_text" style="color:#9400D1;">a[i]</span><span id="lstnumberx249.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx249.4" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx249.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx249.6" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx250" class="ltx_listingline"><span id="lstnumberx250.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx250.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx250.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx250.4" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx250.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx250.6" class="ltx_text" style="color:#9400D1;">1]</span><span id="lstnumberx250.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx250.8" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx250.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx250.10" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx251" class="ltx_listingline">
</span>
<span id="lstnumberx252" class="ltx_listingline"><span id="lstnumberx252.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx252.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx252.3" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx252.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx252.5" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx253" class="ltx_listingline"><span id="lstnumberx253.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx253.2" class="ltx_text" style="color:#9400D1;">"""Calculate</span><span id="lstnumberx253.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx253.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.6" class="ltx_text" style="color:#9400D1;">minimum</span><span id="lstnumberx253.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx253.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx253.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.12" class="ltx_text" style="color:#9400D1;">moves</span><span id="lstnumberx253.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.14" class="ltx_text" style="color:#9400D1;">needed</span><span id="lstnumberx253.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx253.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.18" class="ltx_text" style="color:#9400D1;">finish</span><span id="lstnumberx253.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx253.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx253.22" class="ltx_text" style="color:#9400D1;">game…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx254" class="ltx_listingline"><span id="lstnumberx254.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx254.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx254.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx254.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.6" class="ltx_text" style="color:#9400D1;">==</span><span id="lstnumberx254.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.8" class="ltx_text" style="color:#9400D1;">1</span><span id="lstnumberx254.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.10" class="ltx_text" style="color:#9400D1;">or</span><span id="lstnumberx254.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.12" class="ltx_text" style="color:#9400D1;">not</span><span id="lstnumberx254.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.14" class="ltx_text" style="color:#9400D1;">is_valid_input(n,</span><span id="lstnumberx254.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx254.16" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx255" class="ltx_listingline"><span id="lstnumberx255.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx255.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx255.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx255.4" class="ltx_text" style="color:#9400D1;">-1</span>
</span>
<span id="lstnumberx256" class="ltx_listingline"><span id="lstnumberx256.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx256.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx256.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx256.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx256.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx256.6" class="ltx_text" style="color:#9400D1;">initialize_chests(a)</span>
</span>
<span id="lstnumberx257" class="ltx_listingline"><span id="lstnumberx257.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx257.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx257.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx257.4" class="ltx_text" style="color:#9400D1;">i</span><span id="lstnumberx257.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx257.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx257.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx257.8" class="ltx_text" style="color:#9400D1;">range(n,</span><span id="lstnumberx257.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx257.10" class="ltx_text" style="color:#9400D1;">0,</span><span id="lstnumberx257.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx257.12" class="ltx_text" style="color:#9400D1;">-2):</span>
</span>
<span id="lstnumberx258" class="ltx_listingline"><span id="lstnumberx258.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx258.2" class="ltx_text" style="color:#9400D1;">make_move(a,</span><span id="lstnumberx258.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx258.4" class="ltx_text" style="color:#9400D1;">i)</span>
</span>
<span id="lstnumberx259" class="ltx_listingline"><span id="lstnumberx259.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx259.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx259.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx259.4" class="ltx_text" style="color:#9400D1;">sum(a)</span>
</span>
<span id="lstnumberx260" class="ltx_listingline">
</span>
<span id="lstnumberx261" class="ltx_listingline"><span id="lstnumberx261.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx261.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx261.3" class="ltx_text" style="color:#9400D1;">main():</span>
</span>
<span id="lstnumberx262" class="ltx_listingline"><span id="lstnumberx262.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx262.2" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx262.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx262.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx262.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx262.6" class="ltx_text" style="color:#9400D1;">int(input())</span>
</span>
<span id="lstnumberx263" class="ltx_listingline"><span id="lstnumberx263.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx263.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx263.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx263.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx263.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx263.6" class="ltx_text" style="color:#9400D1;">list(map(int,</span><span id="lstnumberx263.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx263.8" class="ltx_text" style="color:#9400D1;">input().split()))</span>
</span>
<span id="lstnumberx264" class="ltx_listingline"><span id="lstnumberx264.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx264.2" class="ltx_text" style="color:#9400D1;">result</span><span id="lstnumberx264.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx264.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx264.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx264.6" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx264.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx264.8" class="ltx_text" style="color:#9400D1;">a)</span>
</span>
<span id="lstnumberx265" class="ltx_listingline"><span id="lstnumberx265.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx265.2" class="ltx_text" style="color:#9400D1;">print(result)</span>
</span>
<span id="lstnumberx266" class="ltx_listingline"><span id="lstnumberx266.1" class="ltx_text" style="color:#9400D1;">#…[TRUNCATED]</span>
</span>
</span></span>
<span id="A4.T10.7.9.1.1.2" class="ltx_p"><span id="A4.T10.7.9.1.1.2.1" class="ltx_text ltx_font_bold">Decomposed Solution B</span></span>
<span id="A4.T10.7.9.1.1.3" class="ltx_listing ltx_lst_language_Octave ltx_lstlisting ltx_listing" style="background-color:#F2F2EB;"><span class="ltx_listing_data"><a href="data:text/plain;base64,Li4uKFRSVU5DQVRFRCkKZGVmIGNoZWNrX3ZhbGlkaXR5KG4pOgogICAgIiIiQ2hlY2sgaWYgdGhlIGdhbWUgY2FuIGJlIGZpbmlzaGVkLi4uKFRSVU5DQVRFRCkiIiIKICAgIHJldHVybiBuICE9IDEgYW5kIG4gJSAyICE9IDAKCmRlZiBjYWxjdWxhdGVfbWluaW11bV9tb3ZlcyhuLCBhKToKICAgICIiIkNhbGN1bGF0ZSB0aGUgbWluaW11bSBudW1iZXIgb2YgbW92ZXMgbmVlZGVkIHRvIGZpbmlzaCB0aGUgZ2FtZS4uLihUUlVOQ0FURUQpIiIiCiAgICBhID0gWzBdICsgYSArIFswLCAwXQogICAgZm9yIGkgaW4gcmFuZ2UobiwgMCwgLTIpOgogICAgICAgIG0gPSBtaW4oYVtpXSwgYVtpIC0gMV0pCiAgICAgICAgYVtpIC8vIDJdID0gbWF4KDAsIGFbaSAvLyAyXSAtIG0pCiAgICAgICAgYVtpXSAtPSBtCiAgICAgICAgYVtpIC0gMV0gLT0gbQogICAgcmV0dXJuIHN1bShhKQoKZGVmIG1haW4oKToKICAgIG4sIGEgPSByZWFkX2lucHV0KCkKICAgIGlmIG5vdCBjaGVja192YWxpZGl0eShuKToKICAgICAgICBwcmludCgtMSkKICAgIGVsc2U6CiAgICAgICAgcHJpbnQoY2FsY3VsYXRlX21pbmltdW1fbW92ZXMobiwgYSkpCiMuLi5bVFJVTkNBVEVEXQ==" download="">⬇</a></span>
<span id="lstnumberx267" class="ltx_listingline">…(<span id="lstnumberx267.1" class="ltx_text ltx_lst_identifier">TRUNCATED</span>)
</span>
<span id="lstnumberx268" class="ltx_listingline"><span id="lstnumberx268.1" class="ltx_text ltx_lst_identifier">def</span><span id="lstnumberx268.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx268.3" class="ltx_text ltx_lst_identifier">check_validity</span>(<span id="lstnumberx268.4" class="ltx_text ltx_lst_identifier">n</span>):
</span>
<span id="lstnumberx269" class="ltx_listingline"><span id="lstnumberx269.1" class="ltx_text ltx_lst_space">    </span><span id="lstnumberx269.2" class="ltx_text ltx_lst_string" style="color:#9400D1;">"""Check<span id="lstnumberx269.2.1" class="ltx_text ltx_lst_space"> </span>if<span id="lstnumberx269.2.2" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx269.2.3" class="ltx_text ltx_lst_space"> </span>game<span id="lstnumberx269.2.4" class="ltx_text ltx_lst_space"> </span>can<span id="lstnumberx269.2.5" class="ltx_text ltx_lst_space"> </span>be<span id="lstnumberx269.2.6" class="ltx_text ltx_lst_space"> </span>finished…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx270" class="ltx_listingline"><span id="lstnumberx270.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx270.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx270.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.4" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx270.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.6" class="ltx_text" style="color:#9400D1;">!=</span><span id="lstnumberx270.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.8" class="ltx_text" style="color:#9400D1;">1</span><span id="lstnumberx270.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.10" class="ltx_text" style="color:#9400D1;">and</span><span id="lstnumberx270.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.12" class="ltx_text" style="color:#9400D1;">n</span><span id="lstnumberx270.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.14" class="ltx_text" style="color:#9400D1;">%</span><span id="lstnumberx270.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.16" class="ltx_text" style="color:#9400D1;">2</span><span id="lstnumberx270.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.18" class="ltx_text" style="color:#9400D1;">!=</span><span id="lstnumberx270.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx270.20" class="ltx_text" style="color:#9400D1;">0</span>
</span>
<span id="lstnumberx271" class="ltx_listingline">
</span>
<span id="lstnumberx272" class="ltx_listingline"><span id="lstnumberx272.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx272.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx272.3" class="ltx_text" style="color:#9400D1;">calculate_minimum_moves(n,</span><span id="lstnumberx272.4" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx272.5" class="ltx_text" style="color:#9400D1;">a):</span>
</span>
<span id="lstnumberx273" class="ltx_listingline"><span id="lstnumberx273.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx273.2" class="ltx_text" style="color:#9400D1;">"""Calculate</span><span id="lstnumberx273.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.4" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx273.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.6" class="ltx_text" style="color:#9400D1;">minimum</span><span id="lstnumberx273.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.8" class="ltx_text" style="color:#9400D1;">number</span><span id="lstnumberx273.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.10" class="ltx_text" style="color:#9400D1;">of</span><span id="lstnumberx273.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.12" class="ltx_text" style="color:#9400D1;">moves</span><span id="lstnumberx273.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.14" class="ltx_text" style="color:#9400D1;">needed</span><span id="lstnumberx273.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.16" class="ltx_text" style="color:#9400D1;">to</span><span id="lstnumberx273.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.18" class="ltx_text" style="color:#9400D1;">finish</span><span id="lstnumberx273.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.20" class="ltx_text" style="color:#9400D1;">the</span><span id="lstnumberx273.21" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx273.22" class="ltx_text" style="color:#9400D1;">game…(TRUNCATED)"""</span>
</span>
<span id="lstnumberx274" class="ltx_listingline"><span id="lstnumberx274.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx274.2" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx274.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx274.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.6" class="ltx_text" style="color:#9400D1;">[0]</span><span id="lstnumberx274.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.8" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx274.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.10" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx274.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.12" class="ltx_text" style="color:#9400D1;">+</span><span id="lstnumberx274.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.14" class="ltx_text" style="color:#9400D1;">[0,</span><span id="lstnumberx274.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx274.16" class="ltx_text" style="color:#9400D1;">0]</span>
</span>
<span id="lstnumberx275" class="ltx_listingline"><span id="lstnumberx275.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx275.2" class="ltx_text" style="color:#9400D1;">for</span><span id="lstnumberx275.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx275.4" class="ltx_text" style="color:#9400D1;">i</span><span id="lstnumberx275.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx275.6" class="ltx_text" style="color:#9400D1;">in</span><span id="lstnumberx275.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx275.8" class="ltx_text" style="color:#9400D1;">range(n,</span><span id="lstnumberx275.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx275.10" class="ltx_text" style="color:#9400D1;">0,</span><span id="lstnumberx275.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx275.12" class="ltx_text" style="color:#9400D1;">-2):</span>
</span>
<span id="lstnumberx276" class="ltx_listingline"><span id="lstnumberx276.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx276.2" class="ltx_text" style="color:#9400D1;">m</span><span id="lstnumberx276.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx276.4" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx276.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx276.6" class="ltx_text" style="color:#9400D1;">min(a[i],</span><span id="lstnumberx276.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx276.8" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx276.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx276.10" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx276.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx276.12" class="ltx_text" style="color:#9400D1;">1])</span>
</span>
<span id="lstnumberx277" class="ltx_listingline"><span id="lstnumberx277.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx277.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx277.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.4" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx277.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.6" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx277.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.8" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx277.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.10" class="ltx_text" style="color:#9400D1;">max(0,</span><span id="lstnumberx277.11" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.12" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx277.13" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.14" class="ltx_text" style="color:#9400D1;">//</span><span id="lstnumberx277.15" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.16" class="ltx_text" style="color:#9400D1;">2]</span><span id="lstnumberx277.17" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.18" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx277.19" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx277.20" class="ltx_text" style="color:#9400D1;">m)</span>
</span>
<span id="lstnumberx278" class="ltx_listingline"><span id="lstnumberx278.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx278.2" class="ltx_text" style="color:#9400D1;">a[i]</span><span id="lstnumberx278.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx278.4" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx278.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx278.6" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx279" class="ltx_listingline"><span id="lstnumberx279.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx279.2" class="ltx_text" style="color:#9400D1;">a[i</span><span id="lstnumberx279.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx279.4" class="ltx_text" style="color:#9400D1;">-</span><span id="lstnumberx279.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx279.6" class="ltx_text" style="color:#9400D1;">1]</span><span id="lstnumberx279.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx279.8" class="ltx_text" style="color:#9400D1;">-=</span><span id="lstnumberx279.9" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx279.10" class="ltx_text" style="color:#9400D1;">m</span>
</span>
<span id="lstnumberx280" class="ltx_listingline"><span id="lstnumberx280.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx280.2" class="ltx_text" style="color:#9400D1;">return</span><span id="lstnumberx280.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx280.4" class="ltx_text" style="color:#9400D1;">sum(a)</span>
</span>
<span id="lstnumberx281" class="ltx_listingline">
</span>
<span id="lstnumberx282" class="ltx_listingline"><span id="lstnumberx282.1" class="ltx_text" style="color:#9400D1;">def</span><span id="lstnumberx282.2" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx282.3" class="ltx_text" style="color:#9400D1;">main():</span>
</span>
<span id="lstnumberx283" class="ltx_listingline"><span id="lstnumberx283.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx283.2" class="ltx_text" style="color:#9400D1;">n,</span><span id="lstnumberx283.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx283.4" class="ltx_text" style="color:#9400D1;">a</span><span id="lstnumberx283.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx283.6" class="ltx_text" style="color:#9400D1;">=</span><span id="lstnumberx283.7" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx283.8" class="ltx_text" style="color:#9400D1;">read_input()</span>
</span>
<span id="lstnumberx284" class="ltx_listingline"><span id="lstnumberx284.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx284.2" class="ltx_text" style="color:#9400D1;">if</span><span id="lstnumberx284.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx284.4" class="ltx_text" style="color:#9400D1;">not</span><span id="lstnumberx284.5" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx284.6" class="ltx_text" style="color:#9400D1;">check_validity(n):</span>
</span>
<span id="lstnumberx285" class="ltx_listingline"><span id="lstnumberx285.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx285.2" class="ltx_text" style="color:#9400D1;">print(-1)</span>
</span>
<span id="lstnumberx286" class="ltx_listingline"><span id="lstnumberx286.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">    </span><span id="lstnumberx286.2" class="ltx_text" style="color:#9400D1;">else:</span>
</span>
<span id="lstnumberx287" class="ltx_listingline"><span id="lstnumberx287.1" class="ltx_text ltx_lst_space" style="color:#9400D1;">        </span><span id="lstnumberx287.2" class="ltx_text" style="color:#9400D1;">print(calculate_minimum_moves(n,</span><span id="lstnumberx287.3" class="ltx_text ltx_lst_space" style="color:#9400D1;"> </span><span id="lstnumberx287.4" class="ltx_text" style="color:#9400D1;">a))</span>
</span>
<span id="lstnumberx288" class="ltx_listingline"><span id="lstnumberx288.1" class="ltx_text" style="color:#9400D1;">#…[TRUNCATED]</span>
</span>
</span>
<span id="A4.T10.7.9.1.1.4" class="ltx_p"><span id="A4.T10.7.9.1.1.4.1" class="ltx_text ltx_font_bold">Evaluation Request</span>
Given two decomposed solutions A and B, determine which one of them is more effective in assisting humans to debug the solution.</span>
<span id="A4.T10.7.9.1.1.5" class="ltx_p"><span id="A4.T10.7.9.1.1.5.1" class="ltx_text ltx_font_bold">Evaluation Results</span>
Decomposed Solution B is better.</span>
</span>
</td>
</tr>
<tr id="A4.T10.5.5" class="ltx_tr">
<td id="A4.T10.5.5.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.5.5.2.2" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.5.5.2.2.2" class="ltx_p" style="width:433.6pt;">1. Decomposition should be around logical breakpoints, but not syntax. However,  <svg id="A4.T10.4.4.1.1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="82.81"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(41.4,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 38.36 9.69 L -38.36 9.69 C -39.89 9.69 -41.13 8.45 -41.13 6.92 L -41.13 -1.4 C -41.13 -2.93 -39.89 -4.17 -38.36 -4.17 L 38.36 -4.17 C 39.89 -4.17 41.13 -2.93 41.13 -1.4 L 41.13 6.92 C 41.13 8.45 39.89 9.69 38.36 9.69 Z M -41.13 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -36.51 0)" fill="#000000" stroke="#000000"><foreignObject width="73.8" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T10.4.4.1.1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">make_move</span></foreignObject></g></g></svg> and  <svg id="A4.T10.5.5.2.2.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="107.56"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(53.78,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 50.74 9.69 L -50.74 9.69 C -52.26 9.69 -53.5 8.45 -53.5 6.92 L -53.5 -1.4 C -53.5 -2.93 -52.26 -4.17 -50.74 -4.17 L 50.74 -4.17 C 52.26 -4.17 53.5 -2.93 53.5 -1.4 L 53.5 6.92 C 53.5 8.45 52.26 9.69 50.74 9.69 Z M -53.5 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -48.89 0)" fill="#000000" stroke="#000000"><foreignObject width="97.78" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T10.5.5.2.2.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">initialize_chests</span></foreignObject></g></g></svg> in Decomposed Solution A are not the main logic steps of this problem. We should present a clear high-level logic that aligns with how humans reason about programs conceptually. In this problem, the main logic steps are input reading, checking game finishing conditions, calculating minimum steps, and output printing, as shown in Decomposed Solution B.</span>
</span>
</td>
</tr>
<tr id="A4.T10.7.7" class="ltx_tr">
<td id="A4.T10.7.7.2" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.7.2.2" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.7.2.2.2" class="ltx_p" style="width:433.6pt;">2. Boundary conditions (i.e.,  <svg id="A4.T10.6.6.1.1.1.pic1" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="172.36"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(86.18,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 83.14 9.69 L -83.14 9.69 C -84.67 9.69 -85.9 8.45 -85.9 6.92 L -85.9 -1.4 C -85.9 -2.93 -84.67 -4.17 -83.14 -4.17 L 83.14 -4.17 C 84.67 -4.17 85.9 -2.93 85.9 -1.4 L 85.9 6.92 C 85.9 8.45 84.67 9.69 83.14 9.69 Z M -85.9 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -81.29 0)" fill="#000000" stroke="#000000"><foreignObject width="162.59" height="11.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T10.6.6.1.1.1.pic1.1.1.1.1.1" class="ltx_text ltx_font_typewriter">if n == 1 and n % 2 == 1</span></foreignObject></g></g></svg>) should be explicitly highlighted as individual functions in the main logic, while Decomposed Solution A places it in the  <svg id="A4.T10.7.7.2.2.2.pic2" class="ltx_picture" height="14.41" overflow="visible" version="1.1" width="177.44"><g transform="translate(0,14.41) matrix(1 0 0 -1 0 0) translate(88.72,0) translate(0,4.45)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g stroke="#DFDFDF" fill="#DFDFDF"><path d="M 85.67 9.69 L -85.67 9.69 C -87.2 9.69 -88.44 8.45 -88.44 6.92 L -88.44 -1.4 C -88.44 -2.93 -87.2 -4.17 -85.67 -4.17 L 85.67 -4.17 C 87.2 -4.17 88.44 -2.93 88.44 -1.4 L 88.44 6.92 C 88.44 8.45 87.2 9.69 85.67 9.69 Z M -88.44 -4.17"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -83.83 0)" fill="#000000" stroke="#000000"><foreignObject width="168.43" height="12.45" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="A4.T10.7.7.2.2.2.pic2.1.1.1.1.1" class="ltx_text ltx_font_typewriter">calculate_minimum_moves</span></foreignObject></g></g></svg> function.</span>
</span>
</td>
</tr>
<tr id="A4.T10.7.10" class="ltx_tr">
<td id="A4.T10.7.10.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.10.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.10.1.1.1" class="ltx_p" style="width:433.6pt;">(…TRUNCATED)</span>
</span>
</td>
</tr>
<tr id="A4.T10.7.11" class="ltx_tr">
<td id="A4.T10.7.11.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.11.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.11.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T10.7.11.1.1.1.1" class="ltx_text ltx_font_bold">Problem</span></span>
</span>
</td>
</tr>
<tr id="A4.T10.7.12" class="ltx_tr">
<td id="A4.T10.7.12.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.12.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.12.1.1.1" class="ltx_p" style="width:433.6pt;">{{Problem}}</span>
</span>
</td>
</tr>
<tr id="A4.T10.7.13" class="ltx_tr">
<td id="A4.T10.7.13.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.13.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.13.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T10.7.13.1.1.1.1" class="ltx_text ltx_font_bold">Decomposed Solution A</span></span>
</span>
</td>
</tr>
<tr id="A4.T10.7.14" class="ltx_tr">
<td id="A4.T10.7.14.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.14.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.14.1.1.1" class="ltx_p" style="width:433.6pt;">{{Decomposed Solution A}}</span>
</span>
</td>
</tr>
<tr id="A4.T10.7.15" class="ltx_tr">
<td id="A4.T10.7.15.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_l ltx_border_r">
<span id="A4.T10.7.15.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.15.1.1.1" class="ltx_p" style="width:433.6pt;"><span id="A4.T10.7.15.1.1.1.1" class="ltx_text ltx_font_bold">Decomposed Solution B</span></span>
</span>
</td>
</tr>
<tr id="A4.T10.7.16" class="ltx_tr">
<td id="A4.T10.7.16.1" class="ltx_td ltx_align_justify ltx_align_top ltx_border_b ltx_border_l ltx_border_r">
<span id="A4.T10.7.16.1.1" class="ltx_inline-block ltx_align_top">
<span id="A4.T10.7.16.1.1.1" class="ltx_p" style="width:433.6pt;">{{Decomposed Solution B}}</span>
</span>
</td>
</tr>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 10: </span>Excerpt of the input prompt for ranking different decompositions.</figcaption>
</figure>
</section>
</section>
<section id="A5" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix E </span>Case Study</h2>

<div id="A5.p1" class="ltx_para">
<p id="A5.p1.1" class="ltx_p">We present several cases to illustrate how decomposition assists humans in practice by highlighting boundary conditions (Figure <a href="#A5.F13" title="Figure 13 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">13</span></a>, Figure <a href="#A5.F14" title="Figure 14 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">14</span></a>), creating simpler and more managable subtasks (Figure <a href="#A5.F15" title="Figure 15 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">15</span></a>, Figure <a href="#A5.F16" title="Figure 16 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">16</span></a>), and presenting clear high-level logic (Figure <a href="#A5.F17" title="Figure 17 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">17</span></a>, Figure <a href="#A5.F18" title="Figure 18 ‣ Appendix E Case Study ‣ Learning Task Decomposition to Assist Humans in Competitive Programming" class="ltx_ref"><span class="ltx_text ltx_ref_tag">18</span></a>).</p>
</div>
<figure id="A5.F13" class="ltx_figure"><img src="/html/2406.04604/assets/x5.png" id="A5.F13.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="369" height="191" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 13: </span>Excerpt of the decomposition that highlights boundary conditions in the training data.</figcaption>
</figure>
<figure id="A5.F14" class="ltx_figure"><img src="/html/2406.04604/assets/x6.png" id="A5.F14.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="415" height="211" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 14: </span>Excerpt of the decomposition that highlights boundary conditions in the test data.</figcaption>
</figure>
<figure id="A5.F15" class="ltx_figure"><img src="/html/2406.04604/assets/x7.png" id="A5.F15.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="293" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 15: </span>Excerpt of the decomposition that creates simpler subtasks in the training data.</figcaption>
</figure>
<figure id="A5.F16" class="ltx_figure"><img src="/html/2406.04604/assets/x8.png" id="A5.F16.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="415" height="241" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 16: </span>Excerpt of the decomposition that creates simpler subtasks in the test data</figcaption>
</figure>
<figure id="A5.F17" class="ltx_figure"><img src="/html/2406.04604/assets/x9.png" id="A5.F17.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="329" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 17: </span>Excerpt of the decomposition that presents clear high-level logic in the training data.</figcaption>
</figure>
<figure id="A5.F18" class="ltx_figure"><img src="/html/2406.04604/assets/x10.png" id="A5.F18.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="246" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 18: </span>Excerpt of the decomposition that presents clear high-level logic in the test data.</figcaption>
</figure>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2406.04603" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2406.04604" class="ar5iv-text-button ar5iv-severity-error">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2406.04604">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2406.04604" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2406.04605" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Fri Jul  5 23:06:16 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2406.04604
When using language models (LMs)  to solve complex problems, humans might struggle to understand the LM-generated solutions and repair the flawed ones.
To assist humans in repairing them, we propose to automatically decompose complex solutions into multiple simpler pieces that correspond to specific subtasks. We introduce a novel objective for learning task decomposition, termedassistive value(AssistV) , which measures the feasibility and speed for humans to repair the decomposed solution. We collect a dataset of human repair experiences on different decomposed solutions. Utilizing the collected data as in-context examples, we then learn to critique, refine, and rank decomposed solutions to improveAssistV. We validate our method under competitive programming problems: under 177 hours of human study, our method enables non-experts to solve 33.3% more problems, speeds them up by 3.3x, and empowers them to match unassisted experts.
abstract
Learning Task Decomposition to Assist Humansin Competitive Programming
Abstract
2406.04604
1Introduction††footnotetext:†Corresponding authorWith their increased capabilities, language models (LMs)  are used to perform increasingly complex and high-impact problems.
The scalable oversight challenge emerges:
LMs might fail to provide reliable solutions for these problems, but it is also difficult for humans to evaluate and improve LMs’ solutions due to therequired significant time and expertise.Figure 1:Decompositions can assist humans in supervising models to solve complex problems.Left: To solve a problem, an LM would first propose an initial solution; our goal is to decompose the initial solution into multiple simpler pieces such that humans can repair it more easily. (Sub) Task descriptions are truncated for brevity.Right: Theassistive value(AssistV)  of a decomposition measures the feasibility and speed of humans to repair the decomposed solution in the actual problem-solving process. For example, Decomposition B has a higherAssistVvalue than A in practice, as it further decomposes the complex if-statement into two simpler subtasks, which effectively assists humans in identifying a missing condition.One strategy to assist humans is task decomposition: as shown in Figure1, humans can more easily understand and repair complex solutions after they are decomposed into simpler pieces that correspond to specific subtasks.However, not all decompositions are helpful, and it is challenging for humans to design an effective one.
For example,shows that improper decomposition designed by novice programmers can impede human debugging performance.
To decompose better, we need methods beyond using fixed heuristic rulesor learning from author-crafted demonstrations.In this paper, we introduce a novel objective for learning task decomposition:assistive value(AssistV, Equation1) ,
which measures the feasibility and speed of humans to repair a decomposed solution in the actual annotation process (Figure1right) .
To improveAssistV, we first collect a dataset of decompositions, measure theirAssistV, and ask human annotators to provide a natural language critique on what makes a decomposition (not)  helpful.
Then we design a three-stage process to generate high-AssistVdecompositions, where each stage is implemented by an LLM that learns from our dataset in context: 1)  learn a critique modelfor predicting human critique on how to improve the initial decomposition for higherAssistV, 2)  learn a refine modelto incorporate the critique to refine the initial decomposition, and 3)  learn a rank modelto select a decomposition with highAssistV.We chose competitive programming as a testbed to validate our method for scalable oversight, since it is a challenging task that both LMs and humans alone struggle to solve.
We recruit 30 Python programmers, including 11 experts111This group includes medalists in National or International
Olympiad in Informatics.and 19 non-experts, to repair model-generated program solutions for competitive coding problems, resulting in a total of 177 worker hours.
Experiment results show that our method enables humans to solve 33.3% more problems, speeds up non-experts and experts by 3.3 and 2.4 times, and assists non-experts to match the performance of non-assisted experts.
We then analyze LMs’ ability to select decompositions with higherAssistV:
while humans’ intuitive judgment is not better than random (49.5%) , GPT-3.5-Turbo achieves 62.5% accuracy by learning from human repair experiences, and GPT-4 is 15.6% better. This result indicates that LMs could learn to perform better than humans at predicting what is more helpful for humans.Our core contributions are:•We assist humans with scalable oversight via automated task decomposition.•We introduce a novel objective for learning task decomposition:AssistV, and we propose a three-stage method to produce a high-AssistVdecomposition by learning to critique, refine, and rank decompositions.•We show that our method is effective in competitive programming, improving human supervision performance and bridging the expertise gap.Overall, even when LMs cannot solve the problem themselves, they can still learn to assist humans.
By learning from human experiences to repair solutions, more capable models can predict assistive values more accurately and sometimes more accurately than humans, highlighting the potential of learning-based methods to assist humans.
introduction
Learning Task Decomposition to Assist Humansin Competitive Programming
1Introduction
2406.04604
2MethodologyFigure 2:Method overview.Left: we sample multiple decompositions from LMs and evaluate them based on assistive valueand critique. We then construct pair-wise decompositions to demonstrate the difference between low- and high-AssistVdecompositions.Right: Starting from a vanilla decomposition generated by naively prompting LMs, we use the collected pair-wise data as in-context demonstrations to learn three models to critique, refine, and rank decompositions to better assist humans.2.1Task DefinitionOur objective is to assist human labelers in repairing model-generated solutions to complex tasks by learning a decomposition model. This model decomposes input solutions into multiple easier-to-repair pieces corresponding to specific subtasks.Formally, given a problemand an initial model solution, we aim to transforminto a decomposed solutionto improve its assistive value, as defined by:(1) wheredenotes the solution repaired by humans after spending timefrom its initial solution, andis a metric of solution quality. In competitive programming, we setas the passing rates on unit test cases. Assistive valuesummarizes the solution quality over the human repairing process starting from, with example trajectories shown in Figure4. A higher value ofindicates that assisted humans can more easily repair, thus efficiently providing a high-quality label to the problem.While there are diverse ways to decompose one single solution as shown in Figure1, effective decomposition that improves human problem-solving performance is non-trivial and can even be challenging for humans to devise. To decompose better, we need methods beyond prior methods that rely on fixed heuristic rules or learn from author-crafted demonstrations. In this paper, we propose to improve the assistive value of task decomposition by learning from human problem-solving experiences.Specifically, we first construct a training set of human repair experiences between different decomposed solutions, whereandis a natural language critique that explains why the decompositionleads to a lower assistive value than. Utilizing the collected training set, we learn to critique, refine, and rank decompositions to improve assistive value. Figure2presents the overview of our framework.2.2Data CollectionData PreparationFor each problemand an initial solution, we obtaindifferent decomposed solutionsby sampling from various LMs with few-shot prompting. See more details about data preparation in AppendixA.2.Recruiting Human AnnotatorsWe recruit annotators from college students. We conduct a pre-survey about their background and divide them into two groups based on their programming skills. We recruit 11 expert annotators who can solve Leetcode hard-level problems. Especially, 6 of them are medalists in the National or International Olympiad in Informatics. We recruit 19 non-expert annotators who can solve Leetcode medium-level problems but hardly solve hard-level problems. We conduct a warm-up test to train annotators to use our experiment environment. We further verify annotators’ programming skills based on their performance in the warm-up test.Collecting Assistive ValueFollowing our definition of assistive value in Equation1, we collect assistive value labels of different decompositions in the actual human annotation process. Specifically, for each sampled decomposed solution, we collect the following labels:•Assistive Value:We calculate the assistive value offollowing the definition described in Equation1.•Critique on decomposition:After repairing, we ask annotators to provide natural language critiqueon how decomposition assists or hinders their debugging.Constructing Pair-wise DemonstrationsWe makea comparison pair if they meet two requirements: (1)  There is a substantial difference in assistive value betweenand. (2)  The advantages in critiquematch the disadvantages in222In our experiments, we manually inspect human critiques and perform matching, as we only need a few examples for in-context learning. See AppendixA.2for more discussions., and henceandexplain whyleads to more efficient human repair than.2.3ModelsStarting with a vanilla decomposition generated by naively prompting LMs, we predict human critiques on it with, refine the decomposition according to the critique with, and rank candidate decompositions to select the final high-AssistVoutput with. Figure2presents the overview of our framework. Notably, we introduce critique as an integral step instead of directly generating refined decomposition since it provides enriched information for models to learn how decomposition can achieve improved assistive value. Additionally, it also enables controllable decomposition as humans can manually edit critiques (e.g., requiring specific decomposition on certain complex subtasks) .We next describe the detailed inputs and outputs of each model:•Critic Model: It takes a problemand a decomposed solutionas inputs, and outputs critique on how to improvefor higher assistive value.•Refine Model: It takes a problem, a decomposed solution, and a critiqueas inputs, and outputs a refined decomposed solution.•Ranking Model: It takes a problemand two decomposed solutions,as inputs, and outputs a ranking that predicts which decomposition leads to higher assistive value.For model training, inspired by recent works showing that modern LMs can learn to critique and refine model outputs via in-context learning, we use this approach to learn our three models, where the collected training dataare formatted into in-context examples for each model. Example prompts are shown in AppendixD.At inference time, we apply, followed by, to produce a refined decomposition set, and then useto select a decompositionamong them as the final decomposition.
body
Learning Task Decomposition to Assist Humansin Competitive Programming
2Methodology
2406.04604
3ExperimentsFigure 3:Humans provide higher-quality labels with decomposition. Dark color denotes strict accuracy of human-repaired programs; light color denotes test case average accuracy.Figure 4:Decomposition improves human efficiency. We plot the relationship between the human-repaired program’s test case average accuracy and human time cost.3.1SetupBenchmarkWe conduct experiments with the problems from two widely adopted competition-level code generation benchmarks, namely APPSand Code-Contests. For human evaluation, we filter those problems where the model-generated program directly passes all test cases and randomly sample 30 problems as the test data.MetricWe measure the supervision quality in competitive programming by programs’ passing rates on unit test cases. Specifically, we aggregate the program’s performance on test cases with two metrics.Test Case Average Accuracycomputes the average fraction of test cases passed among all the test cases.Strict Accuracycomputes the average fraction of programs that pass all test cases.BaselinesWe consider three baselines:•Initial: It prompts a code language modelwith the problemto generate a solutionwithout explicit instructions for decomposition.•Heuristic Decomposition: Inspired by McCabe’s cyclomatic complexity, a widely adopted metric for measuring code complexity in software engineering, we implement a heuristic baseline to decompose a complex program into simpler pieces. Specifically, we decompose each if-statement as well as for or while loops into a separate function since these code structures contribute to higher cyclomatic complexity. We then generate post-hoc subtask descriptions using GPT-4.•Vanilla Decomposition: It prompts a code language modelto perform decomposition with basic author-crafted demonstrations without learning from human feedback.Annotation ProcedureWe conduct experiments based on an internal Online Judge system. Problems are randomly assigned to each annotator while ensuring that they have not seen the assigned problem before (if they have, the problem will be reassigned)  and never repeatedly debug the same problem. Next, given a competition-level problem, several exemplified public test cases, and a solution, labelers are required to perform debugging. See AppendixCfor more annotation details.Implementation DetailsWe use GPT-4as our default backbone modelfor code generation and decomposition due to its superior in-context learning capabilities. To focus our evaluation on the impact of decomposition, we ensure the consistency between initial and decomposed solutions based on their outputs on test cases. If the consistency check fails, we reject the decomposition and retry within a sampling budget. The consistency check is applied in all baselines for fair comparisons. See AppendixA.1for more implementation details.3.2Assisting Human SupervisionWe evaluate the effectiveness of decomposition in aiding humans to repair programs and provide reliable supervision signals.3.2.1Quantitative AnalysisSupervising competitive programming is extremely challengingIn our experiments, highly experienced experts spend at least 68 minutes repairing 56.7% of the model-generated programs, and non-experts spend at least 74 minutes repairing 30% of the model-generated programs. These results indicate that competitive programming serves as a good testbed for studying scalable oversight.Decomposition improves human efficiencyFigure4demonstrates that humans assisted with our decomposition model significantly outperform those without the assistance in terms of the speed to repair model-generated programs. For instance, the time required to collect repaired programs with 68% test case accuracy from non-experts is reduced from 40 minutes to 12 minutes, achieving a 3.3speedup. Paired-tests confirm that these efficiency gains are statistically significant ()  for both experts and non-experts. These results indicate that our decomposition model can effectively improve the efficiency of human labelers.Humans provide higher-quality labels with decompositionBeyond efficiency, we evaluate how decomposition impacts the final quality of human labels. As shown in Figure4, our decomposition model enables humans to repair more model-generated programs. For instance, decomposition improves the strict accuracy of repaired programs from 56.7% to 90% for experts and from 30.0% to 63.3% for non-experts. Paired-tests confirm the significance ()  of these improvements. This verifies the effectiveness of decomposition in aiding humans to tackle complex tasks by breaking them down into more manageable subtasks.Decomposition enables non-experts to be comparable with expertsRemarkably, when assisted with our decomposition model, non-experts achieve performance comparable to non-assisted experts in labeling efficiency and quality. This aligns with the overarching goal of scalable oversight, empowering human labelers to oversee models in superhuman tasks.Figure 5:Decomposition brings more benefits to human labelers on hard problems.Decomposition benefits more on hard problemsTo further understand the impact of decomposition, we extract two subsets from our test data based on the time spent by non-experts to repair the model-generated programs: (1)  Easy: problems that take less than 25 minutes. (2)  Hard: problems that take more than 40 minutes. The results shown in Figure5reveal that easy problems benefit minimally from decomposition. However, when it comes to hard problems, decomposition significantly improves the labeling efficiency and final quality of human labelers. These results further underscore our decomposition’s utility in complex tasks.Humans prefer our decomposition modelAs shown in Figure4and Figure4, our decomposition model surpasses other decomposition baselines in enhancing human labeling efficiency and quality. We also ask labelers to consider whether the decomposition is helpful for them to repair the program in the post-survey. As shown in Figure6, labelers rate our decomposition results substantially more helpful than the baselines’. These results verify that LMs can learn to produce high-AssistVdecompositions by learning from human repair experiences.3.2.2Qualitative AnalysisTo further gain insight into what our decomposition model has learned from human feedback to enhance its utility, we analyze the collected human repairing traces alongside the corresponding critique, from which we draw the following qualitative observations.Figure 6:Helpfulness of decompositions evaluated by human labelers after repairing.Assisting humans by highlighting boundary conditionsWe observe that decomposing complex programs around boundary conditions effectively aids humans. As boundary conditions typically involve more straightforward logic than the core problem, humans can more easily understand and repair them. For instance, in one training demonstration (Figure13) ,check_validityhighlights all boundary conditions, earning positive human feedback for enabling them to quickly identify that problems are located incalculate_minimum_moves. This principle is learned by our decomposition model, as evidenced in Figure14, where it highlights two boundary conditions incheck_zero_pilesandcheck_same_piles, leading humans to quickly realize the missing third boundary condition.Assisting humans by creating simpler subtasksDecomposition reduces human workload by breaking down the initial solution into multiple simpler pieces, each corresponding to a specific subtask. For instance, in one training demonstration (Figure15) ,find_cyclesimplifies the complexfind_cycles, andgenerate_permutationhelps humans locate the actual bugs in a simple subtask. Similarly, in Figure16, our decomposition model creates a simple taskcalculate_participantswhich effectively isolates bugs. In addition, our decomposition model learns to decompose code pieces that humans typically struggle with (e.g., nested loops, binary search, and dynamic programming) , as exemplified in Figure15and Figure17.Assisting humans by presenting clear high-level logicDecomposition’s ability to offer clear high-level logic can accelerate comprehension and bug identification before delving into low-level details, as demonstrated in Figure17. Next, Figure18illustrates our decomposition model’s integration of this principle by creating two subtaskstoggle_doorsandtoggle_single_door. This high-level logic indicates that the current solution addresses each door independently, thereby enabling humans to locate bugs directly since doors are interrelated in the problem context.3.3Assisting AI SupervisionWith the verified effectiveness of decomposition in assisting human supervision, we further explore its potential to assist AI supervision. We suspect that some patterns of decompositions shown in Section3.2.2(e.g., creating simpler subtasks and presenting clear high-level logic)  can also help AI to evaluate (i.e., discriminate programs’ correctness)  and repair programs. We achieve AI supervision by prompting a code language modelto perform discrimination and repair.ProgramDiscriminationRepairAccAcc (Strict) Acc (Test Case) APPSInitial10.218.3(+0.0) 41.5(-0.3) Heuristic Decomp30.016.7(-1.6) 39.2(-2.6) Vanilla Decomp32.719.2(+0.9) 42.7(+0.9) Assistive Decomp42.921.7(+2.9) 47.4(+5.6) Code-ContestsInitial26.76.3(+0.0) 20.1(+1.3) Heuristic Decomp46.77.3(+1.0) 19.3(+0.5) Vanilla Decomp53.18.3(+2.0) 21.4(+2.6) Assistive Decomp62.512.5(+6.2) 26.3(+7.5) Table 1:Decomposition aids AI systems to discriminate and repair programs, improving discrimination accuracy and repaired solutions’ accuracy. Parenthetical values denote the accuracy variation from non-repair programs.Figure 7:Decomposition aids AI systems to provide self-supervision, where GPT-4 discriminates its own outputs, and weak-to-strong supervision, where GPT-3.5-turbo discriminates GPT-4-generated programs.Decomposition facilitates accurate self-supervisionWe first investigate decomposition’s impact on self-supervision, where we use GPT-4 to supervise itself. Results in Table1reveal that GPT-4 struggles to discriminate and repair programs generated by itself, echoing recent findings in LLMs’ self-correction abilities. However, decomposing the complex programs with our model improves both discrimination accuracy and repaired solution quality by a large margin. For instance, discrimination accuracy improves from 10.2% to 42.9% on APPS and from 26.7% to 62.5% on Code-Contests.Decomposition facilitates accurate weak-to-strong supervisionHaving verified the effectiveness of decomposition in aiding non-expert human labelers, we now investigate its potential to assist weak models in supervising strong models. We evaluate GPT-3.5-turbo’s ability to discriminate the programs generated by GPT-4. The results in Figure7reveal that our method leads to more accurate weak-to-strong AI supervision. For instance, it improves GPT-3.5-turbo’s discrimination accuracy from 6.1% to 22.4% on APPS, surpassing the non-assisted accuracy of GPT-4 (10.2%) .
body
Learning Task Decomposition to Assist Humansin Competitive Programming
3Experiments
2406.04604
4AnalysisMethodAccuracyIntuitive Human Preference49.5Cyclomatic Complexity48.4GPT-3.5-turbo Zero-shot54.3GPT-3.5-turbo Few-shot62.5GPT-4 Zero-shot66.7GPT-4 Few-shot () 78.1Table 2:Accuracy of the rank model in predicting the assistive value of decompositions in actual human repair processes.4.1Validity of the Rank ModelWe evaluate the effectiveness of our ranking modelin predicting the assistive value of decompositions. We construct paired decompositions annotated with real assistive value labels as the test data. The results shown in Table2indicate that LLMs can learn to select higher-AssistVdecomposition for assisting humans via in-context learning, surpassing prior heuristic or zero-shot baselines. Notably, we ask humans to provide preferences on paired decompositions without repairing them, which is shown to align poorly with their actual assistive value for human repair. These results highlight the importance of measuring assistive value based on actual human supervision experiences.See AppendixB.4for more details about these ranking baselines.4.2Ablations on the Critic ModelWe perform an ablation to explore the impact of our critic model. In the ablated version, we directly generate refined decompositions without incorporating predicted critiques. We useas a proxy to evaluate the assistive value of decompositions due to its reasonable performance (Section4.1) . The win rates between models with and withoutare 58.8% and 35.3%, respectively. These results indicate the effectiveness of learning from informative natural language human feedback.4.3Distilled Decomposition ModelIn this section, we aim to study whether moderate-size open-source LLMs can also learn to generate better decompositions to assist human or AI supervision while reducing API costs and enhancing reproducibility. We hence distill the knowledge of assistive decomposition from proprietary LLMs into an in-house model. Specifically, we create supervised datawith LLMs as illustrated in Section2.3, and optimizingovervia the standard MLE loss:where,, anddenote the problem, the initial model-generated solution, and the decomposed solution, respectively.We evaluate the performance of the Code-LLaMA-baseddistilled decomposition model in aiding human and AI supervision, finding it outperforms the vanilla decomposition based on Code-LLaMA and even GPT-4. These results reveal the effectiveness of distilling the knowledge of assistive decomposition from GPT-4 to Code-LLaMA. See AppendixB.5for detailed results.
body
Learning Task Decomposition to Assist Humansin Competitive Programming
4Analysis
2406.04604
5Related Work5.1Code GenerationLanguage models have achieved impressive performance in generating simple code piecesby learning from human demonstrations, and are being used to generate increasingly complex programs. In this context, our paper studies how to aid humans in efficiently providing reliable supervision to train models that don’t write buggy programs in complex and high-impact scenarios.5.2AI-assisted ProgrammingPrior works have explored various forms to assist human programmers, including code completion, bug location, and program repair. However, these methods rely on assistant models’ capacity for accurate code generation or repair—a requirement unmet in our setup, where models face challenges in performing either task. We thus explore an alternative form of assistance via task decomposition, which alleviates the burden on assistant models.5.3Task DecompositionTask decomposition has been extensively studied for tackling complex tasks, primarily focusing on enhancing model performanceand enabling efficient searching over the subtask space. In contrast, our paper studies decomposition from a human-centric perspective, aiming to facilitate human supervision over complex tasks, akin to’s work in aiding human supervision over book summarization. Concerning the implementation of decomposition, prior works mainly rely on fixed heuristic rules (e.g., decomposing each-line of codes)  or author-crafted demonstrations. In this paper, we take the first step towards advanced decomposition by learning to improve a novel objective: assistive value, which exactly measures the feasibility and speed of human repair in the actual annotation process.5.4Learning from Human FeedbackLearning from human feedback has been widely adopted in recent works for developing human-friendly general language models. Our work can also be understood as introducing this concept to the development of assistant models. Specifically, our introduced objective—assistive value–could also be introduced as a unique type of human feedback, which is gathered from actual human problem-solving experiences. Therefore, the collected human feedback could be more aligned with the ultimate goal of AI assistants: enhancing human problem-solving performance.5.5Scalable OversightAdvancements in LLMs have intensified the need for scalable, reliable human oversight on complex tasks that reach or even exceed the capabilities of human experts. Addressing this, prior research has explored various assistance methods, such as self-critiquing, AI debate, and decomposition. However, previous works overlook the feedback from actual assisted humans, potentially leading to reduced helpfulness of the assistance to humans.
To bridge this gap, we propose to learn assistance models from human feedback.
body
Learning Task Decomposition to Assist Humansin Competitive Programming
5Related Work
2406.04604
6ConclusionThis paper focuses on assisting humans in supervising LMs on complex problems by automated task decomposition. We introduce a novel objective for learning task decomposition: assistive value (AssistV) , which measures the feasibility and speed of humans to repair a decomposed solution. We collect a dataset of decompositions, measure theirAssistV, and ask human annotators to provide a natural language critique on what makes a decomposition (not)  helpful. We then learn to critique, refine, and rank decompositions to generate high-AssistVdecompositions. Experiment results demonstrate that our method can effectively assist humans in providing higher-quality supervision with significantly less time. Notably, our method assists non-experts in matching unassisted experts. Overall, we show that even when LMs cannot solve the problem themselves, they can still learn to assist humans by learning from human repair experiences. These results highlight the potential of learning-based methods to assist humans with scalable oversight.
body
Learning Task Decomposition to Assist Humansin Competitive Programming
6Conclusion
2406.04604
Limitations and Future WorkOne limitation of our work is that our non-expert participants, who could solve 30% of the problems while spending 74 minutes, are still more skilled than novice programmers (e.g., programmers who can only solve Leetcode easy-level problems)  or non-programmers. Nonetheless, we demonstrate the effectiveness of our decomposition model, which successfully enables assisted non-experts to be comparable with non-assisted experts. We leave the investigation of leveraging decomposition to assist novice programmers or even non-programmers as future work.Another limitation of our paper is that we did not conduct experiments on misleading benchmarks, which mainly consist of subtle errors that humans tend to overlook. The effectiveness of decomposition in aiding humans in detecting and fixing these subtle errors is worth studying in future work.Moreover, we only conduct experiments based on in-context learning and supervised learning. Given the reasonable performance of the rank model (Section4.1) , fine-tuning the decomposition model with reinforcement learning is worth studying in the future.Finally, our introduced objectiveAssistV, which highlights the possibility of learning an effective assistance model from human problem-solving experiences, can be further extended to other assistance forms beyond decomposition. For example, recent works explore using model-generated explanations to assist humans in complex decision-making tasks (e.g., complex question answering) . However, the practical benefits of model explanations are mixed as they may introduce additional human workload for understanding the explanation or even mislead humans to incorrect results. Therefore, future works can explore what kind of explanations can better assist humans by measuring theAssistVof different explanations and learning to improveAssistV.
body
Learning Task Decomposition to Assist Humansin Competitive Programming
Limitations and Future Work
2406.04604
Ethics StatementWe recruit annotators mainly from college students. We inform annotators in advance how their annotation data will be collected and used. We pay them 30 USD per hour, which is higher than the average wage of the local residents.
body
Learning Task Decomposition to Assist Humansin Competitive Programming
Ethics Statement
2406.04604
AcknowledgementsThis work was supported by the National Science Foundation for Distinguished Young Scholars (with No. 62125604)  and the NSFC projects (with No. 62306160) . This work was also supported by China National Postdoctoral Program for Innovative Talents (No. BX20230194)  and China Postdoctoral Science Foundation (No. 2023M731952) .
conclusion
Learning Task Decomposition to Assist Humansin Competitive Programming
Acknowledgements