<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2206.15448] Learning Iterative Reasoning through Energy Minimization</title><meta property="og:description" content="Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition.
However, it struggles with tasks requiring nontrivial reasoning, such as algorithmic computation.
Hum…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Learning Iterative Reasoning through Energy Minimization">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Learning Iterative Reasoning through Energy Minimization">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2206.15448">

<!--Generated on Tue Feb 27 03:28:59 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="keywords" lang="en" content="Energy Based Models,  Optimization,  Reasoning,  Algorithms,  Dynamic Programming">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document ltx_pruned_first">
<h1 class="ltx_title ltx_title_document">Learning Iterative Reasoning through Energy Minimization</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Yilun Du
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Shuang Li
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Joshua Tenenbaum
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Igor Mordatch
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id1.id1" class="ltx_p">Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition.
However, it struggles with tasks requiring nontrivial reasoning, such as algorithmic computation.
Humans are able to solve such tasks through iterative reasoning – spending more time thinking about harder tasks.
Most existing neural networks, however, exhibit a fixed computational budget controlled by the neural network architecture, preventing additional computational processing on harder tasks.
In this work, we present a new framework for iterative reasoning with neural networks.
We train a neural network to parameterize an energy landscape over all outputs, and implement each step of the iterative reasoning as an energy minimization step to find a minimal energy solution.
By formulating reasoning as an energy minimization problem, for harder problems that lead to more complex energy landscapes, we may then adjust our underlying computational budget by running a more complex optimization procedure.
We empirically illustrate that our iterative reasoning approach can solve more accurate and generalizable algorithmic reasoning tasks in both graph and continuous domains.
Finally, we illustrate that our approach can recursively solve algorithmic problems requiring nested reasoning. Code and additional information is available at <a target="_blank" href="https://energy-based-model.github.io/iterative-reasoning-as-energy-minimization/" title="" class="ltx_ref ltx_href">https://energy-based-model.github.io/iterative-reasoning-as-energy-minimization/</a>.</p>
</div>
<div class="ltx_keywords">Energy Based Models, Optimization, Reasoning, Algorithms, Dynamic Programming
</div>
<div id="p2" class="ltx_para">
<br class="ltx_break">
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Human thinking is often characterized in terms of mechanisms for two distinct modes of cognitive processing: <span id="S1.p1.1.1" class="ltx_text ltx_font_smallcaps">System 1</span> mechanisms for fast, habitual, and associative processing, and <span id="S1.p1.1.2" class="ltx_text ltx_font_smallcaps">System 2</span> mechanisms for slower, more deliberate and controlled, symbolic reasoning <cite class="ltx_cite ltx_citemacro_citep">(Kahneman, <a href="#bib.bib25" title="" class="ltx_ref">2011</a>)</cite>. Neural networks have excelled at habitual <span id="S1.p1.1.3" class="ltx_text ltx_font_smallcaps">System 1</span>-style processing in familiar environments and task contexts, such as mapping images of familiar objects to their semantic classes, or familiar locations to the corresponding routes of movement. However, confronted with a novel environment or task demanding a more flexible response, humans can invoke controlled <span id="S1.p1.1.4" class="ltx_text ltx_font_smallcaps">System 2</span> processes, often in the form of iteratively reasoning about relationships between observed entities that builds on past experiences primarily through shared abstractions and algorithms rather than direct re-use of concrete, habitual responses. Such flexible processing of novel inputs is difficult for neural networks, with even large pretrained language models <cite class="ltx_cite ltx_citemacro_citep">(Radford et al., <a href="#bib.bib33" title="" class="ltx_ref">2019</a>)</cite> that have proven effective in many zero-shot generalization contexts failing to extrapolate the simplest algorithmic operations, such as addition, to more complex inputs.</p>
</div>
<figure id="S1.F1" class="ltx_figure"><img src="/html/2206.15448/assets/x1.png" id="S1.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="438" height="235" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span><span id="S1.F1.14.7" class="ltx_text ltx_font_bold" style="font-size:90%;">Reasoning as Energy Minimization<span id="S1.F1.9.2.2" class="ltx_text ltx_font_medium"> – IREM formulates an algorithmic reasoning problem with inputs <math id="S1.F1.8.1.1.m1.2" class="ltx_Math" alttext="{\bm{x}},{\bm{y}}" display="inline"><semantics id="S1.F1.8.1.1.m1.2b"><mrow id="S1.F1.8.1.1.m1.2.3.2" xref="S1.F1.8.1.1.m1.2.3.1.cmml"><mi id="S1.F1.8.1.1.m1.1.1" xref="S1.F1.8.1.1.m1.1.1.cmml">𝒙</mi><mo id="S1.F1.8.1.1.m1.2.3.2.1" xref="S1.F1.8.1.1.m1.2.3.1.cmml">,</mo><mi id="S1.F1.8.1.1.m1.2.2" xref="S1.F1.8.1.1.m1.2.2.cmml">𝒚</mi></mrow><annotation-xml encoding="MathML-Content" id="S1.F1.8.1.1.m1.2c"><list id="S1.F1.8.1.1.m1.2.3.1.cmml" xref="S1.F1.8.1.1.m1.2.3.2"><ci id="S1.F1.8.1.1.m1.1.1.cmml" xref="S1.F1.8.1.1.m1.1.1">𝒙</ci><ci id="S1.F1.8.1.1.m1.2.2.cmml" xref="S1.F1.8.1.1.m1.2.2">𝒚</ci></list></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.8.1.1.m1.2d">{\bm{x}},{\bm{y}}</annotation></semantics></math> and output <math id="S1.F1.9.2.2.m2.1" class="ltx_Math" alttext="{\bm{z}}" display="inline"><semantics id="S1.F1.9.2.2.m2.1b"><mi id="S1.F1.9.2.2.m2.1.1" xref="S1.F1.9.2.2.m2.1.1.cmml">𝒛</mi><annotation-xml encoding="MathML-Content" id="S1.F1.9.2.2.m2.1c"><ci id="S1.F1.9.2.2.m2.1.1.cmml" xref="S1.F1.9.2.2.m2.1.1">𝒛</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.9.2.2.m2.1d">{\bm{z}}</annotation></semantics></math>, such as summation (</span>left<span id="S1.F1.12.5.5" class="ltx_text ltx_font_medium">), as an iterative energy minimization problem over a learned energy function <math id="S1.F1.10.3.3.m1.3" class="ltx_Math" alttext="E_{\theta}({\bm{z}}|{\bm{x}},{\bm{y}})" display="inline"><semantics id="S1.F1.10.3.3.m1.3b"><mrow id="S1.F1.10.3.3.m1.3.3" xref="S1.F1.10.3.3.m1.3.3.cmml"><msub id="S1.F1.10.3.3.m1.3.3.3" xref="S1.F1.10.3.3.m1.3.3.3.cmml"><mi id="S1.F1.10.3.3.m1.3.3.3.2" xref="S1.F1.10.3.3.m1.3.3.3.2.cmml">E</mi><mi id="S1.F1.10.3.3.m1.3.3.3.3" xref="S1.F1.10.3.3.m1.3.3.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S1.F1.10.3.3.m1.3.3.2" xref="S1.F1.10.3.3.m1.3.3.2.cmml">​</mo><mrow id="S1.F1.10.3.3.m1.3.3.1.1" xref="S1.F1.10.3.3.m1.3.3.1.1.1.cmml"><mo stretchy="false" id="S1.F1.10.3.3.m1.3.3.1.1.2" xref="S1.F1.10.3.3.m1.3.3.1.1.1.cmml">(</mo><mrow id="S1.F1.10.3.3.m1.3.3.1.1.1" xref="S1.F1.10.3.3.m1.3.3.1.1.1.cmml"><mi id="S1.F1.10.3.3.m1.3.3.1.1.1.2" xref="S1.F1.10.3.3.m1.3.3.1.1.1.2.cmml">𝒛</mi><mo fence="false" id="S1.F1.10.3.3.m1.3.3.1.1.1.1" xref="S1.F1.10.3.3.m1.3.3.1.1.1.1.cmml">|</mo><mrow id="S1.F1.10.3.3.m1.3.3.1.1.1.3.2" xref="S1.F1.10.3.3.m1.3.3.1.1.1.3.1.cmml"><mi id="S1.F1.10.3.3.m1.1.1" xref="S1.F1.10.3.3.m1.1.1.cmml">𝒙</mi><mo id="S1.F1.10.3.3.m1.3.3.1.1.1.3.2.1" xref="S1.F1.10.3.3.m1.3.3.1.1.1.3.1.cmml">,</mo><mi id="S1.F1.10.3.3.m1.2.2" xref="S1.F1.10.3.3.m1.2.2.cmml">𝒚</mi></mrow></mrow><mo stretchy="false" id="S1.F1.10.3.3.m1.3.3.1.1.3" xref="S1.F1.10.3.3.m1.3.3.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.F1.10.3.3.m1.3c"><apply id="S1.F1.10.3.3.m1.3.3.cmml" xref="S1.F1.10.3.3.m1.3.3"><times id="S1.F1.10.3.3.m1.3.3.2.cmml" xref="S1.F1.10.3.3.m1.3.3.2"></times><apply id="S1.F1.10.3.3.m1.3.3.3.cmml" xref="S1.F1.10.3.3.m1.3.3.3"><csymbol cd="ambiguous" id="S1.F1.10.3.3.m1.3.3.3.1.cmml" xref="S1.F1.10.3.3.m1.3.3.3">subscript</csymbol><ci id="S1.F1.10.3.3.m1.3.3.3.2.cmml" xref="S1.F1.10.3.3.m1.3.3.3.2">𝐸</ci><ci id="S1.F1.10.3.3.m1.3.3.3.3.cmml" xref="S1.F1.10.3.3.m1.3.3.3.3">𝜃</ci></apply><apply id="S1.F1.10.3.3.m1.3.3.1.1.1.cmml" xref="S1.F1.10.3.3.m1.3.3.1.1"><csymbol cd="latexml" id="S1.F1.10.3.3.m1.3.3.1.1.1.1.cmml" xref="S1.F1.10.3.3.m1.3.3.1.1.1.1">conditional</csymbol><ci id="S1.F1.10.3.3.m1.3.3.1.1.1.2.cmml" xref="S1.F1.10.3.3.m1.3.3.1.1.1.2">𝒛</ci><list id="S1.F1.10.3.3.m1.3.3.1.1.1.3.1.cmml" xref="S1.F1.10.3.3.m1.3.3.1.1.1.3.2"><ci id="S1.F1.10.3.3.m1.1.1.cmml" xref="S1.F1.10.3.3.m1.1.1">𝒙</ci><ci id="S1.F1.10.3.3.m1.2.2.cmml" xref="S1.F1.10.3.3.m1.2.2">𝒚</ci></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.10.3.3.m1.3d">E_{\theta}({\bm{z}}|{\bm{x}},{\bm{y}})</annotation></semantics></math> which parameterizes an energy landscape over all possible outputs <math id="S1.F1.11.4.4.m2.1" class="ltx_Math" alttext="{\bm{z}}" display="inline"><semantics id="S1.F1.11.4.4.m2.1b"><mi id="S1.F1.11.4.4.m2.1.1" xref="S1.F1.11.4.4.m2.1.1.cmml">𝒛</mi><annotation-xml encoding="MathML-Content" id="S1.F1.11.4.4.m2.1c"><ci id="S1.F1.11.4.4.m2.1.1.cmml" xref="S1.F1.11.4.4.m2.1.1">𝒛</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.11.4.4.m2.1d">{\bm{z}}</annotation></semantics></math> conditioned on inputs <math id="S1.F1.12.5.5.m3.2" class="ltx_Math" alttext="{\bm{x}},{\bm{y}}" display="inline"><semantics id="S1.F1.12.5.5.m3.2b"><mrow id="S1.F1.12.5.5.m3.2.3.2" xref="S1.F1.12.5.5.m3.2.3.1.cmml"><mi id="S1.F1.12.5.5.m3.1.1" xref="S1.F1.12.5.5.m3.1.1.cmml">𝒙</mi><mo id="S1.F1.12.5.5.m3.2.3.2.1" xref="S1.F1.12.5.5.m3.2.3.1.cmml">,</mo><mi id="S1.F1.12.5.5.m3.2.2" xref="S1.F1.12.5.5.m3.2.2.cmml">𝒚</mi></mrow><annotation-xml encoding="MathML-Content" id="S1.F1.12.5.5.m3.2c"><list id="S1.F1.12.5.5.m3.2.3.1.cmml" xref="S1.F1.12.5.5.m3.2.3.2"><ci id="S1.F1.12.5.5.m3.1.1.cmml" xref="S1.F1.12.5.5.m3.1.1">𝒙</ci><ci id="S1.F1.12.5.5.m3.2.2.cmml" xref="S1.F1.12.5.5.m3.2.2">𝒚</ci></list></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.12.5.5.m3.2d">{\bm{x}},{\bm{y}}</annotation></semantics></math> (</span>right<span id="S1.F1.14.7.7" class="ltx_text ltx_font_medium">). We visualize the first two dimensions <math id="S1.F1.13.6.6.m1.1" class="ltx_Math" alttext="{\bm{z}}_{1}" display="inline"><semantics id="S1.F1.13.6.6.m1.1b"><msub id="S1.F1.13.6.6.m1.1.1" xref="S1.F1.13.6.6.m1.1.1.cmml"><mi id="S1.F1.13.6.6.m1.1.1.2" xref="S1.F1.13.6.6.m1.1.1.2.cmml">𝒛</mi><mn id="S1.F1.13.6.6.m1.1.1.3" xref="S1.F1.13.6.6.m1.1.1.3.cmml">1</mn></msub><annotation-xml encoding="MathML-Content" id="S1.F1.13.6.6.m1.1c"><apply id="S1.F1.13.6.6.m1.1.1.cmml" xref="S1.F1.13.6.6.m1.1.1"><csymbol cd="ambiguous" id="S1.F1.13.6.6.m1.1.1.1.cmml" xref="S1.F1.13.6.6.m1.1.1">subscript</csymbol><ci id="S1.F1.13.6.6.m1.1.1.2.cmml" xref="S1.F1.13.6.6.m1.1.1.2">𝒛</ci><cn type="integer" id="S1.F1.13.6.6.m1.1.1.3.cmml" xref="S1.F1.13.6.6.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.13.6.6.m1.1d">{\bm{z}}_{1}</annotation></semantics></math> and <math id="S1.F1.14.7.7.m2.1" class="ltx_Math" alttext="{\bm{z}}_{2}" display="inline"><semantics id="S1.F1.14.7.7.m2.1b"><msub id="S1.F1.14.7.7.m2.1.1" xref="S1.F1.14.7.7.m2.1.1.cmml"><mi id="S1.F1.14.7.7.m2.1.1.2" xref="S1.F1.14.7.7.m2.1.1.2.cmml">𝒛</mi><mn id="S1.F1.14.7.7.m2.1.1.3" xref="S1.F1.14.7.7.m2.1.1.3.cmml">2</mn></msub><annotation-xml encoding="MathML-Content" id="S1.F1.14.7.7.m2.1c"><apply id="S1.F1.14.7.7.m2.1.1.cmml" xref="S1.F1.14.7.7.m2.1.1"><csymbol cd="ambiguous" id="S1.F1.14.7.7.m2.1.1.1.cmml" xref="S1.F1.14.7.7.m2.1.1">subscript</csymbol><ci id="S1.F1.14.7.7.m2.1.1.2.cmml" xref="S1.F1.14.7.7.m2.1.1.2">𝒛</ci><cn type="integer" id="S1.F1.14.7.7.m2.1.1.3.cmml" xref="S1.F1.14.7.7.m2.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.14.7.7.m2.1d">{\bm{z}}_{2}</annotation></semantics></math>. Each iteration of energy minimization is shown by a star. </span></span></figcaption>
</figure>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">Iterative reasoning, the ability to repeatedly apply an underlying computation to the outputs of previous reasoning steps, is a crucial component of scalable <span id="S1.p2.1.1" class="ltx_text ltx_font_smallcaps">System 2</span>-style processing. We may take abstractions learned from simpler variants of a problem, and iteratively apply them to solve harder variants of potentially unbounded complexity. As an example, having learned how to compute shortest paths on small graphs, we might generalize and apply our algorithmic intuition sequentially to substantially larger graph problems at test time through iterative application of learned computations. Such iterative processing enables humans to solve more challenging tasks, such as question answering, arithmetic calculation, or proof writing, even in novel or unfamiliar domains.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.1" class="ltx_p">We present a new neural approach towards iterative reasoning, which we formulate as an energy minimization process on a learned energy landscape (Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>). By representing individual steps of reasoning as an optimization process, we may iteratively reason for longer on harder problems by running additional steps of optimization on the induced energy landscapes. Simultaneously, by monitoring the geometric energy landscape surrounding an optimized solution, we may automatically determine the completion of the algorithmic computation (by checking the presence of a local energy minimum). We refer to our underlying reasoning framework as Iterative Reasoning as Energy Minimization (IREM).</p>
</div>
<div id="S1.p4" class="ltx_para">
<p id="S1.p4.1" class="ltx_p">To evaluate and benchmark the effectiveness of iterative reasoning using IREM, we propose and construct a suite of different algorithmic reasoning tasks on both graph and continuous domains. Effective algorithmic reasoning requires repetitive application of underlying algorithmic computations, dependent on problem complexity, and thus serves as a natural benchmark for iterative reasoning. We compare IREM with past works on our algorithmic benchmark for iterative reasoning and find that IREM outperforms prior works in performance and generalization.</p>
</div>
<div id="S1.p5" class="ltx_para">
<p id="S1.p5.1" class="ltx_p">Our contributions in this paper are threefold. First, we present IREM, a new framework for iterative reasoning and analyze why it is beneficial to use such a framework for reasoning. Second, we present a benchmark for iterative algorithmic reasoning, both on graphs and continuous vector inputs, and show that IREM significantly outperforms prior approaches in both performance and generalization. Finally, we show how our approach can recursively solve algorithmic computation requiring nested reasoning. Our results point to IREM as a promising new approach towards iterative reasoning.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Related Work</h2>

<div id="S2.p1" class="ltx_para">
<p id="S2.p1.1" class="ltx_p"><span id="S2.p1.1.1" class="ltx_text ltx_font_bold">Iterative Reasoning</span>  A variety of recent works have explored the integration of iterative reasoning into neural networks. One branch of works implements iterative reasoning by constructing neural programmatic operations <cite class="ltx_cite ltx_citemacro_citep">(Graves et al., <a href="#bib.bib21" title="" class="ltx_ref">2014</a>; Reed &amp; De Freitas, <a href="#bib.bib35" title="" class="ltx_ref">2015</a>; Banino et al., <a href="#bib.bib5" title="" class="ltx_ref">2021</a>)</cite> which are repeatedly executed till halting. Another branch of work implements iterative reasoning through recurrent computation <cite class="ltx_cite ltx_citemacro_citep">(Graves, <a href="#bib.bib20" title="" class="ltx_ref">2016</a>; Bolukbasi et al., <a href="#bib.bib7" title="" class="ltx_ref">2017</a>; Chung et al., <a href="#bib.bib11" title="" class="ltx_ref">2016</a>; Schwarzschild et al., <a href="#bib.bib38" title="" class="ltx_ref">2021</a>)</cite>. A key challenge with both types of approaches lies in the halting time of computation. Existing approaches learn halting policies through reinforcement learning <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib10" title="" class="ltx_ref">2020</a>; Chung et al., <a href="#bib.bib11" title="" class="ltx_ref">2016</a>)</cite>, heuristic policies <cite class="ltx_cite ltx_citemacro_citep">(Bolukbasi et al., <a href="#bib.bib7" title="" class="ltx_ref">2017</a>)</cite>, or variational inference <cite class="ltx_cite ltx_citemacro_citep">(Graves, <a href="#bib.bib20" title="" class="ltx_ref">2016</a>; Banino et al., <a href="#bib.bib5" title="" class="ltx_ref">2021</a>)</cite>. Such approaches are unstable in nature <cite class="ltx_cite ltx_citemacro_citep">(Banino et al., <a href="#bib.bib5" title="" class="ltx_ref">2021</a>)</cite>, and many of them require manual hyper-parameter specification. We present an orthogonal approach towards implementing iterative reasoning as an energy minimization procedure on a learned energy landscape. By determining when a local energy minimum has been found, our approach provides a natural mechanism for terminating computation.</p>
</div>
<div id="S2.p2" class="ltx_para">
<p id="S2.p2.1" class="ltx_p"><span id="S2.p2.1.1" class="ltx_text ltx_font_bold">Algorithmic Reasoning with External Memory</span>  Several approaches towards iterative reasoning utilize an external memory scratchpad for algorithmic computation. Such a scratchpad enables models to store intermediate algorithmic computations, and thus boosts the underlying performance of the algorithm <cite class="ltx_cite ltx_citemacro_citep">(Graves et al., <a href="#bib.bib21" title="" class="ltx_ref">2014</a>; Reed &amp; De Freitas, <a href="#bib.bib35" title="" class="ltx_ref">2015</a>; Cai et al., <a href="#bib.bib9" title="" class="ltx_ref">2017</a>; Kaiser &amp; Sutskever, <a href="#bib.bib26" title="" class="ltx_ref">2015</a>)</cite>. In the Appendix B, we illustrate a manner through which we may utilize an external memory with IREM to improve underlying further improve reasoning performance.</p>
</div>
<div id="S2.p3" class="ltx_para">
<p id="S2.p3.1" class="ltx_p"><span id="S2.p3.1.1" class="ltx_text ltx_font_bold">Optimization Based Computational Blocks</span>  Prior works have explored optimization as a computation block to solve different tasks. In <cite class="ltx_cite ltx_citemacro_citep">(Brockett, <a href="#bib.bib8" title="" class="ltx_ref">1991</a>)</cite>, a dynamical system is constructed that can solve various algorithmic tasks. Optimization has since been used as an intermediate neural network computation block for quadratic programs <cite class="ltx_cite ltx_citemacro_citep">(Amos &amp; Kolter, <a href="#bib.bib1" title="" class="ltx_ref">2017</a>; Donti et al., <a href="#bib.bib13" title="" class="ltx_ref">2017</a>)</cite> and submodular programs <cite class="ltx_cite ltx_citemacro_citep">(Djolonga &amp; Krause, <a href="#bib.bib12" title="" class="ltx_ref">2017</a>; Wilder et al., <a href="#bib.bib40" title="" class="ltx_ref">2019</a>)</cite> for flexible neural networks. Most similar to our work, <cite class="ltx_cite ltx_citemacro_citet">Bai et al. (<a href="#bib.bib4" title="" class="ltx_ref">2019</a>)</cite> utilizes equilibrium energy minimization as an intermediate computation block for memory-efficient neural networks. In contrast, we explore how direct optimization over a learned energy landscape can enable generalizable iterative reasoning. Concurrent to our work, <cite class="ltx_cite ltx_citemacro_citep">(Rubanova et al., <a href="#bib.bib36" title="" class="ltx_ref">2021</a>)</cite> utilizes energy minimization to simulate physical dynamics.</p>
</div>
<div id="S2.p4" class="ltx_para">
<p id="S2.p4.1" class="ltx_p"><span id="S2.p4.1.1" class="ltx_text ltx_font_bold">Energy-Based Models</span>  Our work is related to works in Energy-Based Models (EBMs) <cite class="ltx_cite ltx_citemacro_citep">(LeCun et al., <a href="#bib.bib29" title="" class="ltx_ref">2006</a>)</cite>. Most recent works using EBMs have focused on learning probabilistic models over data <cite class="ltx_cite ltx_citemacro_citep">(Du &amp; Mordatch, <a href="#bib.bib14" title="" class="ltx_ref">2019</a>; Nijkamp et al., <a href="#bib.bib32" title="" class="ltx_ref">2019</a>; Grathwohl et al., <a href="#bib.bib19" title="" class="ltx_ref">2020</a>; Du et al., <a href="#bib.bib15" title="" class="ltx_ref">2020</a>, <a href="#bib.bib17" title="" class="ltx_ref">2021b</a>; Arbel et al., <a href="#bib.bib3" title="" class="ltx_ref">2020</a>; Li et al., <a href="#bib.bib31" title="" class="ltx_ref">2020</a>; Xiao et al., <a href="#bib.bib41" title="" class="ltx_ref">2020</a>)</cite>. Instead of using EBMs as a probabilistic model, we use EBMs to define an energy landscape for solving iterative reasoning problems.</p>
</div>
<div id="S2.p5" class="ltx_para">
<p id="S2.p5.1" class="ltx_p"><span id="S2.p5.1.1" class="ltx_text ltx_font_bold">Learning Optimizers</span>  Our work utilizes backpropagation through intermediate optimization steps to train our energy function. Prior work has explored a similar idea of backpropagation through optimization to learn meta optimizers <cite class="ltx_cite ltx_citemacro_citep">(Andrychowicz et al., <a href="#bib.bib2" title="" class="ltx_ref">2016</a>; Ravi &amp; Larochelle, <a href="#bib.bib34" title="" class="ltx_ref">2016</a>; Bengio et al., <a href="#bib.bib6" title="" class="ltx_ref">1995</a>; Schmidhuber, <a href="#bib.bib37" title="" class="ltx_ref">1992</a>; Hochreiter et al., <a href="#bib.bib22" title="" class="ltx_ref">2001</a>; Li &amp; Malik, <a href="#bib.bib30" title="" class="ltx_ref">2016</a>)</cite> which enable more sample efficient and faster training of neural networks. In this work, we utilize this approach to learn an energy function so that a number of optimization steps on the energy function results in a solution to the algorithmic reasoning problem.</p>
</div>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Learning Iterative Reasoning through Energy Optimization</h2>

<div id="S3.p1" class="ltx_para">
<p id="S3.p1.8" class="ltx_p">Let <math id="S3.p1.1.m1.2" class="ltx_Math" alttext="\mathcal{D}=\{X,Y\}" display="inline"><semantics id="S3.p1.1.m1.2a"><mrow id="S3.p1.1.m1.2.3" xref="S3.p1.1.m1.2.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.p1.1.m1.2.3.2" xref="S3.p1.1.m1.2.3.2.cmml">𝒟</mi><mo id="S3.p1.1.m1.2.3.1" xref="S3.p1.1.m1.2.3.1.cmml">=</mo><mrow id="S3.p1.1.m1.2.3.3.2" xref="S3.p1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.p1.1.m1.2.3.3.2.1" xref="S3.p1.1.m1.2.3.3.1.cmml">{</mo><mi id="S3.p1.1.m1.1.1" xref="S3.p1.1.m1.1.1.cmml">X</mi><mo id="S3.p1.1.m1.2.3.3.2.2" xref="S3.p1.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.p1.1.m1.2.2" xref="S3.p1.1.m1.2.2.cmml">Y</mi><mo stretchy="false" id="S3.p1.1.m1.2.3.3.2.3" xref="S3.p1.1.m1.2.3.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.1.m1.2b"><apply id="S3.p1.1.m1.2.3.cmml" xref="S3.p1.1.m1.2.3"><eq id="S3.p1.1.m1.2.3.1.cmml" xref="S3.p1.1.m1.2.3.1"></eq><ci id="S3.p1.1.m1.2.3.2.cmml" xref="S3.p1.1.m1.2.3.2">𝒟</ci><set id="S3.p1.1.m1.2.3.3.1.cmml" xref="S3.p1.1.m1.2.3.3.2"><ci id="S3.p1.1.m1.1.1.cmml" xref="S3.p1.1.m1.1.1">𝑋</ci><ci id="S3.p1.1.m1.2.2.cmml" xref="S3.p1.1.m1.2.2">𝑌</ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.1.m1.2c">\mathcal{D}=\{X,Y\}</annotation></semantics></math> be a dataset of algorithmic reasoning problems consisting of inputs <math id="S3.p1.2.m2.1" class="ltx_Math" alttext="{\bm{x}}\in\mathbb{R}^{O}" display="inline"><semantics id="S3.p1.2.m2.1a"><mrow id="S3.p1.2.m2.1.1" xref="S3.p1.2.m2.1.1.cmml"><mi id="S3.p1.2.m2.1.1.2" xref="S3.p1.2.m2.1.1.2.cmml">𝒙</mi><mo id="S3.p1.2.m2.1.1.1" xref="S3.p1.2.m2.1.1.1.cmml">∈</mo><msup id="S3.p1.2.m2.1.1.3" xref="S3.p1.2.m2.1.1.3.cmml"><mi id="S3.p1.2.m2.1.1.3.2" xref="S3.p1.2.m2.1.1.3.2.cmml">ℝ</mi><mi id="S3.p1.2.m2.1.1.3.3" xref="S3.p1.2.m2.1.1.3.3.cmml">O</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.2.m2.1b"><apply id="S3.p1.2.m2.1.1.cmml" xref="S3.p1.2.m2.1.1"><in id="S3.p1.2.m2.1.1.1.cmml" xref="S3.p1.2.m2.1.1.1"></in><ci id="S3.p1.2.m2.1.1.2.cmml" xref="S3.p1.2.m2.1.1.2">𝒙</ci><apply id="S3.p1.2.m2.1.1.3.cmml" xref="S3.p1.2.m2.1.1.3"><csymbol cd="ambiguous" id="S3.p1.2.m2.1.1.3.1.cmml" xref="S3.p1.2.m2.1.1.3">superscript</csymbol><ci id="S3.p1.2.m2.1.1.3.2.cmml" xref="S3.p1.2.m2.1.1.3.2">ℝ</ci><ci id="S3.p1.2.m2.1.1.3.3.cmml" xref="S3.p1.2.m2.1.1.3.3">𝑂</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.2.m2.1c">{\bm{x}}\in\mathbb{R}^{O}</annotation></semantics></math> and corresponding solutions <math id="S3.p1.3.m3.1" class="ltx_Math" alttext="{\bm{y}}\in\mathbb{R}^{M}" display="inline"><semantics id="S3.p1.3.m3.1a"><mrow id="S3.p1.3.m3.1.1" xref="S3.p1.3.m3.1.1.cmml"><mi id="S3.p1.3.m3.1.1.2" xref="S3.p1.3.m3.1.1.2.cmml">𝒚</mi><mo id="S3.p1.3.m3.1.1.1" xref="S3.p1.3.m3.1.1.1.cmml">∈</mo><msup id="S3.p1.3.m3.1.1.3" xref="S3.p1.3.m3.1.1.3.cmml"><mi id="S3.p1.3.m3.1.1.3.2" xref="S3.p1.3.m3.1.1.3.2.cmml">ℝ</mi><mi id="S3.p1.3.m3.1.1.3.3" xref="S3.p1.3.m3.1.1.3.3.cmml">M</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.3.m3.1b"><apply id="S3.p1.3.m3.1.1.cmml" xref="S3.p1.3.m3.1.1"><in id="S3.p1.3.m3.1.1.1.cmml" xref="S3.p1.3.m3.1.1.1"></in><ci id="S3.p1.3.m3.1.1.2.cmml" xref="S3.p1.3.m3.1.1.2">𝒚</ci><apply id="S3.p1.3.m3.1.1.3.cmml" xref="S3.p1.3.m3.1.1.3"><csymbol cd="ambiguous" id="S3.p1.3.m3.1.1.3.1.cmml" xref="S3.p1.3.m3.1.1.3">superscript</csymbol><ci id="S3.p1.3.m3.1.1.3.2.cmml" xref="S3.p1.3.m3.1.1.3.2">ℝ</ci><ci id="S3.p1.3.m3.1.1.3.3.cmml" xref="S3.p1.3.m3.1.1.3.3">𝑀</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.3.m3.1c">{\bm{y}}\in\mathbb{R}^{M}</annotation></semantics></math>. Our goal is to learn a neural network operator <math id="S3.p1.4.m4.1" class="ltx_Math" alttext="\text{Alg}_{\theta}(\cdot)" display="inline"><semantics id="S3.p1.4.m4.1a"><mrow id="S3.p1.4.m4.1.2" xref="S3.p1.4.m4.1.2.cmml"><msub id="S3.p1.4.m4.1.2.2" xref="S3.p1.4.m4.1.2.2.cmml"><mtext id="S3.p1.4.m4.1.2.2.2" xref="S3.p1.4.m4.1.2.2.2a.cmml">Alg</mtext><mi id="S3.p1.4.m4.1.2.2.3" xref="S3.p1.4.m4.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.2.1" xref="S3.p1.4.m4.1.2.1.cmml">​</mo><mrow id="S3.p1.4.m4.1.2.3.2" xref="S3.p1.4.m4.1.2.cmml"><mo stretchy="false" id="S3.p1.4.m4.1.2.3.2.1" xref="S3.p1.4.m4.1.2.cmml">(</mo><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.1" xref="S3.p1.4.m4.1.1.cmml">⋅</mo><mo stretchy="false" id="S3.p1.4.m4.1.2.3.2.2" xref="S3.p1.4.m4.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.4.m4.1b"><apply id="S3.p1.4.m4.1.2.cmml" xref="S3.p1.4.m4.1.2"><times id="S3.p1.4.m4.1.2.1.cmml" xref="S3.p1.4.m4.1.2.1"></times><apply id="S3.p1.4.m4.1.2.2.cmml" xref="S3.p1.4.m4.1.2.2"><csymbol cd="ambiguous" id="S3.p1.4.m4.1.2.2.1.cmml" xref="S3.p1.4.m4.1.2.2">subscript</csymbol><ci id="S3.p1.4.m4.1.2.2.2a.cmml" xref="S3.p1.4.m4.1.2.2.2"><mtext id="S3.p1.4.m4.1.2.2.2.cmml" xref="S3.p1.4.m4.1.2.2.2">Alg</mtext></ci><ci id="S3.p1.4.m4.1.2.2.3.cmml" xref="S3.p1.4.m4.1.2.2.3">𝜃</ci></apply><ci id="S3.p1.4.m4.1.1.cmml" xref="S3.p1.4.m4.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.4.m4.1c">\text{Alg}_{\theta}(\cdot)</annotation></semantics></math> which can generalize <math id="S3.p1.5.m5.1" class="ltx_Math" alttext="\text{Alg}_{\theta}({\bm{x}}^{\prime})" display="inline"><semantics id="S3.p1.5.m5.1a"><mrow id="S3.p1.5.m5.1.1" xref="S3.p1.5.m5.1.1.cmml"><msub id="S3.p1.5.m5.1.1.3" xref="S3.p1.5.m5.1.1.3.cmml"><mtext id="S3.p1.5.m5.1.1.3.2" xref="S3.p1.5.m5.1.1.3.2a.cmml">Alg</mtext><mi id="S3.p1.5.m5.1.1.3.3" xref="S3.p1.5.m5.1.1.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.p1.5.m5.1.1.2" xref="S3.p1.5.m5.1.1.2.cmml">​</mo><mrow id="S3.p1.5.m5.1.1.1.1" xref="S3.p1.5.m5.1.1.1.1.1.cmml"><mo stretchy="false" id="S3.p1.5.m5.1.1.1.1.2" xref="S3.p1.5.m5.1.1.1.1.1.cmml">(</mo><msup id="S3.p1.5.m5.1.1.1.1.1" xref="S3.p1.5.m5.1.1.1.1.1.cmml"><mi id="S3.p1.5.m5.1.1.1.1.1.2" xref="S3.p1.5.m5.1.1.1.1.1.2.cmml">𝒙</mi><mo id="S3.p1.5.m5.1.1.1.1.1.3" xref="S3.p1.5.m5.1.1.1.1.1.3.cmml">′</mo></msup><mo stretchy="false" id="S3.p1.5.m5.1.1.1.1.3" xref="S3.p1.5.m5.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.5.m5.1b"><apply id="S3.p1.5.m5.1.1.cmml" xref="S3.p1.5.m5.1.1"><times id="S3.p1.5.m5.1.1.2.cmml" xref="S3.p1.5.m5.1.1.2"></times><apply id="S3.p1.5.m5.1.1.3.cmml" xref="S3.p1.5.m5.1.1.3"><csymbol cd="ambiguous" id="S3.p1.5.m5.1.1.3.1.cmml" xref="S3.p1.5.m5.1.1.3">subscript</csymbol><ci id="S3.p1.5.m5.1.1.3.2a.cmml" xref="S3.p1.5.m5.1.1.3.2"><mtext id="S3.p1.5.m5.1.1.3.2.cmml" xref="S3.p1.5.m5.1.1.3.2">Alg</mtext></ci><ci id="S3.p1.5.m5.1.1.3.3.cmml" xref="S3.p1.5.m5.1.1.3.3">𝜃</ci></apply><apply id="S3.p1.5.m5.1.1.1.1.1.cmml" xref="S3.p1.5.m5.1.1.1.1"><csymbol cd="ambiguous" id="S3.p1.5.m5.1.1.1.1.1.1.cmml" xref="S3.p1.5.m5.1.1.1.1">superscript</csymbol><ci id="S3.p1.5.m5.1.1.1.1.1.2.cmml" xref="S3.p1.5.m5.1.1.1.1.1.2">𝒙</ci><ci id="S3.p1.5.m5.1.1.1.1.1.3.cmml" xref="S3.p1.5.m5.1.1.1.1.1.3">′</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.5.m5.1c">\text{Alg}_{\theta}({\bm{x}}^{\prime})</annotation></semantics></math> to a test input <math id="S3.p1.6.m6.1" class="ltx_Math" alttext="{\bm{x}}^{\prime}\in\mathbb{R}^{O^{\prime}}" display="inline"><semantics id="S3.p1.6.m6.1a"><mrow id="S3.p1.6.m6.1.1" xref="S3.p1.6.m6.1.1.cmml"><msup id="S3.p1.6.m6.1.1.2" xref="S3.p1.6.m6.1.1.2.cmml"><mi id="S3.p1.6.m6.1.1.2.2" xref="S3.p1.6.m6.1.1.2.2.cmml">𝒙</mi><mo id="S3.p1.6.m6.1.1.2.3" xref="S3.p1.6.m6.1.1.2.3.cmml">′</mo></msup><mo id="S3.p1.6.m6.1.1.1" xref="S3.p1.6.m6.1.1.1.cmml">∈</mo><msup id="S3.p1.6.m6.1.1.3" xref="S3.p1.6.m6.1.1.3.cmml"><mi id="S3.p1.6.m6.1.1.3.2" xref="S3.p1.6.m6.1.1.3.2.cmml">ℝ</mi><msup id="S3.p1.6.m6.1.1.3.3" xref="S3.p1.6.m6.1.1.3.3.cmml"><mi id="S3.p1.6.m6.1.1.3.3.2" xref="S3.p1.6.m6.1.1.3.3.2.cmml">O</mi><mo id="S3.p1.6.m6.1.1.3.3.3" xref="S3.p1.6.m6.1.1.3.3.3.cmml">′</mo></msup></msup></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.6.m6.1b"><apply id="S3.p1.6.m6.1.1.cmml" xref="S3.p1.6.m6.1.1"><in id="S3.p1.6.m6.1.1.1.cmml" xref="S3.p1.6.m6.1.1.1"></in><apply id="S3.p1.6.m6.1.1.2.cmml" xref="S3.p1.6.m6.1.1.2"><csymbol cd="ambiguous" id="S3.p1.6.m6.1.1.2.1.cmml" xref="S3.p1.6.m6.1.1.2">superscript</csymbol><ci id="S3.p1.6.m6.1.1.2.2.cmml" xref="S3.p1.6.m6.1.1.2.2">𝒙</ci><ci id="S3.p1.6.m6.1.1.2.3.cmml" xref="S3.p1.6.m6.1.1.2.3">′</ci></apply><apply id="S3.p1.6.m6.1.1.3.cmml" xref="S3.p1.6.m6.1.1.3"><csymbol cd="ambiguous" id="S3.p1.6.m6.1.1.3.1.cmml" xref="S3.p1.6.m6.1.1.3">superscript</csymbol><ci id="S3.p1.6.m6.1.1.3.2.cmml" xref="S3.p1.6.m6.1.1.3.2">ℝ</ci><apply id="S3.p1.6.m6.1.1.3.3.cmml" xref="S3.p1.6.m6.1.1.3.3"><csymbol cd="ambiguous" id="S3.p1.6.m6.1.1.3.3.1.cmml" xref="S3.p1.6.m6.1.1.3.3">superscript</csymbol><ci id="S3.p1.6.m6.1.1.3.3.2.cmml" xref="S3.p1.6.m6.1.1.3.3.2">𝑂</ci><ci id="S3.p1.6.m6.1.1.3.3.3.cmml" xref="S3.p1.6.m6.1.1.3.3.3">′</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.6.m6.1c">{\bm{x}}^{\prime}\in\mathbb{R}^{O^{\prime}}</annotation></semantics></math> , where <math id="S3.p1.7.m7.1" class="ltx_Math" alttext="{\bm{x}}^{\prime}" display="inline"><semantics id="S3.p1.7.m7.1a"><msup id="S3.p1.7.m7.1.1" xref="S3.p1.7.m7.1.1.cmml"><mi id="S3.p1.7.m7.1.1.2" xref="S3.p1.7.m7.1.1.2.cmml">𝒙</mi><mo id="S3.p1.7.m7.1.1.3" xref="S3.p1.7.m7.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S3.p1.7.m7.1b"><apply id="S3.p1.7.m7.1.1.cmml" xref="S3.p1.7.m7.1.1"><csymbol cd="ambiguous" id="S3.p1.7.m7.1.1.1.cmml" xref="S3.p1.7.m7.1.1">superscript</csymbol><ci id="S3.p1.7.m7.1.1.2.cmml" xref="S3.p1.7.m7.1.1.2">𝒙</ci><ci id="S3.p1.7.m7.1.1.3.cmml" xref="S3.p1.7.m7.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.7.m7.1c">{\bm{x}}^{\prime}</annotation></semantics></math> can be significantly larger and more challenging than the training data <math id="S3.p1.8.m8.1" class="ltx_Math" alttext="{\bm{x}}\in X" display="inline"><semantics id="S3.p1.8.m8.1a"><mrow id="S3.p1.8.m8.1.1" xref="S3.p1.8.m8.1.1.cmml"><mi id="S3.p1.8.m8.1.1.2" xref="S3.p1.8.m8.1.1.2.cmml">𝒙</mi><mo id="S3.p1.8.m8.1.1.1" xref="S3.p1.8.m8.1.1.1.cmml">∈</mo><mi id="S3.p1.8.m8.1.1.3" xref="S3.p1.8.m8.1.1.3.cmml">X</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.8.m8.1b"><apply id="S3.p1.8.m8.1.1.cmml" xref="S3.p1.8.m8.1.1"><in id="S3.p1.8.m8.1.1.1.cmml" xref="S3.p1.8.m8.1.1.1"></in><ci id="S3.p1.8.m8.1.1.2.cmml" xref="S3.p1.8.m8.1.1.2">𝒙</ci><ci id="S3.p1.8.m8.1.1.3.cmml" xref="S3.p1.8.m8.1.1.3">𝑋</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.8.m8.1c">{\bm{x}}\in X</annotation></semantics></math>.
We present our approach, IREM, to adaptive algorithmic computation in Section <a href="#S3.SS1" title="3.1 Iterative Reasoning as Energy Minimization ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.1</span></a>, and discuss how to learn such an energy landscape in Section <a href="#S3.SS2" title="3.2 Learning Energy Landscapes for Reasoning ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.2</span></a>. We further provide analysis on why such a framework is favorable in Section <a href="#S3.SS3" title="3.3 Analysis ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.3</span></a>.</p>
</div>
<section id="S3.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.1 </span>Iterative Reasoning as Energy Minimization</h3>

<div id="S3.SS1.p1" class="ltx_para">
<p id="S3.SS1.p1.2" class="ltx_p">Iterative reasoning is typically formulated as the repeated application of a neural network operator <math id="S3.SS1.p1.1.m1.2" class="ltx_Math" alttext="f_{\theta}({\bm{x}},{\bm{y}}):\mathbb{R}^{O}\times\mathbb{R}^{M}\rightarrow\mathbb{R}^{M}" display="inline"><semantics id="S3.SS1.p1.1.m1.2a"><mrow id="S3.SS1.p1.1.m1.2.3" xref="S3.SS1.p1.1.m1.2.3.cmml"><mrow id="S3.SS1.p1.1.m1.2.3.2" xref="S3.SS1.p1.1.m1.2.3.2.cmml"><msub id="S3.SS1.p1.1.m1.2.3.2.2" xref="S3.SS1.p1.1.m1.2.3.2.2.cmml"><mi id="S3.SS1.p1.1.m1.2.3.2.2.2" xref="S3.SS1.p1.1.m1.2.3.2.2.2.cmml">f</mi><mi id="S3.SS1.p1.1.m1.2.3.2.2.3" xref="S3.SS1.p1.1.m1.2.3.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS1.p1.1.m1.2.3.2.1" xref="S3.SS1.p1.1.m1.2.3.2.1.cmml">​</mo><mrow id="S3.SS1.p1.1.m1.2.3.2.3.2" xref="S3.SS1.p1.1.m1.2.3.2.3.1.cmml"><mo stretchy="false" id="S3.SS1.p1.1.m1.2.3.2.3.2.1" xref="S3.SS1.p1.1.m1.2.3.2.3.1.cmml">(</mo><mi id="S3.SS1.p1.1.m1.1.1" xref="S3.SS1.p1.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS1.p1.1.m1.2.3.2.3.2.2" xref="S3.SS1.p1.1.m1.2.3.2.3.1.cmml">,</mo><mi id="S3.SS1.p1.1.m1.2.2" xref="S3.SS1.p1.1.m1.2.2.cmml">𝒚</mi><mo rspace="0.278em" stretchy="false" id="S3.SS1.p1.1.m1.2.3.2.3.2.3" xref="S3.SS1.p1.1.m1.2.3.2.3.1.cmml">)</mo></mrow></mrow><mo rspace="0.278em" id="S3.SS1.p1.1.m1.2.3.1" xref="S3.SS1.p1.1.m1.2.3.1.cmml">:</mo><mrow id="S3.SS1.p1.1.m1.2.3.3" xref="S3.SS1.p1.1.m1.2.3.3.cmml"><mrow id="S3.SS1.p1.1.m1.2.3.3.2" xref="S3.SS1.p1.1.m1.2.3.3.2.cmml"><msup id="S3.SS1.p1.1.m1.2.3.3.2.2" xref="S3.SS1.p1.1.m1.2.3.3.2.2.cmml"><mi id="S3.SS1.p1.1.m1.2.3.3.2.2.2" xref="S3.SS1.p1.1.m1.2.3.3.2.2.2.cmml">ℝ</mi><mi id="S3.SS1.p1.1.m1.2.3.3.2.2.3" xref="S3.SS1.p1.1.m1.2.3.3.2.2.3.cmml">O</mi></msup><mo lspace="0.222em" rspace="0.222em" id="S3.SS1.p1.1.m1.2.3.3.2.1" xref="S3.SS1.p1.1.m1.2.3.3.2.1.cmml">×</mo><msup id="S3.SS1.p1.1.m1.2.3.3.2.3" xref="S3.SS1.p1.1.m1.2.3.3.2.3.cmml"><mi id="S3.SS1.p1.1.m1.2.3.3.2.3.2" xref="S3.SS1.p1.1.m1.2.3.3.2.3.2.cmml">ℝ</mi><mi id="S3.SS1.p1.1.m1.2.3.3.2.3.3" xref="S3.SS1.p1.1.m1.2.3.3.2.3.3.cmml">M</mi></msup></mrow><mo stretchy="false" id="S3.SS1.p1.1.m1.2.3.3.1" xref="S3.SS1.p1.1.m1.2.3.3.1.cmml">→</mo><msup id="S3.SS1.p1.1.m1.2.3.3.3" xref="S3.SS1.p1.1.m1.2.3.3.3.cmml"><mi id="S3.SS1.p1.1.m1.2.3.3.3.2" xref="S3.SS1.p1.1.m1.2.3.3.3.2.cmml">ℝ</mi><mi id="S3.SS1.p1.1.m1.2.3.3.3.3" xref="S3.SS1.p1.1.m1.2.3.3.3.3.cmml">M</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p1.1.m1.2b"><apply id="S3.SS1.p1.1.m1.2.3.cmml" xref="S3.SS1.p1.1.m1.2.3"><ci id="S3.SS1.p1.1.m1.2.3.1.cmml" xref="S3.SS1.p1.1.m1.2.3.1">:</ci><apply id="S3.SS1.p1.1.m1.2.3.2.cmml" xref="S3.SS1.p1.1.m1.2.3.2"><times id="S3.SS1.p1.1.m1.2.3.2.1.cmml" xref="S3.SS1.p1.1.m1.2.3.2.1"></times><apply id="S3.SS1.p1.1.m1.2.3.2.2.cmml" xref="S3.SS1.p1.1.m1.2.3.2.2"><csymbol cd="ambiguous" id="S3.SS1.p1.1.m1.2.3.2.2.1.cmml" xref="S3.SS1.p1.1.m1.2.3.2.2">subscript</csymbol><ci id="S3.SS1.p1.1.m1.2.3.2.2.2.cmml" xref="S3.SS1.p1.1.m1.2.3.2.2.2">𝑓</ci><ci id="S3.SS1.p1.1.m1.2.3.2.2.3.cmml" xref="S3.SS1.p1.1.m1.2.3.2.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS1.p1.1.m1.2.3.2.3.1.cmml" xref="S3.SS1.p1.1.m1.2.3.2.3.2"><ci id="S3.SS1.p1.1.m1.1.1.cmml" xref="S3.SS1.p1.1.m1.1.1">𝒙</ci><ci id="S3.SS1.p1.1.m1.2.2.cmml" xref="S3.SS1.p1.1.m1.2.2">𝒚</ci></interval></apply><apply id="S3.SS1.p1.1.m1.2.3.3.cmml" xref="S3.SS1.p1.1.m1.2.3.3"><ci id="S3.SS1.p1.1.m1.2.3.3.1.cmml" xref="S3.SS1.p1.1.m1.2.3.3.1">→</ci><apply id="S3.SS1.p1.1.m1.2.3.3.2.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2"><times id="S3.SS1.p1.1.m1.2.3.3.2.1.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.1"></times><apply id="S3.SS1.p1.1.m1.2.3.3.2.2.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.2"><csymbol cd="ambiguous" id="S3.SS1.p1.1.m1.2.3.3.2.2.1.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.2">superscript</csymbol><ci id="S3.SS1.p1.1.m1.2.3.3.2.2.2.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.2.2">ℝ</ci><ci id="S3.SS1.p1.1.m1.2.3.3.2.2.3.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.2.3">𝑂</ci></apply><apply id="S3.SS1.p1.1.m1.2.3.3.2.3.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.3"><csymbol cd="ambiguous" id="S3.SS1.p1.1.m1.2.3.3.2.3.1.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.3">superscript</csymbol><ci id="S3.SS1.p1.1.m1.2.3.3.2.3.2.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.3.2">ℝ</ci><ci id="S3.SS1.p1.1.m1.2.3.3.2.3.3.cmml" xref="S3.SS1.p1.1.m1.2.3.3.2.3.3">𝑀</ci></apply></apply><apply id="S3.SS1.p1.1.m1.2.3.3.3.cmml" xref="S3.SS1.p1.1.m1.2.3.3.3"><csymbol cd="ambiguous" id="S3.SS1.p1.1.m1.2.3.3.3.1.cmml" xref="S3.SS1.p1.1.m1.2.3.3.3">superscript</csymbol><ci id="S3.SS1.p1.1.m1.2.3.3.3.2.cmml" xref="S3.SS1.p1.1.m1.2.3.3.3.2">ℝ</ci><ci id="S3.SS1.p1.1.m1.2.3.3.3.3.cmml" xref="S3.SS1.p1.1.m1.2.3.3.3.3">𝑀</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p1.1.m1.2c">f_{\theta}({\bm{x}},{\bm{y}}):\mathbb{R}^{O}\times\mathbb{R}^{M}\rightarrow\mathbb{R}^{M}</annotation></semantics></math>, to generate partial solutions <math id="S3.SS1.p1.2.m2.1" class="ltx_Math" alttext="{\bm{y}}^{t}" display="inline"><semantics id="S3.SS1.p1.2.m2.1a"><msup id="S3.SS1.p1.2.m2.1.1" xref="S3.SS1.p1.2.m2.1.1.cmml"><mi id="S3.SS1.p1.2.m2.1.1.2" xref="S3.SS1.p1.2.m2.1.1.2.cmml">𝒚</mi><mi id="S3.SS1.p1.2.m2.1.1.3" xref="S3.SS1.p1.2.m2.1.1.3.cmml">t</mi></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p1.2.m2.1b"><apply id="S3.SS1.p1.2.m2.1.1.cmml" xref="S3.SS1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p1.2.m2.1.1.1.cmml" xref="S3.SS1.p1.2.m2.1.1">superscript</csymbol><ci id="S3.SS1.p1.2.m2.1.1.2.cmml" xref="S3.SS1.p1.2.m2.1.1.2">𝒚</ci><ci id="S3.SS1.p1.2.m2.1.1.3.cmml" xref="S3.SS1.p1.2.m2.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p1.2.m2.1c">{\bm{y}}^{t}</annotation></semantics></math></p>
<table id="S3.E1" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E1.m1.2" class="ltx_Math" alttext="{\bm{y}}^{t}=f_{\theta}({\bm{x}},{\bm{y}}^{t-1})," display="block"><semantics id="S3.E1.m1.2a"><mrow id="S3.E1.m1.2.2.1" xref="S3.E1.m1.2.2.1.1.cmml"><mrow id="S3.E1.m1.2.2.1.1" xref="S3.E1.m1.2.2.1.1.cmml"><msup id="S3.E1.m1.2.2.1.1.3" xref="S3.E1.m1.2.2.1.1.3.cmml"><mi id="S3.E1.m1.2.2.1.1.3.2" xref="S3.E1.m1.2.2.1.1.3.2.cmml">𝒚</mi><mi id="S3.E1.m1.2.2.1.1.3.3" xref="S3.E1.m1.2.2.1.1.3.3.cmml">t</mi></msup><mo id="S3.E1.m1.2.2.1.1.2" xref="S3.E1.m1.2.2.1.1.2.cmml">=</mo><mrow id="S3.E1.m1.2.2.1.1.1" xref="S3.E1.m1.2.2.1.1.1.cmml"><msub id="S3.E1.m1.2.2.1.1.1.3" xref="S3.E1.m1.2.2.1.1.1.3.cmml"><mi id="S3.E1.m1.2.2.1.1.1.3.2" xref="S3.E1.m1.2.2.1.1.1.3.2.cmml">f</mi><mi id="S3.E1.m1.2.2.1.1.1.3.3" xref="S3.E1.m1.2.2.1.1.1.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.E1.m1.2.2.1.1.1.2" xref="S3.E1.m1.2.2.1.1.1.2.cmml">​</mo><mrow id="S3.E1.m1.2.2.1.1.1.1.1" xref="S3.E1.m1.2.2.1.1.1.1.2.cmml"><mo stretchy="false" id="S3.E1.m1.2.2.1.1.1.1.1.2" xref="S3.E1.m1.2.2.1.1.1.1.2.cmml">(</mo><mi id="S3.E1.m1.1.1" xref="S3.E1.m1.1.1.cmml">𝒙</mi><mo id="S3.E1.m1.2.2.1.1.1.1.1.3" xref="S3.E1.m1.2.2.1.1.1.1.2.cmml">,</mo><msup id="S3.E1.m1.2.2.1.1.1.1.1.1" xref="S3.E1.m1.2.2.1.1.1.1.1.1.cmml"><mi id="S3.E1.m1.2.2.1.1.1.1.1.1.2" xref="S3.E1.m1.2.2.1.1.1.1.1.1.2.cmml">𝒚</mi><mrow id="S3.E1.m1.2.2.1.1.1.1.1.1.3" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.cmml"><mi id="S3.E1.m1.2.2.1.1.1.1.1.1.3.2" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.2.cmml">t</mi><mo id="S3.E1.m1.2.2.1.1.1.1.1.1.3.1" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.1.cmml">−</mo><mn id="S3.E1.m1.2.2.1.1.1.1.1.1.3.3" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.3.cmml">1</mn></mrow></msup><mo stretchy="false" id="S3.E1.m1.2.2.1.1.1.1.1.4" xref="S3.E1.m1.2.2.1.1.1.1.2.cmml">)</mo></mrow></mrow></mrow><mo id="S3.E1.m1.2.2.1.2" xref="S3.E1.m1.2.2.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.E1.m1.2b"><apply id="S3.E1.m1.2.2.1.1.cmml" xref="S3.E1.m1.2.2.1"><eq id="S3.E1.m1.2.2.1.1.2.cmml" xref="S3.E1.m1.2.2.1.1.2"></eq><apply id="S3.E1.m1.2.2.1.1.3.cmml" xref="S3.E1.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S3.E1.m1.2.2.1.1.3.1.cmml" xref="S3.E1.m1.2.2.1.1.3">superscript</csymbol><ci id="S3.E1.m1.2.2.1.1.3.2.cmml" xref="S3.E1.m1.2.2.1.1.3.2">𝒚</ci><ci id="S3.E1.m1.2.2.1.1.3.3.cmml" xref="S3.E1.m1.2.2.1.1.3.3">𝑡</ci></apply><apply id="S3.E1.m1.2.2.1.1.1.cmml" xref="S3.E1.m1.2.2.1.1.1"><times id="S3.E1.m1.2.2.1.1.1.2.cmml" xref="S3.E1.m1.2.2.1.1.1.2"></times><apply id="S3.E1.m1.2.2.1.1.1.3.cmml" xref="S3.E1.m1.2.2.1.1.1.3"><csymbol cd="ambiguous" id="S3.E1.m1.2.2.1.1.1.3.1.cmml" xref="S3.E1.m1.2.2.1.1.1.3">subscript</csymbol><ci id="S3.E1.m1.2.2.1.1.1.3.2.cmml" xref="S3.E1.m1.2.2.1.1.1.3.2">𝑓</ci><ci id="S3.E1.m1.2.2.1.1.1.3.3.cmml" xref="S3.E1.m1.2.2.1.1.1.3.3">𝜃</ci></apply><interval closure="open" id="S3.E1.m1.2.2.1.1.1.1.2.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1"><ci id="S3.E1.m1.1.1.cmml" xref="S3.E1.m1.1.1">𝒙</ci><apply id="S3.E1.m1.2.2.1.1.1.1.1.1.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E1.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1">superscript</csymbol><ci id="S3.E1.m1.2.2.1.1.1.1.1.1.2.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1.2">𝒚</ci><apply id="S3.E1.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3"><minus id="S3.E1.m1.2.2.1.1.1.1.1.1.3.1.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.1"></minus><ci id="S3.E1.m1.2.2.1.1.1.1.1.1.3.2.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.2">𝑡</ci><cn type="integer" id="S3.E1.m1.2.2.1.1.1.1.1.1.3.3.cmml" xref="S3.E1.m1.2.2.1.1.1.1.1.1.3.3">1</cn></apply></apply></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E1.m1.2c">{\bm{y}}^{t}=f_{\theta}({\bm{x}},{\bm{y}}^{t-1}),</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(1)</span></td>
</tr></tbody>
</table>
<p id="S3.SS1.p1.5" class="ltx_p">where the final prediction <math id="S3.SS1.p1.3.m1.1" class="ltx_Math" alttext="{\bm{y}}={\bm{y}}^{T}" display="inline"><semantics id="S3.SS1.p1.3.m1.1a"><mrow id="S3.SS1.p1.3.m1.1.1" xref="S3.SS1.p1.3.m1.1.1.cmml"><mi id="S3.SS1.p1.3.m1.1.1.2" xref="S3.SS1.p1.3.m1.1.1.2.cmml">𝒚</mi><mo id="S3.SS1.p1.3.m1.1.1.1" xref="S3.SS1.p1.3.m1.1.1.1.cmml">=</mo><msup id="S3.SS1.p1.3.m1.1.1.3" xref="S3.SS1.p1.3.m1.1.1.3.cmml"><mi id="S3.SS1.p1.3.m1.1.1.3.2" xref="S3.SS1.p1.3.m1.1.1.3.2.cmml">𝒚</mi><mi id="S3.SS1.p1.3.m1.1.1.3.3" xref="S3.SS1.p1.3.m1.1.1.3.3.cmml">T</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p1.3.m1.1b"><apply id="S3.SS1.p1.3.m1.1.1.cmml" xref="S3.SS1.p1.3.m1.1.1"><eq id="S3.SS1.p1.3.m1.1.1.1.cmml" xref="S3.SS1.p1.3.m1.1.1.1"></eq><ci id="S3.SS1.p1.3.m1.1.1.2.cmml" xref="S3.SS1.p1.3.m1.1.1.2">𝒚</ci><apply id="S3.SS1.p1.3.m1.1.1.3.cmml" xref="S3.SS1.p1.3.m1.1.1.3"><csymbol cd="ambiguous" id="S3.SS1.p1.3.m1.1.1.3.1.cmml" xref="S3.SS1.p1.3.m1.1.1.3">superscript</csymbol><ci id="S3.SS1.p1.3.m1.1.1.3.2.cmml" xref="S3.SS1.p1.3.m1.1.1.3.2">𝒚</ci><ci id="S3.SS1.p1.3.m1.1.1.3.3.cmml" xref="S3.SS1.p1.3.m1.1.1.3.3">𝑇</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p1.3.m1.1c">{\bm{y}}={\bm{y}}^{T}</annotation></semantics></math> is output after <math id="S3.SS1.p1.4.m2.1" class="ltx_Math" alttext="T" display="inline"><semantics id="S3.SS1.p1.4.m2.1a"><mi id="S3.SS1.p1.4.m2.1.1" xref="S3.SS1.p1.4.m2.1.1.cmml">T</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p1.4.m2.1b"><ci id="S3.SS1.p1.4.m2.1.1.cmml" xref="S3.SS1.p1.4.m2.1.1">𝑇</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p1.4.m2.1c">T</annotation></semantics></math> successive applications of <math id="S3.SS1.p1.5.m3.1" class="ltx_Math" alttext="f_{\theta}" display="inline"><semantics id="S3.SS1.p1.5.m3.1a"><msub id="S3.SS1.p1.5.m3.1.1" xref="S3.SS1.p1.5.m3.1.1.cmml"><mi id="S3.SS1.p1.5.m3.1.1.2" xref="S3.SS1.p1.5.m3.1.1.2.cmml">f</mi><mi id="S3.SS1.p1.5.m3.1.1.3" xref="S3.SS1.p1.5.m3.1.1.3.cmml">θ</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p1.5.m3.1b"><apply id="S3.SS1.p1.5.m3.1.1.cmml" xref="S3.SS1.p1.5.m3.1.1"><csymbol cd="ambiguous" id="S3.SS1.p1.5.m3.1.1.1.cmml" xref="S3.SS1.p1.5.m3.1.1">subscript</csymbol><ci id="S3.SS1.p1.5.m3.1.1.2.cmml" xref="S3.SS1.p1.5.m3.1.1.2">𝑓</ci><ci id="S3.SS1.p1.5.m3.1.1.3.cmml" xref="S3.SS1.p1.5.m3.1.1.3">𝜃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p1.5.m3.1c">f_{\theta}</annotation></semantics></math>. The termination of iterative computation is often difficult to specify, with methods typically utilizing a halting policy that is difficult to train <cite class="ltx_cite ltx_citemacro_citep">(Graves, <a href="#bib.bib20" title="" class="ltx_ref">2016</a>; Chung et al., <a href="#bib.bib11" title="" class="ltx_ref">2016</a>)</cite>.</p>
</div>
<div id="S3.SS1.p2" class="ltx_para">
<p id="S3.SS1.p2.1" class="ltx_p">In this work, we propose an alternative approach towards iterative computation with a natural termination criteria, IREM, which represents iterative reasoning as an optimization process over an Energy-Based Model (EBM) <math id="S3.SS1.p2.1.m1.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}}):\mathbb{R}^{O}\times\mathbb{R}^{M}\rightarrow\mathbb{R}" display="inline"><semantics id="S3.SS1.p2.1.m1.2a"><mrow id="S3.SS1.p2.1.m1.2.3" xref="S3.SS1.p2.1.m1.2.3.cmml"><mrow id="S3.SS1.p2.1.m1.2.3.2" xref="S3.SS1.p2.1.m1.2.3.2.cmml"><msub id="S3.SS1.p2.1.m1.2.3.2.2" xref="S3.SS1.p2.1.m1.2.3.2.2.cmml"><mi id="S3.SS1.p2.1.m1.2.3.2.2.2" xref="S3.SS1.p2.1.m1.2.3.2.2.2.cmml">E</mi><mi id="S3.SS1.p2.1.m1.2.3.2.2.3" xref="S3.SS1.p2.1.m1.2.3.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS1.p2.1.m1.2.3.2.1" xref="S3.SS1.p2.1.m1.2.3.2.1.cmml">​</mo><mrow id="S3.SS1.p2.1.m1.2.3.2.3.2" xref="S3.SS1.p2.1.m1.2.3.2.3.1.cmml"><mo stretchy="false" id="S3.SS1.p2.1.m1.2.3.2.3.2.1" xref="S3.SS1.p2.1.m1.2.3.2.3.1.cmml">(</mo><mi id="S3.SS1.p2.1.m1.1.1" xref="S3.SS1.p2.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS1.p2.1.m1.2.3.2.3.2.2" xref="S3.SS1.p2.1.m1.2.3.2.3.1.cmml">,</mo><mi id="S3.SS1.p2.1.m1.2.2" xref="S3.SS1.p2.1.m1.2.2.cmml">𝒚</mi><mo rspace="0.278em" stretchy="false" id="S3.SS1.p2.1.m1.2.3.2.3.2.3" xref="S3.SS1.p2.1.m1.2.3.2.3.1.cmml">)</mo></mrow></mrow><mo rspace="0.278em" id="S3.SS1.p2.1.m1.2.3.1" xref="S3.SS1.p2.1.m1.2.3.1.cmml">:</mo><mrow id="S3.SS1.p2.1.m1.2.3.3" xref="S3.SS1.p2.1.m1.2.3.3.cmml"><mrow id="S3.SS1.p2.1.m1.2.3.3.2" xref="S3.SS1.p2.1.m1.2.3.3.2.cmml"><msup id="S3.SS1.p2.1.m1.2.3.3.2.2" xref="S3.SS1.p2.1.m1.2.3.3.2.2.cmml"><mi id="S3.SS1.p2.1.m1.2.3.3.2.2.2" xref="S3.SS1.p2.1.m1.2.3.3.2.2.2.cmml">ℝ</mi><mi id="S3.SS1.p2.1.m1.2.3.3.2.2.3" xref="S3.SS1.p2.1.m1.2.3.3.2.2.3.cmml">O</mi></msup><mo lspace="0.222em" rspace="0.222em" id="S3.SS1.p2.1.m1.2.3.3.2.1" xref="S3.SS1.p2.1.m1.2.3.3.2.1.cmml">×</mo><msup id="S3.SS1.p2.1.m1.2.3.3.2.3" xref="S3.SS1.p2.1.m1.2.3.3.2.3.cmml"><mi id="S3.SS1.p2.1.m1.2.3.3.2.3.2" xref="S3.SS1.p2.1.m1.2.3.3.2.3.2.cmml">ℝ</mi><mi id="S3.SS1.p2.1.m1.2.3.3.2.3.3" xref="S3.SS1.p2.1.m1.2.3.3.2.3.3.cmml">M</mi></msup></mrow><mo stretchy="false" id="S3.SS1.p2.1.m1.2.3.3.1" xref="S3.SS1.p2.1.m1.2.3.3.1.cmml">→</mo><mi id="S3.SS1.p2.1.m1.2.3.3.3" xref="S3.SS1.p2.1.m1.2.3.3.3.cmml">ℝ</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.1.m1.2b"><apply id="S3.SS1.p2.1.m1.2.3.cmml" xref="S3.SS1.p2.1.m1.2.3"><ci id="S3.SS1.p2.1.m1.2.3.1.cmml" xref="S3.SS1.p2.1.m1.2.3.1">:</ci><apply id="S3.SS1.p2.1.m1.2.3.2.cmml" xref="S3.SS1.p2.1.m1.2.3.2"><times id="S3.SS1.p2.1.m1.2.3.2.1.cmml" xref="S3.SS1.p2.1.m1.2.3.2.1"></times><apply id="S3.SS1.p2.1.m1.2.3.2.2.cmml" xref="S3.SS1.p2.1.m1.2.3.2.2"><csymbol cd="ambiguous" id="S3.SS1.p2.1.m1.2.3.2.2.1.cmml" xref="S3.SS1.p2.1.m1.2.3.2.2">subscript</csymbol><ci id="S3.SS1.p2.1.m1.2.3.2.2.2.cmml" xref="S3.SS1.p2.1.m1.2.3.2.2.2">𝐸</ci><ci id="S3.SS1.p2.1.m1.2.3.2.2.3.cmml" xref="S3.SS1.p2.1.m1.2.3.2.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS1.p2.1.m1.2.3.2.3.1.cmml" xref="S3.SS1.p2.1.m1.2.3.2.3.2"><ci id="S3.SS1.p2.1.m1.1.1.cmml" xref="S3.SS1.p2.1.m1.1.1">𝒙</ci><ci id="S3.SS1.p2.1.m1.2.2.cmml" xref="S3.SS1.p2.1.m1.2.2">𝒚</ci></interval></apply><apply id="S3.SS1.p2.1.m1.2.3.3.cmml" xref="S3.SS1.p2.1.m1.2.3.3"><ci id="S3.SS1.p2.1.m1.2.3.3.1.cmml" xref="S3.SS1.p2.1.m1.2.3.3.1">→</ci><apply id="S3.SS1.p2.1.m1.2.3.3.2.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2"><times id="S3.SS1.p2.1.m1.2.3.3.2.1.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.1"></times><apply id="S3.SS1.p2.1.m1.2.3.3.2.2.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.2"><csymbol cd="ambiguous" id="S3.SS1.p2.1.m1.2.3.3.2.2.1.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.2">superscript</csymbol><ci id="S3.SS1.p2.1.m1.2.3.3.2.2.2.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.2.2">ℝ</ci><ci id="S3.SS1.p2.1.m1.2.3.3.2.2.3.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.2.3">𝑂</ci></apply><apply id="S3.SS1.p2.1.m1.2.3.3.2.3.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.3"><csymbol cd="ambiguous" id="S3.SS1.p2.1.m1.2.3.3.2.3.1.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.3">superscript</csymbol><ci id="S3.SS1.p2.1.m1.2.3.3.2.3.2.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.3.2">ℝ</ci><ci id="S3.SS1.p2.1.m1.2.3.3.2.3.3.cmml" xref="S3.SS1.p2.1.m1.2.3.3.2.3.3">𝑀</ci></apply></apply><ci id="S3.SS1.p2.1.m1.2.3.3.3.cmml" xref="S3.SS1.p2.1.m1.2.3.3.3">ℝ</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.1.m1.2c">E_{\theta}({\bm{x}},{\bm{y}}):\mathbb{R}^{O}\times\mathbb{R}^{M}\rightarrow\mathbb{R}</annotation></semantics></math>:</p>
<table id="S3.E2" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E2.m1.3" class="ltx_Math" alttext="{\bm{y}}=\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}},{\bm{y}})." display="block"><semantics id="S3.E2.m1.3a"><mrow id="S3.E2.m1.3.3.1" xref="S3.E2.m1.3.3.1.1.cmml"><mrow id="S3.E2.m1.3.3.1.1" xref="S3.E2.m1.3.3.1.1.cmml"><mi id="S3.E2.m1.3.3.1.1.2" xref="S3.E2.m1.3.3.1.1.2.cmml">𝒚</mi><mo id="S3.E2.m1.3.3.1.1.1" xref="S3.E2.m1.3.3.1.1.1.cmml">=</mo><mrow id="S3.E2.m1.3.3.1.1.3" xref="S3.E2.m1.3.3.1.1.3.cmml"><mrow id="S3.E2.m1.3.3.1.1.3.2" xref="S3.E2.m1.3.3.1.1.3.2.cmml"><munder id="S3.E2.m1.3.3.1.1.3.2.1" xref="S3.E2.m1.3.3.1.1.3.2.1.cmml"><mrow id="S3.E2.m1.3.3.1.1.3.2.1.2" xref="S3.E2.m1.3.3.1.1.3.2.1.2.cmml"><mi id="S3.E2.m1.3.3.1.1.3.2.1.2.2" xref="S3.E2.m1.3.3.1.1.3.2.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.E2.m1.3.3.1.1.3.2.1.2.1" xref="S3.E2.m1.3.3.1.1.3.2.1.2.1.cmml">​</mo><mi id="S3.E2.m1.3.3.1.1.3.2.1.2.3" xref="S3.E2.m1.3.3.1.1.3.2.1.2.3.cmml">min</mi></mrow><mi id="S3.E2.m1.3.3.1.1.3.2.1.3" xref="S3.E2.m1.3.3.1.1.3.2.1.3.cmml">𝒚</mi></munder><mo id="S3.E2.m1.3.3.1.1.3.2a" xref="S3.E2.m1.3.3.1.1.3.2.cmml">⁡</mo><msub id="S3.E2.m1.3.3.1.1.3.2.2" xref="S3.E2.m1.3.3.1.1.3.2.2.cmml"><mi id="S3.E2.m1.3.3.1.1.3.2.2.2" xref="S3.E2.m1.3.3.1.1.3.2.2.2.cmml">E</mi><mi id="S3.E2.m1.3.3.1.1.3.2.2.3" xref="S3.E2.m1.3.3.1.1.3.2.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.E2.m1.3.3.1.1.3.1" xref="S3.E2.m1.3.3.1.1.3.1.cmml">​</mo><mrow id="S3.E2.m1.3.3.1.1.3.3.2" xref="S3.E2.m1.3.3.1.1.3.3.1.cmml"><mo stretchy="false" id="S3.E2.m1.3.3.1.1.3.3.2.1" xref="S3.E2.m1.3.3.1.1.3.3.1.cmml">(</mo><mi id="S3.E2.m1.1.1" xref="S3.E2.m1.1.1.cmml">𝒙</mi><mo id="S3.E2.m1.3.3.1.1.3.3.2.2" xref="S3.E2.m1.3.3.1.1.3.3.1.cmml">,</mo><mi id="S3.E2.m1.2.2" xref="S3.E2.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.E2.m1.3.3.1.1.3.3.2.3" xref="S3.E2.m1.3.3.1.1.3.3.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="S3.E2.m1.3.3.1.2" xref="S3.E2.m1.3.3.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.E2.m1.3b"><apply id="S3.E2.m1.3.3.1.1.cmml" xref="S3.E2.m1.3.3.1"><eq id="S3.E2.m1.3.3.1.1.1.cmml" xref="S3.E2.m1.3.3.1.1.1"></eq><ci id="S3.E2.m1.3.3.1.1.2.cmml" xref="S3.E2.m1.3.3.1.1.2">𝒚</ci><apply id="S3.E2.m1.3.3.1.1.3.cmml" xref="S3.E2.m1.3.3.1.1.3"><times id="S3.E2.m1.3.3.1.1.3.1.cmml" xref="S3.E2.m1.3.3.1.1.3.1"></times><apply id="S3.E2.m1.3.3.1.1.3.2.cmml" xref="S3.E2.m1.3.3.1.1.3.2"><apply id="S3.E2.m1.3.3.1.1.3.2.1.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1"><csymbol cd="ambiguous" id="S3.E2.m1.3.3.1.1.3.2.1.1.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1">subscript</csymbol><apply id="S3.E2.m1.3.3.1.1.3.2.1.2.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1.2"><times id="S3.E2.m1.3.3.1.1.3.2.1.2.1.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1.2.1"></times><ci id="S3.E2.m1.3.3.1.1.3.2.1.2.2.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1.2.2">arg</ci><ci id="S3.E2.m1.3.3.1.1.3.2.1.2.3.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1.2.3">min</ci></apply><ci id="S3.E2.m1.3.3.1.1.3.2.1.3.cmml" xref="S3.E2.m1.3.3.1.1.3.2.1.3">𝒚</ci></apply><apply id="S3.E2.m1.3.3.1.1.3.2.2.cmml" xref="S3.E2.m1.3.3.1.1.3.2.2"><csymbol cd="ambiguous" id="S3.E2.m1.3.3.1.1.3.2.2.1.cmml" xref="S3.E2.m1.3.3.1.1.3.2.2">subscript</csymbol><ci id="S3.E2.m1.3.3.1.1.3.2.2.2.cmml" xref="S3.E2.m1.3.3.1.1.3.2.2.2">𝐸</ci><ci id="S3.E2.m1.3.3.1.1.3.2.2.3.cmml" xref="S3.E2.m1.3.3.1.1.3.2.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.E2.m1.3.3.1.1.3.3.1.cmml" xref="S3.E2.m1.3.3.1.1.3.3.2"><ci id="S3.E2.m1.1.1.cmml" xref="S3.E2.m1.1.1">𝒙</ci><ci id="S3.E2.m1.2.2.cmml" xref="S3.E2.m1.2.2">𝒚</ci></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E2.m1.3c">{\bm{y}}=\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}},{\bm{y}}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(2)</span></td>
</tr></tbody>
</table>
<p id="S3.SS1.p2.6" class="ltx_p">An individual reasoning step is then represented as</p>
<table id="S3.E3" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E3.m1.2" class="ltx_Math" alttext="{\bm{y}}^{t}={\bm{y}}^{t-1}-\lambda\nabla_{{\bm{y}}}E_{\theta}({\bm{x}},{\bm{y}}^{t-1})," display="block"><semantics id="S3.E3.m1.2a"><mrow id="S3.E3.m1.2.2.1" xref="S3.E3.m1.2.2.1.1.cmml"><mrow id="S3.E3.m1.2.2.1.1" xref="S3.E3.m1.2.2.1.1.cmml"><msup id="S3.E3.m1.2.2.1.1.3" xref="S3.E3.m1.2.2.1.1.3.cmml"><mi id="S3.E3.m1.2.2.1.1.3.2" xref="S3.E3.m1.2.2.1.1.3.2.cmml">𝒚</mi><mi id="S3.E3.m1.2.2.1.1.3.3" xref="S3.E3.m1.2.2.1.1.3.3.cmml">t</mi></msup><mo id="S3.E3.m1.2.2.1.1.2" xref="S3.E3.m1.2.2.1.1.2.cmml">=</mo><mrow id="S3.E3.m1.2.2.1.1.1" xref="S3.E3.m1.2.2.1.1.1.cmml"><msup id="S3.E3.m1.2.2.1.1.1.3" xref="S3.E3.m1.2.2.1.1.1.3.cmml"><mi id="S3.E3.m1.2.2.1.1.1.3.2" xref="S3.E3.m1.2.2.1.1.1.3.2.cmml">𝒚</mi><mrow id="S3.E3.m1.2.2.1.1.1.3.3" xref="S3.E3.m1.2.2.1.1.1.3.3.cmml"><mi id="S3.E3.m1.2.2.1.1.1.3.3.2" xref="S3.E3.m1.2.2.1.1.1.3.3.2.cmml">t</mi><mo id="S3.E3.m1.2.2.1.1.1.3.3.1" xref="S3.E3.m1.2.2.1.1.1.3.3.1.cmml">−</mo><mn id="S3.E3.m1.2.2.1.1.1.3.3.3" xref="S3.E3.m1.2.2.1.1.1.3.3.3.cmml">1</mn></mrow></msup><mo id="S3.E3.m1.2.2.1.1.1.2" xref="S3.E3.m1.2.2.1.1.1.2.cmml">−</mo><mrow id="S3.E3.m1.2.2.1.1.1.1" xref="S3.E3.m1.2.2.1.1.1.1.cmml"><mi id="S3.E3.m1.2.2.1.1.1.1.3" xref="S3.E3.m1.2.2.1.1.1.1.3.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="S3.E3.m1.2.2.1.1.1.1.2" xref="S3.E3.m1.2.2.1.1.1.1.2.cmml">​</mo><mrow id="S3.E3.m1.2.2.1.1.1.1.4" xref="S3.E3.m1.2.2.1.1.1.1.4.cmml"><msub id="S3.E3.m1.2.2.1.1.1.1.4.1" xref="S3.E3.m1.2.2.1.1.1.1.4.1.cmml"><mo rspace="0.167em" id="S3.E3.m1.2.2.1.1.1.1.4.1.2" xref="S3.E3.m1.2.2.1.1.1.1.4.1.2.cmml">∇</mo><mi id="S3.E3.m1.2.2.1.1.1.1.4.1.3" xref="S3.E3.m1.2.2.1.1.1.1.4.1.3.cmml">𝒚</mi></msub><msub id="S3.E3.m1.2.2.1.1.1.1.4.2" xref="S3.E3.m1.2.2.1.1.1.1.4.2.cmml"><mi id="S3.E3.m1.2.2.1.1.1.1.4.2.2" xref="S3.E3.m1.2.2.1.1.1.1.4.2.2.cmml">E</mi><mi id="S3.E3.m1.2.2.1.1.1.1.4.2.3" xref="S3.E3.m1.2.2.1.1.1.1.4.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.E3.m1.2.2.1.1.1.1.2a" xref="S3.E3.m1.2.2.1.1.1.1.2.cmml">​</mo><mrow id="S3.E3.m1.2.2.1.1.1.1.1.1" xref="S3.E3.m1.2.2.1.1.1.1.1.2.cmml"><mo stretchy="false" id="S3.E3.m1.2.2.1.1.1.1.1.1.2" xref="S3.E3.m1.2.2.1.1.1.1.1.2.cmml">(</mo><mi id="S3.E3.m1.1.1" xref="S3.E3.m1.1.1.cmml">𝒙</mi><mo id="S3.E3.m1.2.2.1.1.1.1.1.1.3" xref="S3.E3.m1.2.2.1.1.1.1.1.2.cmml">,</mo><msup id="S3.E3.m1.2.2.1.1.1.1.1.1.1" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.cmml"><mi id="S3.E3.m1.2.2.1.1.1.1.1.1.1.2" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.2.cmml">𝒚</mi><mrow id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.cmml"><mi id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.2" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.2.cmml">t</mi><mo id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.1" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.1.cmml">−</mo><mn id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.3" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.3.cmml">1</mn></mrow></msup><mo stretchy="false" id="S3.E3.m1.2.2.1.1.1.1.1.1.4" xref="S3.E3.m1.2.2.1.1.1.1.1.2.cmml">)</mo></mrow></mrow></mrow></mrow><mo id="S3.E3.m1.2.2.1.2" xref="S3.E3.m1.2.2.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.E3.m1.2b"><apply id="S3.E3.m1.2.2.1.1.cmml" xref="S3.E3.m1.2.2.1"><eq id="S3.E3.m1.2.2.1.1.2.cmml" xref="S3.E3.m1.2.2.1.1.2"></eq><apply id="S3.E3.m1.2.2.1.1.3.cmml" xref="S3.E3.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S3.E3.m1.2.2.1.1.3.1.cmml" xref="S3.E3.m1.2.2.1.1.3">superscript</csymbol><ci id="S3.E3.m1.2.2.1.1.3.2.cmml" xref="S3.E3.m1.2.2.1.1.3.2">𝒚</ci><ci id="S3.E3.m1.2.2.1.1.3.3.cmml" xref="S3.E3.m1.2.2.1.1.3.3">𝑡</ci></apply><apply id="S3.E3.m1.2.2.1.1.1.cmml" xref="S3.E3.m1.2.2.1.1.1"><minus id="S3.E3.m1.2.2.1.1.1.2.cmml" xref="S3.E3.m1.2.2.1.1.1.2"></minus><apply id="S3.E3.m1.2.2.1.1.1.3.cmml" xref="S3.E3.m1.2.2.1.1.1.3"><csymbol cd="ambiguous" id="S3.E3.m1.2.2.1.1.1.3.1.cmml" xref="S3.E3.m1.2.2.1.1.1.3">superscript</csymbol><ci id="S3.E3.m1.2.2.1.1.1.3.2.cmml" xref="S3.E3.m1.2.2.1.1.1.3.2">𝒚</ci><apply id="S3.E3.m1.2.2.1.1.1.3.3.cmml" xref="S3.E3.m1.2.2.1.1.1.3.3"><minus id="S3.E3.m1.2.2.1.1.1.3.3.1.cmml" xref="S3.E3.m1.2.2.1.1.1.3.3.1"></minus><ci id="S3.E3.m1.2.2.1.1.1.3.3.2.cmml" xref="S3.E3.m1.2.2.1.1.1.3.3.2">𝑡</ci><cn type="integer" id="S3.E3.m1.2.2.1.1.1.3.3.3.cmml" xref="S3.E3.m1.2.2.1.1.1.3.3.3">1</cn></apply></apply><apply id="S3.E3.m1.2.2.1.1.1.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1"><times id="S3.E3.m1.2.2.1.1.1.1.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.2"></times><ci id="S3.E3.m1.2.2.1.1.1.1.3.cmml" xref="S3.E3.m1.2.2.1.1.1.1.3">𝜆</ci><apply id="S3.E3.m1.2.2.1.1.1.1.4.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4"><apply id="S3.E3.m1.2.2.1.1.1.1.4.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.1"><csymbol cd="ambiguous" id="S3.E3.m1.2.2.1.1.1.1.4.1.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.1">subscript</csymbol><ci id="S3.E3.m1.2.2.1.1.1.1.4.1.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.1.2">∇</ci><ci id="S3.E3.m1.2.2.1.1.1.1.4.1.3.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.1.3">𝒚</ci></apply><apply id="S3.E3.m1.2.2.1.1.1.1.4.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.2"><csymbol cd="ambiguous" id="S3.E3.m1.2.2.1.1.1.1.4.2.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.2">subscript</csymbol><ci id="S3.E3.m1.2.2.1.1.1.1.4.2.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.2.2">𝐸</ci><ci id="S3.E3.m1.2.2.1.1.1.1.4.2.3.cmml" xref="S3.E3.m1.2.2.1.1.1.1.4.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.E3.m1.2.2.1.1.1.1.1.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1"><ci id="S3.E3.m1.1.1.cmml" xref="S3.E3.m1.1.1">𝒙</ci><apply id="S3.E3.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E3.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1">superscript</csymbol><ci id="S3.E3.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.2">𝒚</ci><apply id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3"><minus id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.1.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.1"></minus><ci id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.2.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.2">𝑡</ci><cn type="integer" id="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.3.cmml" xref="S3.E3.m1.2.2.1.1.1.1.1.1.1.3.3">1</cn></apply></apply></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E3.m1.2c">{\bm{y}}^{t}={\bm{y}}^{t-1}-\lambda\nabla_{{\bm{y}}}E_{\theta}({\bm{x}},{\bm{y}}^{t-1}),</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(3)</span></td>
</tr></tbody>
</table>
<p id="S3.SS1.p2.5" class="ltx_p">where <math id="S3.SS1.p2.2.m1.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="S3.SS1.p2.2.m1.1a"><mi id="S3.SS1.p2.2.m1.1.1" xref="S3.SS1.p2.2.m1.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.2.m1.1b"><ci id="S3.SS1.p2.2.m1.1.1.cmml" xref="S3.SS1.p2.2.m1.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.2.m1.1c">\lambda</annotation></semantics></math> is the step size of each gradient descent step. We repeat the iterative computation in Equation <a href="#S3.E3" title="In 3.1 Iterative Reasoning as Energy Minimization ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> until <math id="S3.SS1.p2.3.m2.1" class="ltx_Math" alttext="{\bm{y}}^{t}" display="inline"><semantics id="S3.SS1.p2.3.m2.1a"><msup id="S3.SS1.p2.3.m2.1.1" xref="S3.SS1.p2.3.m2.1.1.cmml"><mi id="S3.SS1.p2.3.m2.1.1.2" xref="S3.SS1.p2.3.m2.1.1.2.cmml">𝒚</mi><mi id="S3.SS1.p2.3.m2.1.1.3" xref="S3.SS1.p2.3.m2.1.1.3.cmml">t</mi></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.3.m2.1b"><apply id="S3.SS1.p2.3.m2.1.1.cmml" xref="S3.SS1.p2.3.m2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.3.m2.1.1.1.cmml" xref="S3.SS1.p2.3.m2.1.1">superscript</csymbol><ci id="S3.SS1.p2.3.m2.1.1.2.cmml" xref="S3.SS1.p2.3.m2.1.1.2">𝒚</ci><ci id="S3.SS1.p2.3.m2.1.1.3.cmml" xref="S3.SS1.p2.3.m2.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.3.m2.1c">{\bm{y}}^{t}</annotation></semantics></math> is a local minima of the energy landscape, <math id="S3.SS1.p2.4.m3.2" class="ltx_Math" alttext="E_{\theta}({\bm{y}}^{t})=E_{\theta}({\bm{y}}^{t-1})" display="inline"><semantics id="S3.SS1.p2.4.m3.2a"><mrow id="S3.SS1.p2.4.m3.2.2" xref="S3.SS1.p2.4.m3.2.2.cmml"><mrow id="S3.SS1.p2.4.m3.1.1.1" xref="S3.SS1.p2.4.m3.1.1.1.cmml"><msub id="S3.SS1.p2.4.m3.1.1.1.3" xref="S3.SS1.p2.4.m3.1.1.1.3.cmml"><mi id="S3.SS1.p2.4.m3.1.1.1.3.2" xref="S3.SS1.p2.4.m3.1.1.1.3.2.cmml">E</mi><mi id="S3.SS1.p2.4.m3.1.1.1.3.3" xref="S3.SS1.p2.4.m3.1.1.1.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS1.p2.4.m3.1.1.1.2" xref="S3.SS1.p2.4.m3.1.1.1.2.cmml">​</mo><mrow id="S3.SS1.p2.4.m3.1.1.1.1.1" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S3.SS1.p2.4.m3.1.1.1.1.1.2" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.cmml">(</mo><msup id="S3.SS1.p2.4.m3.1.1.1.1.1.1" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.cmml"><mi id="S3.SS1.p2.4.m3.1.1.1.1.1.1.2" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.2.cmml">𝒚</mi><mi id="S3.SS1.p2.4.m3.1.1.1.1.1.1.3" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.3.cmml">t</mi></msup><mo stretchy="false" id="S3.SS1.p2.4.m3.1.1.1.1.1.3" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S3.SS1.p2.4.m3.2.2.3" xref="S3.SS1.p2.4.m3.2.2.3.cmml">=</mo><mrow id="S3.SS1.p2.4.m3.2.2.2" xref="S3.SS1.p2.4.m3.2.2.2.cmml"><msub id="S3.SS1.p2.4.m3.2.2.2.3" xref="S3.SS1.p2.4.m3.2.2.2.3.cmml"><mi id="S3.SS1.p2.4.m3.2.2.2.3.2" xref="S3.SS1.p2.4.m3.2.2.2.3.2.cmml">E</mi><mi id="S3.SS1.p2.4.m3.2.2.2.3.3" xref="S3.SS1.p2.4.m3.2.2.2.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS1.p2.4.m3.2.2.2.2" xref="S3.SS1.p2.4.m3.2.2.2.2.cmml">​</mo><mrow id="S3.SS1.p2.4.m3.2.2.2.1.1" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.cmml"><mo stretchy="false" id="S3.SS1.p2.4.m3.2.2.2.1.1.2" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.cmml">(</mo><msup id="S3.SS1.p2.4.m3.2.2.2.1.1.1" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.cmml"><mi id="S3.SS1.p2.4.m3.2.2.2.1.1.1.2" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.2.cmml">𝒚</mi><mrow id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.cmml"><mi id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.2" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.2.cmml">t</mi><mo id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.1" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.1.cmml">−</mo><mn id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.3" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.3.cmml">1</mn></mrow></msup><mo stretchy="false" id="S3.SS1.p2.4.m3.2.2.2.1.1.3" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.4.m3.2b"><apply id="S3.SS1.p2.4.m3.2.2.cmml" xref="S3.SS1.p2.4.m3.2.2"><eq id="S3.SS1.p2.4.m3.2.2.3.cmml" xref="S3.SS1.p2.4.m3.2.2.3"></eq><apply id="S3.SS1.p2.4.m3.1.1.1.cmml" xref="S3.SS1.p2.4.m3.1.1.1"><times id="S3.SS1.p2.4.m3.1.1.1.2.cmml" xref="S3.SS1.p2.4.m3.1.1.1.2"></times><apply id="S3.SS1.p2.4.m3.1.1.1.3.cmml" xref="S3.SS1.p2.4.m3.1.1.1.3"><csymbol cd="ambiguous" id="S3.SS1.p2.4.m3.1.1.1.3.1.cmml" xref="S3.SS1.p2.4.m3.1.1.1.3">subscript</csymbol><ci id="S3.SS1.p2.4.m3.1.1.1.3.2.cmml" xref="S3.SS1.p2.4.m3.1.1.1.3.2">𝐸</ci><ci id="S3.SS1.p2.4.m3.1.1.1.3.3.cmml" xref="S3.SS1.p2.4.m3.1.1.1.3.3">𝜃</ci></apply><apply id="S3.SS1.p2.4.m3.1.1.1.1.1.1.cmml" xref="S3.SS1.p2.4.m3.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.4.m3.1.1.1.1.1.1.1.cmml" xref="S3.SS1.p2.4.m3.1.1.1.1.1">superscript</csymbol><ci id="S3.SS1.p2.4.m3.1.1.1.1.1.1.2.cmml" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.2">𝒚</ci><ci id="S3.SS1.p2.4.m3.1.1.1.1.1.1.3.cmml" xref="S3.SS1.p2.4.m3.1.1.1.1.1.1.3">𝑡</ci></apply></apply><apply id="S3.SS1.p2.4.m3.2.2.2.cmml" xref="S3.SS1.p2.4.m3.2.2.2"><times id="S3.SS1.p2.4.m3.2.2.2.2.cmml" xref="S3.SS1.p2.4.m3.2.2.2.2"></times><apply id="S3.SS1.p2.4.m3.2.2.2.3.cmml" xref="S3.SS1.p2.4.m3.2.2.2.3"><csymbol cd="ambiguous" id="S3.SS1.p2.4.m3.2.2.2.3.1.cmml" xref="S3.SS1.p2.4.m3.2.2.2.3">subscript</csymbol><ci id="S3.SS1.p2.4.m3.2.2.2.3.2.cmml" xref="S3.SS1.p2.4.m3.2.2.2.3.2">𝐸</ci><ci id="S3.SS1.p2.4.m3.2.2.2.3.3.cmml" xref="S3.SS1.p2.4.m3.2.2.2.3.3">𝜃</ci></apply><apply id="S3.SS1.p2.4.m3.2.2.2.1.1.1.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.4.m3.2.2.2.1.1.1.1.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1">superscript</csymbol><ci id="S3.SS1.p2.4.m3.2.2.2.1.1.1.2.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.2">𝒚</ci><apply id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3"><minus id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.1.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.1"></minus><ci id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.2.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.2">𝑡</ci><cn type="integer" id="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.3.cmml" xref="S3.SS1.p2.4.m3.2.2.2.1.1.1.3.3">1</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.4.m3.2c">E_{\theta}({\bm{y}}^{t})=E_{\theta}({\bm{y}}^{t-1})</annotation></semantics></math>, where additional steps of optimization do not change the energy value of <math id="S3.SS1.p2.5.m4.1" class="ltx_Math" alttext="{\bm{y}}^{t}" display="inline"><semantics id="S3.SS1.p2.5.m4.1a"><msup id="S3.SS1.p2.5.m4.1.1" xref="S3.SS1.p2.5.m4.1.1.cmml"><mi id="S3.SS1.p2.5.m4.1.1.2" xref="S3.SS1.p2.5.m4.1.1.2.cmml">𝒚</mi><mi id="S3.SS1.p2.5.m4.1.1.3" xref="S3.SS1.p2.5.m4.1.1.3.cmml">t</mi></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.5.m4.1b"><apply id="S3.SS1.p2.5.m4.1.1.cmml" xref="S3.SS1.p2.5.m4.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.5.m4.1.1.1.cmml" xref="S3.SS1.p2.5.m4.1.1">superscript</csymbol><ci id="S3.SS1.p2.5.m4.1.1.2.cmml" xref="S3.SS1.p2.5.m4.1.1.2">𝒚</ci><ci id="S3.SS1.p2.5.m4.1.1.3.cmml" xref="S3.SS1.p2.5.m4.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.5.m4.1c">{\bm{y}}^{t}</annotation></semantics></math>. Finding a local minimum of the energy landscape thus provides us with a natural mechanism to halt iterative computation.</p>
</div>
</section>
<section id="S3.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.2 </span>Learning Energy Landscapes for Reasoning</h3>

<div id="S3.SS2.p1" class="ltx_para">
<p id="S3.SS2.p1.5" class="ltx_p">We next discuss how to learn IREM. Given an input problem <math id="S3.SS2.p1.1.m1.1" class="ltx_Math" alttext="{\bm{x}}_{i}" display="inline"><semantics id="S3.SS2.p1.1.m1.1a"><msub id="S3.SS2.p1.1.m1.1.1" xref="S3.SS2.p1.1.m1.1.1.cmml"><mi id="S3.SS2.p1.1.m1.1.1.2" xref="S3.SS2.p1.1.m1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p1.1.m1.1.1.3" xref="S3.SS2.p1.1.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.1.m1.1b"><apply id="S3.SS2.p1.1.m1.1.1.cmml" xref="S3.SS2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.1.m1.1.1.1.cmml" xref="S3.SS2.p1.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p1.1.m1.1.1.2.cmml" xref="S3.SS2.p1.1.m1.1.1.2">𝒙</ci><ci id="S3.SS2.p1.1.m1.1.1.3.cmml" xref="S3.SS2.p1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.1.m1.1c">{\bm{x}}_{i}</annotation></semantics></math> with a unique solution <math id="S3.SS2.p1.2.m2.1" class="ltx_Math" alttext="{\bm{y}}_{i}" display="inline"><semantics id="S3.SS2.p1.2.m2.1a"><msub id="S3.SS2.p1.2.m2.1.1" xref="S3.SS2.p1.2.m2.1.1.cmml"><mi id="S3.SS2.p1.2.m2.1.1.2" xref="S3.SS2.p1.2.m2.1.1.2.cmml">𝒚</mi><mi id="S3.SS2.p1.2.m2.1.1.3" xref="S3.SS2.p1.2.m2.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.2.m2.1b"><apply id="S3.SS2.p1.2.m2.1.1.cmml" xref="S3.SS2.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.2.m2.1.1.1.cmml" xref="S3.SS2.p1.2.m2.1.1">subscript</csymbol><ci id="S3.SS2.p1.2.m2.1.1.2.cmml" xref="S3.SS2.p1.2.m2.1.1.2">𝒚</ci><ci id="S3.SS2.p1.2.m2.1.1.3.cmml" xref="S3.SS2.p1.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.2.m2.1c">{\bm{y}}_{i}</annotation></semantics></math>, the simplest method to learn <math id="S3.SS2.p1.3.m3.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS2.p1.3.m3.2a"><mrow id="S3.SS2.p1.3.m3.2.3" xref="S3.SS2.p1.3.m3.2.3.cmml"><msub id="S3.SS2.p1.3.m3.2.3.2" xref="S3.SS2.p1.3.m3.2.3.2.cmml"><mi id="S3.SS2.p1.3.m3.2.3.2.2" xref="S3.SS2.p1.3.m3.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p1.3.m3.2.3.2.3" xref="S3.SS2.p1.3.m3.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS2.p1.3.m3.2.3.1" xref="S3.SS2.p1.3.m3.2.3.1.cmml">​</mo><mrow id="S3.SS2.p1.3.m3.2.3.3.2" xref="S3.SS2.p1.3.m3.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS2.p1.3.m3.2.3.3.2.1" xref="S3.SS2.p1.3.m3.2.3.3.1.cmml">(</mo><mi id="S3.SS2.p1.3.m3.1.1" xref="S3.SS2.p1.3.m3.1.1.cmml">𝒙</mi><mo id="S3.SS2.p1.3.m3.2.3.3.2.2" xref="S3.SS2.p1.3.m3.2.3.3.1.cmml">,</mo><mi id="S3.SS2.p1.3.m3.2.2" xref="S3.SS2.p1.3.m3.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p1.3.m3.2.3.3.2.3" xref="S3.SS2.p1.3.m3.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.3.m3.2b"><apply id="S3.SS2.p1.3.m3.2.3.cmml" xref="S3.SS2.p1.3.m3.2.3"><times id="S3.SS2.p1.3.m3.2.3.1.cmml" xref="S3.SS2.p1.3.m3.2.3.1"></times><apply id="S3.SS2.p1.3.m3.2.3.2.cmml" xref="S3.SS2.p1.3.m3.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p1.3.m3.2.3.2.1.cmml" xref="S3.SS2.p1.3.m3.2.3.2">subscript</csymbol><ci id="S3.SS2.p1.3.m3.2.3.2.2.cmml" xref="S3.SS2.p1.3.m3.2.3.2.2">𝐸</ci><ci id="S3.SS2.p1.3.m3.2.3.2.3.cmml" xref="S3.SS2.p1.3.m3.2.3.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS2.p1.3.m3.2.3.3.1.cmml" xref="S3.SS2.p1.3.m3.2.3.3.2"><ci id="S3.SS2.p1.3.m3.1.1.cmml" xref="S3.SS2.p1.3.m3.1.1">𝒙</ci><ci id="S3.SS2.p1.3.m3.2.2.cmml" xref="S3.SS2.p1.3.m3.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.3.m3.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math> is to directly supervise the minimal energy state <math id="S3.SS2.p1.4.m4.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})" display="inline"><semantics id="S3.SS2.p1.4.m4.2a"><mrow id="S3.SS2.p1.4.m4.2.2" xref="S3.SS2.p1.4.m4.2.2.cmml"><mrow id="S3.SS2.p1.4.m4.2.2.3" xref="S3.SS2.p1.4.m4.2.2.3.cmml"><msub id="S3.SS2.p1.4.m4.2.2.3.1" xref="S3.SS2.p1.4.m4.2.2.3.1.cmml"><mrow id="S3.SS2.p1.4.m4.2.2.3.1.2" xref="S3.SS2.p1.4.m4.2.2.3.1.2.cmml"><mi id="S3.SS2.p1.4.m4.2.2.3.1.2.2" xref="S3.SS2.p1.4.m4.2.2.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p1.4.m4.2.2.3.1.2.1" xref="S3.SS2.p1.4.m4.2.2.3.1.2.1.cmml">​</mo><mi id="S3.SS2.p1.4.m4.2.2.3.1.2.3" xref="S3.SS2.p1.4.m4.2.2.3.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p1.4.m4.2.2.3.1.3" xref="S3.SS2.p1.4.m4.2.2.3.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p1.4.m4.2.2.3a" xref="S3.SS2.p1.4.m4.2.2.3.cmml">⁡</mo><msub id="S3.SS2.p1.4.m4.2.2.3.2" xref="S3.SS2.p1.4.m4.2.2.3.2.cmml"><mi id="S3.SS2.p1.4.m4.2.2.3.2.2" xref="S3.SS2.p1.4.m4.2.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p1.4.m4.2.2.3.2.3" xref="S3.SS2.p1.4.m4.2.2.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p1.4.m4.2.2.2" xref="S3.SS2.p1.4.m4.2.2.2.cmml">​</mo><mrow id="S3.SS2.p1.4.m4.2.2.1.1" xref="S3.SS2.p1.4.m4.2.2.1.2.cmml"><mo stretchy="false" id="S3.SS2.p1.4.m4.2.2.1.1.2" xref="S3.SS2.p1.4.m4.2.2.1.2.cmml">(</mo><msub id="S3.SS2.p1.4.m4.2.2.1.1.1" xref="S3.SS2.p1.4.m4.2.2.1.1.1.cmml"><mi id="S3.SS2.p1.4.m4.2.2.1.1.1.2" xref="S3.SS2.p1.4.m4.2.2.1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p1.4.m4.2.2.1.1.1.3" xref="S3.SS2.p1.4.m4.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S3.SS2.p1.4.m4.2.2.1.1.3" xref="S3.SS2.p1.4.m4.2.2.1.2.cmml">,</mo><mi id="S3.SS2.p1.4.m4.1.1" xref="S3.SS2.p1.4.m4.1.1.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p1.4.m4.2.2.1.1.4" xref="S3.SS2.p1.4.m4.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.4.m4.2b"><apply id="S3.SS2.p1.4.m4.2.2.cmml" xref="S3.SS2.p1.4.m4.2.2"><times id="S3.SS2.p1.4.m4.2.2.2.cmml" xref="S3.SS2.p1.4.m4.2.2.2"></times><apply id="S3.SS2.p1.4.m4.2.2.3.cmml" xref="S3.SS2.p1.4.m4.2.2.3"><apply id="S3.SS2.p1.4.m4.2.2.3.1.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1"><csymbol cd="ambiguous" id="S3.SS2.p1.4.m4.2.2.3.1.1.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1">subscript</csymbol><apply id="S3.SS2.p1.4.m4.2.2.3.1.2.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1.2"><times id="S3.SS2.p1.4.m4.2.2.3.1.2.1.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1.2.1"></times><ci id="S3.SS2.p1.4.m4.2.2.3.1.2.2.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1.2.2">arg</ci><ci id="S3.SS2.p1.4.m4.2.2.3.1.2.3.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1.2.3">min</ci></apply><ci id="S3.SS2.p1.4.m4.2.2.3.1.3.cmml" xref="S3.SS2.p1.4.m4.2.2.3.1.3">𝒚</ci></apply><apply id="S3.SS2.p1.4.m4.2.2.3.2.cmml" xref="S3.SS2.p1.4.m4.2.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p1.4.m4.2.2.3.2.1.cmml" xref="S3.SS2.p1.4.m4.2.2.3.2">subscript</csymbol><ci id="S3.SS2.p1.4.m4.2.2.3.2.2.cmml" xref="S3.SS2.p1.4.m4.2.2.3.2.2">𝐸</ci><ci id="S3.SS2.p1.4.m4.2.2.3.2.3.cmml" xref="S3.SS2.p1.4.m4.2.2.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p1.4.m4.2.2.1.2.cmml" xref="S3.SS2.p1.4.m4.2.2.1.1"><apply id="S3.SS2.p1.4.m4.2.2.1.1.1.cmml" xref="S3.SS2.p1.4.m4.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.4.m4.2.2.1.1.1.1.cmml" xref="S3.SS2.p1.4.m4.2.2.1.1.1">subscript</csymbol><ci id="S3.SS2.p1.4.m4.2.2.1.1.1.2.cmml" xref="S3.SS2.p1.4.m4.2.2.1.1.1.2">𝒙</ci><ci id="S3.SS2.p1.4.m4.2.2.1.1.1.3.cmml" xref="S3.SS2.p1.4.m4.2.2.1.1.1.3">𝑖</ci></apply><ci id="S3.SS2.p1.4.m4.1.1.cmml" xref="S3.SS2.p1.4.m4.1.1">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.4.m4.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})</annotation></semantics></math> with <math id="S3.SS2.p1.5.m5.1" class="ltx_Math" alttext="{\bm{y}}_{i}" display="inline"><semantics id="S3.SS2.p1.5.m5.1a"><msub id="S3.SS2.p1.5.m5.1.1" xref="S3.SS2.p1.5.m5.1.1.cmml"><mi id="S3.SS2.p1.5.m5.1.1.2" xref="S3.SS2.p1.5.m5.1.1.2.cmml">𝒚</mi><mi id="S3.SS2.p1.5.m5.1.1.3" xref="S3.SS2.p1.5.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.5.m5.1b"><apply id="S3.SS2.p1.5.m5.1.1.cmml" xref="S3.SS2.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.5.m5.1.1.1.cmml" xref="S3.SS2.p1.5.m5.1.1">subscript</csymbol><ci id="S3.SS2.p1.5.m5.1.1.2.cmml" xref="S3.SS2.p1.5.m5.1.1.2">𝒚</ci><ci id="S3.SS2.p1.5.m5.1.1.3.cmml" xref="S3.SS2.p1.5.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.5.m5.1c">{\bm{y}}_{i}</annotation></semantics></math> through regression</p>
<table id="S3.E4" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E4.m1.3" class="ltx_Math" alttext="\mathcal{L}_{\text{MSE}}(\theta)=\|\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})-{\bm{y}}_{i}\|^{2}." display="block"><semantics id="S3.E4.m1.3a"><mrow id="S3.E4.m1.3.3.1" xref="S3.E4.m1.3.3.1.1.cmml"><mrow id="S3.E4.m1.3.3.1.1" xref="S3.E4.m1.3.3.1.1.cmml"><mrow id="S3.E4.m1.3.3.1.1.3" xref="S3.E4.m1.3.3.1.1.3.cmml"><msub id="S3.E4.m1.3.3.1.1.3.2" xref="S3.E4.m1.3.3.1.1.3.2.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.E4.m1.3.3.1.1.3.2.2" xref="S3.E4.m1.3.3.1.1.3.2.2.cmml">ℒ</mi><mtext id="S3.E4.m1.3.3.1.1.3.2.3" xref="S3.E4.m1.3.3.1.1.3.2.3a.cmml">MSE</mtext></msub><mo lspace="0em" rspace="0em" id="S3.E4.m1.3.3.1.1.3.1" xref="S3.E4.m1.3.3.1.1.3.1.cmml">​</mo><mrow id="S3.E4.m1.3.3.1.1.3.3.2" xref="S3.E4.m1.3.3.1.1.3.cmml"><mo stretchy="false" id="S3.E4.m1.3.3.1.1.3.3.2.1" xref="S3.E4.m1.3.3.1.1.3.cmml">(</mo><mi id="S3.E4.m1.1.1" xref="S3.E4.m1.1.1.cmml">θ</mi><mo stretchy="false" id="S3.E4.m1.3.3.1.1.3.3.2.2" xref="S3.E4.m1.3.3.1.1.3.cmml">)</mo></mrow></mrow><mo id="S3.E4.m1.3.3.1.1.2" xref="S3.E4.m1.3.3.1.1.2.cmml">=</mo><msup id="S3.E4.m1.3.3.1.1.1" xref="S3.E4.m1.3.3.1.1.1.cmml"><mrow id="S3.E4.m1.3.3.1.1.1.1.1" xref="S3.E4.m1.3.3.1.1.1.1.2.cmml"><mo stretchy="false" id="S3.E4.m1.3.3.1.1.1.1.1.2" xref="S3.E4.m1.3.3.1.1.1.1.2.1.cmml">‖</mo><mrow id="S3.E4.m1.3.3.1.1.1.1.1.1" xref="S3.E4.m1.3.3.1.1.1.1.1.1.cmml"><mrow id="S3.E4.m1.3.3.1.1.1.1.1.1.1" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.cmml"><mrow id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.cmml"><munder id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.cmml"><mrow id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.cmml"><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.1" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.1.cmml">​</mo><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.3.cmml">min</mi></mrow><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.3.cmml">𝒚</mi></munder><mo id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3a" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.cmml">⁡</mo><msub id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.cmml"><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.2.cmml">E</mi><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.2.cmml">​</mo><mrow id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.2.cmml"><mo stretchy="false" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.2.cmml">(</mo><msub id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.2.cmml">,</mo><mi id="S3.E4.m1.2.2" xref="S3.E4.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.4" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.2.cmml">)</mo></mrow></mrow><mo id="S3.E4.m1.3.3.1.1.1.1.1.1.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.2.cmml">−</mo><msub id="S3.E4.m1.3.3.1.1.1.1.1.1.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3.cmml"><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.3.2" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3.2.cmml">𝒚</mi><mi id="S3.E4.m1.3.3.1.1.1.1.1.1.3.3" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo stretchy="false" id="S3.E4.m1.3.3.1.1.1.1.1.3" xref="S3.E4.m1.3.3.1.1.1.1.2.1.cmml">‖</mo></mrow><mn id="S3.E4.m1.3.3.1.1.1.3" xref="S3.E4.m1.3.3.1.1.1.3.cmml">2</mn></msup></mrow><mo lspace="0em" id="S3.E4.m1.3.3.1.2" xref="S3.E4.m1.3.3.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.E4.m1.3b"><apply id="S3.E4.m1.3.3.1.1.cmml" xref="S3.E4.m1.3.3.1"><eq id="S3.E4.m1.3.3.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.2"></eq><apply id="S3.E4.m1.3.3.1.1.3.cmml" xref="S3.E4.m1.3.3.1.1.3"><times id="S3.E4.m1.3.3.1.1.3.1.cmml" xref="S3.E4.m1.3.3.1.1.3.1"></times><apply id="S3.E4.m1.3.3.1.1.3.2.cmml" xref="S3.E4.m1.3.3.1.1.3.2"><csymbol cd="ambiguous" id="S3.E4.m1.3.3.1.1.3.2.1.cmml" xref="S3.E4.m1.3.3.1.1.3.2">subscript</csymbol><ci id="S3.E4.m1.3.3.1.1.3.2.2.cmml" xref="S3.E4.m1.3.3.1.1.3.2.2">ℒ</ci><ci id="S3.E4.m1.3.3.1.1.3.2.3a.cmml" xref="S3.E4.m1.3.3.1.1.3.2.3"><mtext mathsize="70%" id="S3.E4.m1.3.3.1.1.3.2.3.cmml" xref="S3.E4.m1.3.3.1.1.3.2.3">MSE</mtext></ci></apply><ci id="S3.E4.m1.1.1.cmml" xref="S3.E4.m1.1.1">𝜃</ci></apply><apply id="S3.E4.m1.3.3.1.1.1.cmml" xref="S3.E4.m1.3.3.1.1.1"><csymbol cd="ambiguous" id="S3.E4.m1.3.3.1.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1">superscript</csymbol><apply id="S3.E4.m1.3.3.1.1.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1"><csymbol cd="latexml" id="S3.E4.m1.3.3.1.1.1.1.2.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.2">norm</csymbol><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1"><minus id="S3.E4.m1.3.3.1.1.1.1.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.2"></minus><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1"><times id="S3.E4.m1.3.3.1.1.1.1.1.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.2"></times><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3"><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1"><csymbol cd="ambiguous" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1">subscript</csymbol><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2"><times id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.1"></times><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.2">arg</ci><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.2.3">min</ci></apply><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.1.3">𝒚</ci></apply><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2"><csymbol cd="ambiguous" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2">subscript</csymbol><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.2">𝐸</ci><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1"><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.1.1.1.1.3">𝑖</ci></apply><ci id="S3.E4.m1.2.2.cmml" xref="S3.E4.m1.2.2">𝒚</ci></interval></apply><apply id="S3.E4.m1.3.3.1.1.1.1.1.1.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="S3.E4.m1.3.3.1.1.1.1.1.1.3.1.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3">subscript</csymbol><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.3.2.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3.2">𝒚</ci><ci id="S3.E4.m1.3.3.1.1.1.1.1.1.3.3.cmml" xref="S3.E4.m1.3.3.1.1.1.1.1.1.3.3">𝑖</ci></apply></apply></apply><cn type="integer" id="S3.E4.m1.3.3.1.1.1.3.cmml" xref="S3.E4.m1.3.3.1.1.1.3">2</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E4.m1.3c">\mathcal{L}_{\text{MSE}}(\theta)=\|\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})-{\bm{y}}_{i}\|^{2}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(4)</span></td>
</tr></tbody>
</table>
<p id="S3.SS2.p1.6" class="ltx_p">However, in practice, during training time, it is computationally expensive to compute <math id="S3.SS2.p1.6.m1.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS2.p1.6.m1.2a"><mrow id="S3.SS2.p1.6.m1.2.3" xref="S3.SS2.p1.6.m1.2.3.cmml"><mrow id="S3.SS2.p1.6.m1.2.3.2" xref="S3.SS2.p1.6.m1.2.3.2.cmml"><msub id="S3.SS2.p1.6.m1.2.3.2.1" xref="S3.SS2.p1.6.m1.2.3.2.1.cmml"><mrow id="S3.SS2.p1.6.m1.2.3.2.1.2" xref="S3.SS2.p1.6.m1.2.3.2.1.2.cmml"><mi id="S3.SS2.p1.6.m1.2.3.2.1.2.2" xref="S3.SS2.p1.6.m1.2.3.2.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p1.6.m1.2.3.2.1.2.1" xref="S3.SS2.p1.6.m1.2.3.2.1.2.1.cmml">​</mo><mi id="S3.SS2.p1.6.m1.2.3.2.1.2.3" xref="S3.SS2.p1.6.m1.2.3.2.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p1.6.m1.2.3.2.1.3" xref="S3.SS2.p1.6.m1.2.3.2.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p1.6.m1.2.3.2a" xref="S3.SS2.p1.6.m1.2.3.2.cmml">⁡</mo><msub id="S3.SS2.p1.6.m1.2.3.2.2" xref="S3.SS2.p1.6.m1.2.3.2.2.cmml"><mi id="S3.SS2.p1.6.m1.2.3.2.2.2" xref="S3.SS2.p1.6.m1.2.3.2.2.2.cmml">E</mi><mi id="S3.SS2.p1.6.m1.2.3.2.2.3" xref="S3.SS2.p1.6.m1.2.3.2.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p1.6.m1.2.3.1" xref="S3.SS2.p1.6.m1.2.3.1.cmml">​</mo><mrow id="S3.SS2.p1.6.m1.2.3.3.2" xref="S3.SS2.p1.6.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS2.p1.6.m1.2.3.3.2.1" xref="S3.SS2.p1.6.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS2.p1.6.m1.1.1" xref="S3.SS2.p1.6.m1.1.1.cmml">𝒙</mi><mo id="S3.SS2.p1.6.m1.2.3.3.2.2" xref="S3.SS2.p1.6.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS2.p1.6.m1.2.2" xref="S3.SS2.p1.6.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p1.6.m1.2.3.3.2.3" xref="S3.SS2.p1.6.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.6.m1.2b"><apply id="S3.SS2.p1.6.m1.2.3.cmml" xref="S3.SS2.p1.6.m1.2.3"><times id="S3.SS2.p1.6.m1.2.3.1.cmml" xref="S3.SS2.p1.6.m1.2.3.1"></times><apply id="S3.SS2.p1.6.m1.2.3.2.cmml" xref="S3.SS2.p1.6.m1.2.3.2"><apply id="S3.SS2.p1.6.m1.2.3.2.1.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1"><csymbol cd="ambiguous" id="S3.SS2.p1.6.m1.2.3.2.1.1.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1">subscript</csymbol><apply id="S3.SS2.p1.6.m1.2.3.2.1.2.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1.2"><times id="S3.SS2.p1.6.m1.2.3.2.1.2.1.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1.2.1"></times><ci id="S3.SS2.p1.6.m1.2.3.2.1.2.2.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1.2.2">arg</ci><ci id="S3.SS2.p1.6.m1.2.3.2.1.2.3.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1.2.3">min</ci></apply><ci id="S3.SS2.p1.6.m1.2.3.2.1.3.cmml" xref="S3.SS2.p1.6.m1.2.3.2.1.3">𝒚</ci></apply><apply id="S3.SS2.p1.6.m1.2.3.2.2.cmml" xref="S3.SS2.p1.6.m1.2.3.2.2"><csymbol cd="ambiguous" id="S3.SS2.p1.6.m1.2.3.2.2.1.cmml" xref="S3.SS2.p1.6.m1.2.3.2.2">subscript</csymbol><ci id="S3.SS2.p1.6.m1.2.3.2.2.2.cmml" xref="S3.SS2.p1.6.m1.2.3.2.2.2">𝐸</ci><ci id="S3.SS2.p1.6.m1.2.3.2.2.3.cmml" xref="S3.SS2.p1.6.m1.2.3.2.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p1.6.m1.2.3.3.1.cmml" xref="S3.SS2.p1.6.m1.2.3.3.2"><ci id="S3.SS2.p1.6.m1.1.1.cmml" xref="S3.SS2.p1.6.m1.1.1">𝒙</ci><ci id="S3.SS2.p1.6.m1.2.2.cmml" xref="S3.SS2.p1.6.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.6.m1.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math>, as the underlying energy landscape may be complex.</p>
</div>
<div id="S3.SS2.p2" class="ltx_para">
<p id="S3.SS2.p2.5" class="ltx_p">As a fast approximation to <math id="S3.SS2.p2.1.m1.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})" display="inline"><semantics id="S3.SS2.p2.1.m1.2a"><mrow id="S3.SS2.p2.1.m1.2.2" xref="S3.SS2.p2.1.m1.2.2.cmml"><mrow id="S3.SS2.p2.1.m1.2.2.3" xref="S3.SS2.p2.1.m1.2.2.3.cmml"><msub id="S3.SS2.p2.1.m1.2.2.3.1" xref="S3.SS2.p2.1.m1.2.2.3.1.cmml"><mrow id="S3.SS2.p2.1.m1.2.2.3.1.2" xref="S3.SS2.p2.1.m1.2.2.3.1.2.cmml"><mi id="S3.SS2.p2.1.m1.2.2.3.1.2.2" xref="S3.SS2.p2.1.m1.2.2.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p2.1.m1.2.2.3.1.2.1" xref="S3.SS2.p2.1.m1.2.2.3.1.2.1.cmml">​</mo><mi id="S3.SS2.p2.1.m1.2.2.3.1.2.3" xref="S3.SS2.p2.1.m1.2.2.3.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p2.1.m1.2.2.3.1.3" xref="S3.SS2.p2.1.m1.2.2.3.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p2.1.m1.2.2.3a" xref="S3.SS2.p2.1.m1.2.2.3.cmml">⁡</mo><msub id="S3.SS2.p2.1.m1.2.2.3.2" xref="S3.SS2.p2.1.m1.2.2.3.2.cmml"><mi id="S3.SS2.p2.1.m1.2.2.3.2.2" xref="S3.SS2.p2.1.m1.2.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p2.1.m1.2.2.3.2.3" xref="S3.SS2.p2.1.m1.2.2.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p2.1.m1.2.2.2" xref="S3.SS2.p2.1.m1.2.2.2.cmml">​</mo><mrow id="S3.SS2.p2.1.m1.2.2.1.1" xref="S3.SS2.p2.1.m1.2.2.1.2.cmml"><mo stretchy="false" id="S3.SS2.p2.1.m1.2.2.1.1.2" xref="S3.SS2.p2.1.m1.2.2.1.2.cmml">(</mo><msub id="S3.SS2.p2.1.m1.2.2.1.1.1" xref="S3.SS2.p2.1.m1.2.2.1.1.1.cmml"><mi id="S3.SS2.p2.1.m1.2.2.1.1.1.2" xref="S3.SS2.p2.1.m1.2.2.1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p2.1.m1.2.2.1.1.1.3" xref="S3.SS2.p2.1.m1.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S3.SS2.p2.1.m1.2.2.1.1.3" xref="S3.SS2.p2.1.m1.2.2.1.2.cmml">,</mo><mi id="S3.SS2.p2.1.m1.1.1" xref="S3.SS2.p2.1.m1.1.1.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p2.1.m1.2.2.1.1.4" xref="S3.SS2.p2.1.m1.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.1.m1.2b"><apply id="S3.SS2.p2.1.m1.2.2.cmml" xref="S3.SS2.p2.1.m1.2.2"><times id="S3.SS2.p2.1.m1.2.2.2.cmml" xref="S3.SS2.p2.1.m1.2.2.2"></times><apply id="S3.SS2.p2.1.m1.2.2.3.cmml" xref="S3.SS2.p2.1.m1.2.2.3"><apply id="S3.SS2.p2.1.m1.2.2.3.1.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1"><csymbol cd="ambiguous" id="S3.SS2.p2.1.m1.2.2.3.1.1.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1">subscript</csymbol><apply id="S3.SS2.p2.1.m1.2.2.3.1.2.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1.2"><times id="S3.SS2.p2.1.m1.2.2.3.1.2.1.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1.2.1"></times><ci id="S3.SS2.p2.1.m1.2.2.3.1.2.2.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1.2.2">arg</ci><ci id="S3.SS2.p2.1.m1.2.2.3.1.2.3.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1.2.3">min</ci></apply><ci id="S3.SS2.p2.1.m1.2.2.3.1.3.cmml" xref="S3.SS2.p2.1.m1.2.2.3.1.3">𝒚</ci></apply><apply id="S3.SS2.p2.1.m1.2.2.3.2.cmml" xref="S3.SS2.p2.1.m1.2.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p2.1.m1.2.2.3.2.1.cmml" xref="S3.SS2.p2.1.m1.2.2.3.2">subscript</csymbol><ci id="S3.SS2.p2.1.m1.2.2.3.2.2.cmml" xref="S3.SS2.p2.1.m1.2.2.3.2.2">𝐸</ci><ci id="S3.SS2.p2.1.m1.2.2.3.2.3.cmml" xref="S3.SS2.p2.1.m1.2.2.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p2.1.m1.2.2.1.2.cmml" xref="S3.SS2.p2.1.m1.2.2.1.1"><apply id="S3.SS2.p2.1.m1.2.2.1.1.1.cmml" xref="S3.SS2.p2.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.1.m1.2.2.1.1.1.1.cmml" xref="S3.SS2.p2.1.m1.2.2.1.1.1">subscript</csymbol><ci id="S3.SS2.p2.1.m1.2.2.1.1.1.2.cmml" xref="S3.SS2.p2.1.m1.2.2.1.1.1.2">𝒙</ci><ci id="S3.SS2.p2.1.m1.2.2.1.1.1.3.cmml" xref="S3.SS2.p2.1.m1.2.2.1.1.1.3">𝑖</ci></apply><ci id="S3.SS2.p2.1.m1.1.1.cmml" xref="S3.SS2.p2.1.m1.1.1">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.1.m1.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})</annotation></semantics></math>, we may approximate the <math id="S3.SS2.p2.2.m2.1" class="ltx_Math" alttext="\operatorname*{arg\,min}" display="inline"><semantics id="S3.SS2.p2.2.m2.1a"><mrow id="S3.SS2.p2.2.m2.1.1" xref="S3.SS2.p2.2.m2.1.1.cmml"><mi id="S3.SS2.p2.2.m2.1.1.2" xref="S3.SS2.p2.2.m2.1.1.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p2.2.m2.1.1.1" xref="S3.SS2.p2.2.m2.1.1.1.cmml">​</mo><mi id="S3.SS2.p2.2.m2.1.1.3" xref="S3.SS2.p2.2.m2.1.1.3.cmml">min</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.2.m2.1b"><apply id="S3.SS2.p2.2.m2.1.1.cmml" xref="S3.SS2.p2.2.m2.1.1"><times id="S3.SS2.p2.2.m2.1.1.1.cmml" xref="S3.SS2.p2.2.m2.1.1.1"></times><ci id="S3.SS2.p2.2.m2.1.1.2.cmml" xref="S3.SS2.p2.2.m2.1.1.2">arg</ci><ci id="S3.SS2.p2.2.m2.1.1.3.cmml" xref="S3.SS2.p2.2.m2.1.1.3">min</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.2.m2.1c">\operatorname*{arg\,min}</annotation></semantics></math> operation with respect to <math id="S3.SS2.p2.3.m3.1" class="ltx_Math" alttext="{\bm{y}}_{i}" display="inline"><semantics id="S3.SS2.p2.3.m3.1a"><msub id="S3.SS2.p2.3.m3.1.1" xref="S3.SS2.p2.3.m3.1.1.cmml"><mi id="S3.SS2.p2.3.m3.1.1.2" xref="S3.SS2.p2.3.m3.1.1.2.cmml">𝒚</mi><mi id="S3.SS2.p2.3.m3.1.1.3" xref="S3.SS2.p2.3.m3.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.3.m3.1b"><apply id="S3.SS2.p2.3.m3.1.1.cmml" xref="S3.SS2.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.3.m3.1.1.1.cmml" xref="S3.SS2.p2.3.m3.1.1">subscript</csymbol><ci id="S3.SS2.p2.3.m3.1.1.2.cmml" xref="S3.SS2.p2.3.m3.1.1.2">𝒚</ci><ci id="S3.SS2.p2.3.m3.1.1.3.cmml" xref="S3.SS2.p2.3.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.3.m3.1c">{\bm{y}}_{i}</annotation></semantics></math> via <math id="S3.SS2.p2.4.m4.1" class="ltx_Math" alttext="N" display="inline"><semantics id="S3.SS2.p2.4.m4.1a"><mi id="S3.SS2.p2.4.m4.1.1" xref="S3.SS2.p2.4.m4.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.4.m4.1b"><ci id="S3.SS2.p2.4.m4.1.1.cmml" xref="S3.SS2.p2.4.m4.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.4.m4.1c">N</annotation></semantics></math> steps of gradient optimization. An approximate optimum <math id="S3.SS2.p2.5.m5.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}" display="inline"><semantics id="S3.SS2.p2.5.m5.1a"><msubsup id="S3.SS2.p2.5.m5.1.1" xref="S3.SS2.p2.5.m5.1.1.cmml"><mi id="S3.SS2.p2.5.m5.1.1.2.2" xref="S3.SS2.p2.5.m5.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p2.5.m5.1.1.2.3" xref="S3.SS2.p2.5.m5.1.1.2.3.cmml">i</mi><mi id="S3.SS2.p2.5.m5.1.1.3" xref="S3.SS2.p2.5.m5.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.5.m5.1b"><apply id="S3.SS2.p2.5.m5.1.1.cmml" xref="S3.SS2.p2.5.m5.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.5.m5.1.1.1.cmml" xref="S3.SS2.p2.5.m5.1.1">superscript</csymbol><apply id="S3.SS2.p2.5.m5.1.1.2.cmml" xref="S3.SS2.p2.5.m5.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.5.m5.1.1.2.1.cmml" xref="S3.SS2.p2.5.m5.1.1">subscript</csymbol><ci id="S3.SS2.p2.5.m5.1.1.2.2.cmml" xref="S3.SS2.p2.5.m5.1.1.2.2">𝒚</ci><ci id="S3.SS2.p2.5.m5.1.1.2.3.cmml" xref="S3.SS2.p2.5.m5.1.1.2.3">𝑖</ci></apply><ci id="S3.SS2.p2.5.m5.1.1.3.cmml" xref="S3.SS2.p2.5.m5.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.5.m5.1c">{\bm{y}}_{i}^{N}</annotation></semantics></math> is obtained by:</p>
<table id="S3.E5" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E5.m1.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}={\bm{y}}_{i}^{N-1}-\lambda\nabla_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}}_{i}^{N-1})." display="block"><semantics id="S3.E5.m1.1a"><mrow id="S3.E5.m1.1.1.1" xref="S3.E5.m1.1.1.1.1.cmml"><mrow id="S3.E5.m1.1.1.1.1" xref="S3.E5.m1.1.1.1.1.cmml"><msubsup id="S3.E5.m1.1.1.1.1.4" xref="S3.E5.m1.1.1.1.1.4.cmml"><mi id="S3.E5.m1.1.1.1.1.4.2.2" xref="S3.E5.m1.1.1.1.1.4.2.2.cmml">𝒚</mi><mi id="S3.E5.m1.1.1.1.1.4.2.3" xref="S3.E5.m1.1.1.1.1.4.2.3.cmml">i</mi><mi id="S3.E5.m1.1.1.1.1.4.3" xref="S3.E5.m1.1.1.1.1.4.3.cmml">N</mi></msubsup><mo id="S3.E5.m1.1.1.1.1.3" xref="S3.E5.m1.1.1.1.1.3.cmml">=</mo><mrow id="S3.E5.m1.1.1.1.1.2" xref="S3.E5.m1.1.1.1.1.2.cmml"><msubsup id="S3.E5.m1.1.1.1.1.2.4" xref="S3.E5.m1.1.1.1.1.2.4.cmml"><mi id="S3.E5.m1.1.1.1.1.2.4.2.2" xref="S3.E5.m1.1.1.1.1.2.4.2.2.cmml">𝒚</mi><mi id="S3.E5.m1.1.1.1.1.2.4.2.3" xref="S3.E5.m1.1.1.1.1.2.4.2.3.cmml">i</mi><mrow id="S3.E5.m1.1.1.1.1.2.4.3" xref="S3.E5.m1.1.1.1.1.2.4.3.cmml"><mi id="S3.E5.m1.1.1.1.1.2.4.3.2" xref="S3.E5.m1.1.1.1.1.2.4.3.2.cmml">N</mi><mo id="S3.E5.m1.1.1.1.1.2.4.3.1" xref="S3.E5.m1.1.1.1.1.2.4.3.1.cmml">−</mo><mn id="S3.E5.m1.1.1.1.1.2.4.3.3" xref="S3.E5.m1.1.1.1.1.2.4.3.3.cmml">1</mn></mrow></msubsup><mo id="S3.E5.m1.1.1.1.1.2.3" xref="S3.E5.m1.1.1.1.1.2.3.cmml">−</mo><mrow id="S3.E5.m1.1.1.1.1.2.2" xref="S3.E5.m1.1.1.1.1.2.2.cmml"><mi id="S3.E5.m1.1.1.1.1.2.2.4" xref="S3.E5.m1.1.1.1.1.2.2.4.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="S3.E5.m1.1.1.1.1.2.2.3" xref="S3.E5.m1.1.1.1.1.2.2.3.cmml">​</mo><mrow id="S3.E5.m1.1.1.1.1.2.2.5" xref="S3.E5.m1.1.1.1.1.2.2.5.cmml"><msub id="S3.E5.m1.1.1.1.1.2.2.5.1" xref="S3.E5.m1.1.1.1.1.2.2.5.1.cmml"><mo rspace="0.167em" id="S3.E5.m1.1.1.1.1.2.2.5.1.2" xref="S3.E5.m1.1.1.1.1.2.2.5.1.2.cmml">∇</mo><mi id="S3.E5.m1.1.1.1.1.2.2.5.1.3" xref="S3.E5.m1.1.1.1.1.2.2.5.1.3.cmml">𝒚</mi></msub><msub id="S3.E5.m1.1.1.1.1.2.2.5.2" xref="S3.E5.m1.1.1.1.1.2.2.5.2.cmml"><mi id="S3.E5.m1.1.1.1.1.2.2.5.2.2" xref="S3.E5.m1.1.1.1.1.2.2.5.2.2.cmml">E</mi><mi id="S3.E5.m1.1.1.1.1.2.2.5.2.3" xref="S3.E5.m1.1.1.1.1.2.2.5.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.E5.m1.1.1.1.1.2.2.3a" xref="S3.E5.m1.1.1.1.1.2.2.3.cmml">​</mo><mrow id="S3.E5.m1.1.1.1.1.2.2.2.2" xref="S3.E5.m1.1.1.1.1.2.2.2.3.cmml"><mo stretchy="false" id="S3.E5.m1.1.1.1.1.2.2.2.2.3" xref="S3.E5.m1.1.1.1.1.2.2.2.3.cmml">(</mo><msub id="S3.E5.m1.1.1.1.1.1.1.1.1.1" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S3.E5.m1.1.1.1.1.1.1.1.1.1.2" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi id="S3.E5.m1.1.1.1.1.1.1.1.1.1.3" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo id="S3.E5.m1.1.1.1.1.2.2.2.2.4" xref="S3.E5.m1.1.1.1.1.2.2.2.3.cmml">,</mo><msubsup id="S3.E5.m1.1.1.1.1.2.2.2.2.2" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.cmml"><mi id="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.2" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.2.cmml">𝒚</mi><mi id="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.3" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.3.cmml">i</mi><mrow id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.cmml"><mi id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.2" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.2.cmml">N</mi><mo id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.1" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.1.cmml">−</mo><mn id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.3" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.3.cmml">1</mn></mrow></msubsup><mo stretchy="false" id="S3.E5.m1.1.1.1.1.2.2.2.2.5" xref="S3.E5.m1.1.1.1.1.2.2.2.3.cmml">)</mo></mrow></mrow></mrow></mrow><mo lspace="0em" id="S3.E5.m1.1.1.1.2" xref="S3.E5.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.E5.m1.1b"><apply id="S3.E5.m1.1.1.1.1.cmml" xref="S3.E5.m1.1.1.1"><eq id="S3.E5.m1.1.1.1.1.3.cmml" xref="S3.E5.m1.1.1.1.1.3"></eq><apply id="S3.E5.m1.1.1.1.1.4.cmml" xref="S3.E5.m1.1.1.1.1.4"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.4.1.cmml" xref="S3.E5.m1.1.1.1.1.4">superscript</csymbol><apply id="S3.E5.m1.1.1.1.1.4.2.cmml" xref="S3.E5.m1.1.1.1.1.4"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.4.2.1.cmml" xref="S3.E5.m1.1.1.1.1.4">subscript</csymbol><ci id="S3.E5.m1.1.1.1.1.4.2.2.cmml" xref="S3.E5.m1.1.1.1.1.4.2.2">𝒚</ci><ci id="S3.E5.m1.1.1.1.1.4.2.3.cmml" xref="S3.E5.m1.1.1.1.1.4.2.3">𝑖</ci></apply><ci id="S3.E5.m1.1.1.1.1.4.3.cmml" xref="S3.E5.m1.1.1.1.1.4.3">𝑁</ci></apply><apply id="S3.E5.m1.1.1.1.1.2.cmml" xref="S3.E5.m1.1.1.1.1.2"><minus id="S3.E5.m1.1.1.1.1.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.3"></minus><apply id="S3.E5.m1.1.1.1.1.2.4.cmml" xref="S3.E5.m1.1.1.1.1.2.4"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.2.4.1.cmml" xref="S3.E5.m1.1.1.1.1.2.4">superscript</csymbol><apply id="S3.E5.m1.1.1.1.1.2.4.2.cmml" xref="S3.E5.m1.1.1.1.1.2.4"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.2.4.2.1.cmml" xref="S3.E5.m1.1.1.1.1.2.4">subscript</csymbol><ci id="S3.E5.m1.1.1.1.1.2.4.2.2.cmml" xref="S3.E5.m1.1.1.1.1.2.4.2.2">𝒚</ci><ci id="S3.E5.m1.1.1.1.1.2.4.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.4.2.3">𝑖</ci></apply><apply id="S3.E5.m1.1.1.1.1.2.4.3.cmml" xref="S3.E5.m1.1.1.1.1.2.4.3"><minus id="S3.E5.m1.1.1.1.1.2.4.3.1.cmml" xref="S3.E5.m1.1.1.1.1.2.4.3.1"></minus><ci id="S3.E5.m1.1.1.1.1.2.4.3.2.cmml" xref="S3.E5.m1.1.1.1.1.2.4.3.2">𝑁</ci><cn type="integer" id="S3.E5.m1.1.1.1.1.2.4.3.3.cmml" xref="S3.E5.m1.1.1.1.1.2.4.3.3">1</cn></apply></apply><apply id="S3.E5.m1.1.1.1.1.2.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2"><times id="S3.E5.m1.1.1.1.1.2.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.3"></times><ci id="S3.E5.m1.1.1.1.1.2.2.4.cmml" xref="S3.E5.m1.1.1.1.1.2.2.4">𝜆</ci><apply id="S3.E5.m1.1.1.1.1.2.2.5.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5"><apply id="S3.E5.m1.1.1.1.1.2.2.5.1.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.1"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.2.2.5.1.1.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.1">subscript</csymbol><ci id="S3.E5.m1.1.1.1.1.2.2.5.1.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.1.2">∇</ci><ci id="S3.E5.m1.1.1.1.1.2.2.5.1.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.1.3">𝒚</ci></apply><apply id="S3.E5.m1.1.1.1.1.2.2.5.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.2"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.2.2.5.2.1.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.2">subscript</csymbol><ci id="S3.E5.m1.1.1.1.1.2.2.5.2.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.2.2">𝐸</ci><ci id="S3.E5.m1.1.1.1.1.2.2.5.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.5.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.E5.m1.1.1.1.1.2.2.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2"><apply id="S3.E5.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S3.E5.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="S3.E5.m1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S3.E5.m1.1.1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="S3.E5.m1.1.1.1.1.2.2.2.2.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.2.2.2.2.2.1.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2">superscript</csymbol><apply id="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2"><csymbol cd="ambiguous" id="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.1.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2">subscript</csymbol><ci id="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.2">𝒚</ci><ci id="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3"><minus id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.1.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.1"></minus><ci id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.2.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.2">𝑁</ci><cn type="integer" id="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.3.cmml" xref="S3.E5.m1.1.1.1.1.2.2.2.2.2.3.3">1</cn></apply></apply></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E5.m1.1c">{\bm{y}}_{i}^{N}={\bm{y}}_{i}^{N-1}-\lambda\nabla_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}}_{i}^{N-1}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(5)</span></td>
</tr></tbody>
</table>
<p id="S3.SS2.p2.9" class="ltx_p">In Equation <a href="#S3.E5" title="In 3.2 Learning Energy Landscapes for Reasoning ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, we initialize <math id="S3.SS2.p2.6.m1.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{0}" display="inline"><semantics id="S3.SS2.p2.6.m1.1a"><msubsup id="S3.SS2.p2.6.m1.1.1" xref="S3.SS2.p2.6.m1.1.1.cmml"><mi id="S3.SS2.p2.6.m1.1.1.2.2" xref="S3.SS2.p2.6.m1.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p2.6.m1.1.1.2.3" xref="S3.SS2.p2.6.m1.1.1.2.3.cmml">i</mi><mn id="S3.SS2.p2.6.m1.1.1.3" xref="S3.SS2.p2.6.m1.1.1.3.cmml">0</mn></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.6.m1.1b"><apply id="S3.SS2.p2.6.m1.1.1.cmml" xref="S3.SS2.p2.6.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.6.m1.1.1.1.cmml" xref="S3.SS2.p2.6.m1.1.1">superscript</csymbol><apply id="S3.SS2.p2.6.m1.1.1.2.cmml" xref="S3.SS2.p2.6.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.6.m1.1.1.2.1.cmml" xref="S3.SS2.p2.6.m1.1.1">subscript</csymbol><ci id="S3.SS2.p2.6.m1.1.1.2.2.cmml" xref="S3.SS2.p2.6.m1.1.1.2.2">𝒚</ci><ci id="S3.SS2.p2.6.m1.1.1.2.3.cmml" xref="S3.SS2.p2.6.m1.1.1.2.3">𝑖</ci></apply><cn type="integer" id="S3.SS2.p2.6.m1.1.1.3.cmml" xref="S3.SS2.p2.6.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.6.m1.1c">{\bm{y}}_{i}^{0}</annotation></semantics></math> using uniform noise, with <math id="S3.SS2.p2.7.m2.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="S3.SS2.p2.7.m2.1a"><mi id="S3.SS2.p2.7.m2.1.1" xref="S3.SS2.p2.7.m2.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.7.m2.1b"><ci id="S3.SS2.p2.7.m2.1.1.cmml" xref="S3.SS2.p2.7.m2.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.7.m2.1c">\lambda</annotation></semantics></math> denoting the step size for each gradient step, and sample <math id="S3.SS2.p2.8.m3.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}" display="inline"><semantics id="S3.SS2.p2.8.m3.1a"><msubsup id="S3.SS2.p2.8.m3.1.1" xref="S3.SS2.p2.8.m3.1.1.cmml"><mi id="S3.SS2.p2.8.m3.1.1.2.2" xref="S3.SS2.p2.8.m3.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p2.8.m3.1.1.2.3" xref="S3.SS2.p2.8.m3.1.1.2.3.cmml">i</mi><mi id="S3.SS2.p2.8.m3.1.1.3" xref="S3.SS2.p2.8.m3.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.8.m3.1b"><apply id="S3.SS2.p2.8.m3.1.1.cmml" xref="S3.SS2.p2.8.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.8.m3.1.1.1.cmml" xref="S3.SS2.p2.8.m3.1.1">superscript</csymbol><apply id="S3.SS2.p2.8.m3.1.1.2.cmml" xref="S3.SS2.p2.8.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p2.8.m3.1.1.2.1.cmml" xref="S3.SS2.p2.8.m3.1.1">subscript</csymbol><ci id="S3.SS2.p2.8.m3.1.1.2.2.cmml" xref="S3.SS2.p2.8.m3.1.1.2.2">𝒚</ci><ci id="S3.SS2.p2.8.m3.1.1.2.3.cmml" xref="S3.SS2.p2.8.m3.1.1.2.3">𝑖</ci></apply><ci id="S3.SS2.p2.8.m3.1.1.3.cmml" xref="S3.SS2.p2.8.m3.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.8.m3.1c">{\bm{y}}_{i}^{N}</annotation></semantics></math> corresponding to the result after <math id="S3.SS2.p2.9.m4.1" class="ltx_Math" alttext="N" display="inline"><semantics id="S3.SS2.p2.9.m4.1a"><mi id="S3.SS2.p2.9.m4.1.1" xref="S3.SS2.p2.9.m4.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.p2.9.m4.1b"><ci id="S3.SS2.p2.9.m4.1.1.cmml" xref="S3.SS2.p2.9.m4.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p2.9.m4.1c">N</annotation></semantics></math> steps of gradient descent. We then minimize the corresponding loss:</p>
<table id="S3.E6" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E6.m1.2" class="ltx_Math" alttext="\mathcal{L}_{\text{MSE}}(\theta)=\|{\bm{y}}_{i}^{N}-{\bm{y}}_{i}\|^{2}," display="block"><semantics id="S3.E6.m1.2a"><mrow id="S3.E6.m1.2.2.1" xref="S3.E6.m1.2.2.1.1.cmml"><mrow id="S3.E6.m1.2.2.1.1" xref="S3.E6.m1.2.2.1.1.cmml"><mrow id="S3.E6.m1.2.2.1.1.3" xref="S3.E6.m1.2.2.1.1.3.cmml"><msub id="S3.E6.m1.2.2.1.1.3.2" xref="S3.E6.m1.2.2.1.1.3.2.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.E6.m1.2.2.1.1.3.2.2" xref="S3.E6.m1.2.2.1.1.3.2.2.cmml">ℒ</mi><mtext id="S3.E6.m1.2.2.1.1.3.2.3" xref="S3.E6.m1.2.2.1.1.3.2.3a.cmml">MSE</mtext></msub><mo lspace="0em" rspace="0em" id="S3.E6.m1.2.2.1.1.3.1" xref="S3.E6.m1.2.2.1.1.3.1.cmml">​</mo><mrow id="S3.E6.m1.2.2.1.1.3.3.2" xref="S3.E6.m1.2.2.1.1.3.cmml"><mo stretchy="false" id="S3.E6.m1.2.2.1.1.3.3.2.1" xref="S3.E6.m1.2.2.1.1.3.cmml">(</mo><mi id="S3.E6.m1.1.1" xref="S3.E6.m1.1.1.cmml">θ</mi><mo stretchy="false" id="S3.E6.m1.2.2.1.1.3.3.2.2" xref="S3.E6.m1.2.2.1.1.3.cmml">)</mo></mrow></mrow><mo id="S3.E6.m1.2.2.1.1.2" xref="S3.E6.m1.2.2.1.1.2.cmml">=</mo><msup id="S3.E6.m1.2.2.1.1.1" xref="S3.E6.m1.2.2.1.1.1.cmml"><mrow id="S3.E6.m1.2.2.1.1.1.1.1" xref="S3.E6.m1.2.2.1.1.1.1.2.cmml"><mo stretchy="false" id="S3.E6.m1.2.2.1.1.1.1.1.2" xref="S3.E6.m1.2.2.1.1.1.1.2.1.cmml">‖</mo><mrow id="S3.E6.m1.2.2.1.1.1.1.1.1" xref="S3.E6.m1.2.2.1.1.1.1.1.1.cmml"><msubsup id="S3.E6.m1.2.2.1.1.1.1.1.1.2" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.cmml"><mi id="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.2" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.2.cmml">𝒚</mi><mi id="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.3" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.3.cmml">i</mi><mi id="S3.E6.m1.2.2.1.1.1.1.1.1.2.3" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.3.cmml">N</mi></msubsup><mo id="S3.E6.m1.2.2.1.1.1.1.1.1.1" xref="S3.E6.m1.2.2.1.1.1.1.1.1.1.cmml">−</mo><msub id="S3.E6.m1.2.2.1.1.1.1.1.1.3" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3.cmml"><mi id="S3.E6.m1.2.2.1.1.1.1.1.1.3.2" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3.2.cmml">𝒚</mi><mi id="S3.E6.m1.2.2.1.1.1.1.1.1.3.3" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo stretchy="false" id="S3.E6.m1.2.2.1.1.1.1.1.3" xref="S3.E6.m1.2.2.1.1.1.1.2.1.cmml">‖</mo></mrow><mn id="S3.E6.m1.2.2.1.1.1.3" xref="S3.E6.m1.2.2.1.1.1.3.cmml">2</mn></msup></mrow><mo id="S3.E6.m1.2.2.1.2" xref="S3.E6.m1.2.2.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.E6.m1.2b"><apply id="S3.E6.m1.2.2.1.1.cmml" xref="S3.E6.m1.2.2.1"><eq id="S3.E6.m1.2.2.1.1.2.cmml" xref="S3.E6.m1.2.2.1.1.2"></eq><apply id="S3.E6.m1.2.2.1.1.3.cmml" xref="S3.E6.m1.2.2.1.1.3"><times id="S3.E6.m1.2.2.1.1.3.1.cmml" xref="S3.E6.m1.2.2.1.1.3.1"></times><apply id="S3.E6.m1.2.2.1.1.3.2.cmml" xref="S3.E6.m1.2.2.1.1.3.2"><csymbol cd="ambiguous" id="S3.E6.m1.2.2.1.1.3.2.1.cmml" xref="S3.E6.m1.2.2.1.1.3.2">subscript</csymbol><ci id="S3.E6.m1.2.2.1.1.3.2.2.cmml" xref="S3.E6.m1.2.2.1.1.3.2.2">ℒ</ci><ci id="S3.E6.m1.2.2.1.1.3.2.3a.cmml" xref="S3.E6.m1.2.2.1.1.3.2.3"><mtext mathsize="70%" id="S3.E6.m1.2.2.1.1.3.2.3.cmml" xref="S3.E6.m1.2.2.1.1.3.2.3">MSE</mtext></ci></apply><ci id="S3.E6.m1.1.1.cmml" xref="S3.E6.m1.1.1">𝜃</ci></apply><apply id="S3.E6.m1.2.2.1.1.1.cmml" xref="S3.E6.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.E6.m1.2.2.1.1.1.2.cmml" xref="S3.E6.m1.2.2.1.1.1">superscript</csymbol><apply id="S3.E6.m1.2.2.1.1.1.1.2.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1"><csymbol cd="latexml" id="S3.E6.m1.2.2.1.1.1.1.2.1.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.2">norm</csymbol><apply id="S3.E6.m1.2.2.1.1.1.1.1.1.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1"><minus id="S3.E6.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.1"></minus><apply id="S3.E6.m1.2.2.1.1.1.1.1.1.2.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S3.E6.m1.2.2.1.1.1.1.1.1.2.1.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2">superscript</csymbol><apply id="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.1.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2">subscript</csymbol><ci id="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.2.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.2">𝒚</ci><ci id="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.3.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.2.3">𝑖</ci></apply><ci id="S3.E6.m1.2.2.1.1.1.1.1.1.2.3.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.2.3">𝑁</ci></apply><apply id="S3.E6.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="S3.E6.m1.2.2.1.1.1.1.1.1.3.1.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3">subscript</csymbol><ci id="S3.E6.m1.2.2.1.1.1.1.1.1.3.2.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3.2">𝒚</ci><ci id="S3.E6.m1.2.2.1.1.1.1.1.1.3.3.cmml" xref="S3.E6.m1.2.2.1.1.1.1.1.1.3.3">𝑖</ci></apply></apply></apply><cn type="integer" id="S3.E6.m1.2.2.1.1.1.3.cmml" xref="S3.E6.m1.2.2.1.1.1.3">2</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E6.m1.2c">\mathcal{L}_{\text{MSE}}(\theta)=\|{\bm{y}}_{i}^{N}-{\bm{y}}_{i}\|^{2},</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(6)</span></td>
</tr></tbody>
</table>
<p id="S3.SS2.p2.10" class="ltx_p">where we may directly differentiate through the underlying optimization procedure <cite class="ltx_cite ltx_citemacro_citep">(Finn et al., <a href="#bib.bib18" title="" class="ltx_ref">2017</a>)</cite>. To alleviate the computational burden of computing second-order gradients across optimization steps, we empirically found that simply truncating back-propagation to the last optimization step maintained good performance at a significantly faster training speed as illustrated in Table <a href="#S4.T2" title="Table 2 ‣ 4.2 Graphical Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>.</p>
</div>
<div id="S3.SS2.p3" class="ltx_para">
<p id="S3.SS2.p3.5" class="ltx_p">An underlying difficulty of utilizing <math id="S3.SS2.p3.1.m1.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}" display="inline"><semantics id="S3.SS2.p3.1.m1.1a"><msubsup id="S3.SS2.p3.1.m1.1.1" xref="S3.SS2.p3.1.m1.1.1.cmml"><mi id="S3.SS2.p3.1.m1.1.1.2.2" xref="S3.SS2.p3.1.m1.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p3.1.m1.1.1.2.3" xref="S3.SS2.p3.1.m1.1.1.2.3.cmml">i</mi><mi id="S3.SS2.p3.1.m1.1.1.3" xref="S3.SS2.p3.1.m1.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.1.m1.1b"><apply id="S3.SS2.p3.1.m1.1.1.cmml" xref="S3.SS2.p3.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.1.m1.1.1.1.cmml" xref="S3.SS2.p3.1.m1.1.1">superscript</csymbol><apply id="S3.SS2.p3.1.m1.1.1.2.cmml" xref="S3.SS2.p3.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.1.m1.1.1.2.1.cmml" xref="S3.SS2.p3.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p3.1.m1.1.1.2.2.cmml" xref="S3.SS2.p3.1.m1.1.1.2.2">𝒚</ci><ci id="S3.SS2.p3.1.m1.1.1.2.3.cmml" xref="S3.SS2.p3.1.m1.1.1.2.3">𝑖</ci></apply><ci id="S3.SS2.p3.1.m1.1.1.3.cmml" xref="S3.SS2.p3.1.m1.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.1.m1.1c">{\bm{y}}_{i}^{N}</annotation></semantics></math> as an approximation of <math id="S3.SS2.p3.2.m2.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})" display="inline"><semantics id="S3.SS2.p3.2.m2.2a"><mrow id="S3.SS2.p3.2.m2.2.2" xref="S3.SS2.p3.2.m2.2.2.cmml"><mrow id="S3.SS2.p3.2.m2.2.2.3" xref="S3.SS2.p3.2.m2.2.2.3.cmml"><msub id="S3.SS2.p3.2.m2.2.2.3.1" xref="S3.SS2.p3.2.m2.2.2.3.1.cmml"><mrow id="S3.SS2.p3.2.m2.2.2.3.1.2" xref="S3.SS2.p3.2.m2.2.2.3.1.2.cmml"><mi id="S3.SS2.p3.2.m2.2.2.3.1.2.2" xref="S3.SS2.p3.2.m2.2.2.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p3.2.m2.2.2.3.1.2.1" xref="S3.SS2.p3.2.m2.2.2.3.1.2.1.cmml">​</mo><mi id="S3.SS2.p3.2.m2.2.2.3.1.2.3" xref="S3.SS2.p3.2.m2.2.2.3.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p3.2.m2.2.2.3.1.3" xref="S3.SS2.p3.2.m2.2.2.3.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p3.2.m2.2.2.3a" xref="S3.SS2.p3.2.m2.2.2.3.cmml">⁡</mo><msub id="S3.SS2.p3.2.m2.2.2.3.2" xref="S3.SS2.p3.2.m2.2.2.3.2.cmml"><mi id="S3.SS2.p3.2.m2.2.2.3.2.2" xref="S3.SS2.p3.2.m2.2.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p3.2.m2.2.2.3.2.3" xref="S3.SS2.p3.2.m2.2.2.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p3.2.m2.2.2.2" xref="S3.SS2.p3.2.m2.2.2.2.cmml">​</mo><mrow id="S3.SS2.p3.2.m2.2.2.1.1" xref="S3.SS2.p3.2.m2.2.2.1.2.cmml"><mo stretchy="false" id="S3.SS2.p3.2.m2.2.2.1.1.2" xref="S3.SS2.p3.2.m2.2.2.1.2.cmml">(</mo><msub id="S3.SS2.p3.2.m2.2.2.1.1.1" xref="S3.SS2.p3.2.m2.2.2.1.1.1.cmml"><mi id="S3.SS2.p3.2.m2.2.2.1.1.1.2" xref="S3.SS2.p3.2.m2.2.2.1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p3.2.m2.2.2.1.1.1.3" xref="S3.SS2.p3.2.m2.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S3.SS2.p3.2.m2.2.2.1.1.3" xref="S3.SS2.p3.2.m2.2.2.1.2.cmml">,</mo><mi id="S3.SS2.p3.2.m2.1.1" xref="S3.SS2.p3.2.m2.1.1.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p3.2.m2.2.2.1.1.4" xref="S3.SS2.p3.2.m2.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.2.m2.2b"><apply id="S3.SS2.p3.2.m2.2.2.cmml" xref="S3.SS2.p3.2.m2.2.2"><times id="S3.SS2.p3.2.m2.2.2.2.cmml" xref="S3.SS2.p3.2.m2.2.2.2"></times><apply id="S3.SS2.p3.2.m2.2.2.3.cmml" xref="S3.SS2.p3.2.m2.2.2.3"><apply id="S3.SS2.p3.2.m2.2.2.3.1.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1"><csymbol cd="ambiguous" id="S3.SS2.p3.2.m2.2.2.3.1.1.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1">subscript</csymbol><apply id="S3.SS2.p3.2.m2.2.2.3.1.2.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1.2"><times id="S3.SS2.p3.2.m2.2.2.3.1.2.1.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1.2.1"></times><ci id="S3.SS2.p3.2.m2.2.2.3.1.2.2.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1.2.2">arg</ci><ci id="S3.SS2.p3.2.m2.2.2.3.1.2.3.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1.2.3">min</ci></apply><ci id="S3.SS2.p3.2.m2.2.2.3.1.3.cmml" xref="S3.SS2.p3.2.m2.2.2.3.1.3">𝒚</ci></apply><apply id="S3.SS2.p3.2.m2.2.2.3.2.cmml" xref="S3.SS2.p3.2.m2.2.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p3.2.m2.2.2.3.2.1.cmml" xref="S3.SS2.p3.2.m2.2.2.3.2">subscript</csymbol><ci id="S3.SS2.p3.2.m2.2.2.3.2.2.cmml" xref="S3.SS2.p3.2.m2.2.2.3.2.2">𝐸</ci><ci id="S3.SS2.p3.2.m2.2.2.3.2.3.cmml" xref="S3.SS2.p3.2.m2.2.2.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p3.2.m2.2.2.1.2.cmml" xref="S3.SS2.p3.2.m2.2.2.1.1"><apply id="S3.SS2.p3.2.m2.2.2.1.1.1.cmml" xref="S3.SS2.p3.2.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.2.m2.2.2.1.1.1.1.cmml" xref="S3.SS2.p3.2.m2.2.2.1.1.1">subscript</csymbol><ci id="S3.SS2.p3.2.m2.2.2.1.1.1.2.cmml" xref="S3.SS2.p3.2.m2.2.2.1.1.1.2">𝒙</ci><ci id="S3.SS2.p3.2.m2.2.2.1.1.1.3.cmml" xref="S3.SS2.p3.2.m2.2.2.1.1.1.3">𝑖</ci></apply><ci id="S3.SS2.p3.2.m2.1.1.cmml" xref="S3.SS2.p3.2.m2.1.1">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.2.m2.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})</annotation></semantics></math> is that since only a small finite number of steps of gradient descent is applied, <math id="S3.SS2.p3.3.m3.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}" display="inline"><semantics id="S3.SS2.p3.3.m3.1a"><msubsup id="S3.SS2.p3.3.m3.1.1" xref="S3.SS2.p3.3.m3.1.1.cmml"><mi id="S3.SS2.p3.3.m3.1.1.2.2" xref="S3.SS2.p3.3.m3.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p3.3.m3.1.1.2.3" xref="S3.SS2.p3.3.m3.1.1.2.3.cmml">i</mi><mi id="S3.SS2.p3.3.m3.1.1.3" xref="S3.SS2.p3.3.m3.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.3.m3.1b"><apply id="S3.SS2.p3.3.m3.1.1.cmml" xref="S3.SS2.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.3.m3.1.1.1.cmml" xref="S3.SS2.p3.3.m3.1.1">superscript</csymbol><apply id="S3.SS2.p3.3.m3.1.1.2.cmml" xref="S3.SS2.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.3.m3.1.1.2.1.cmml" xref="S3.SS2.p3.3.m3.1.1">subscript</csymbol><ci id="S3.SS2.p3.3.m3.1.1.2.2.cmml" xref="S3.SS2.p3.3.m3.1.1.2.2">𝒚</ci><ci id="S3.SS2.p3.3.m3.1.1.2.3.cmml" xref="S3.SS2.p3.3.m3.1.1.2.3">𝑖</ci></apply><ci id="S3.SS2.p3.3.m3.1.1.3.cmml" xref="S3.SS2.p3.3.m3.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.3.m3.1c">{\bm{y}}_{i}^{N}</annotation></semantics></math> may be far from reaching <math id="S3.SS2.p3.4.m4.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})" display="inline"><semantics id="S3.SS2.p3.4.m4.2a"><mrow id="S3.SS2.p3.4.m4.2.2" xref="S3.SS2.p3.4.m4.2.2.cmml"><mrow id="S3.SS2.p3.4.m4.2.2.3" xref="S3.SS2.p3.4.m4.2.2.3.cmml"><msub id="S3.SS2.p3.4.m4.2.2.3.1" xref="S3.SS2.p3.4.m4.2.2.3.1.cmml"><mrow id="S3.SS2.p3.4.m4.2.2.3.1.2" xref="S3.SS2.p3.4.m4.2.2.3.1.2.cmml"><mi id="S3.SS2.p3.4.m4.2.2.3.1.2.2" xref="S3.SS2.p3.4.m4.2.2.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p3.4.m4.2.2.3.1.2.1" xref="S3.SS2.p3.4.m4.2.2.3.1.2.1.cmml">​</mo><mi id="S3.SS2.p3.4.m4.2.2.3.1.2.3" xref="S3.SS2.p3.4.m4.2.2.3.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p3.4.m4.2.2.3.1.3" xref="S3.SS2.p3.4.m4.2.2.3.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p3.4.m4.2.2.3a" xref="S3.SS2.p3.4.m4.2.2.3.cmml">⁡</mo><msub id="S3.SS2.p3.4.m4.2.2.3.2" xref="S3.SS2.p3.4.m4.2.2.3.2.cmml"><mi id="S3.SS2.p3.4.m4.2.2.3.2.2" xref="S3.SS2.p3.4.m4.2.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p3.4.m4.2.2.3.2.3" xref="S3.SS2.p3.4.m4.2.2.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p3.4.m4.2.2.2" xref="S3.SS2.p3.4.m4.2.2.2.cmml">​</mo><mrow id="S3.SS2.p3.4.m4.2.2.1.1" xref="S3.SS2.p3.4.m4.2.2.1.2.cmml"><mo stretchy="false" id="S3.SS2.p3.4.m4.2.2.1.1.2" xref="S3.SS2.p3.4.m4.2.2.1.2.cmml">(</mo><msub id="S3.SS2.p3.4.m4.2.2.1.1.1" xref="S3.SS2.p3.4.m4.2.2.1.1.1.cmml"><mi id="S3.SS2.p3.4.m4.2.2.1.1.1.2" xref="S3.SS2.p3.4.m4.2.2.1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p3.4.m4.2.2.1.1.1.3" xref="S3.SS2.p3.4.m4.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S3.SS2.p3.4.m4.2.2.1.1.3" xref="S3.SS2.p3.4.m4.2.2.1.2.cmml">,</mo><mi id="S3.SS2.p3.4.m4.1.1" xref="S3.SS2.p3.4.m4.1.1.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p3.4.m4.2.2.1.1.4" xref="S3.SS2.p3.4.m4.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.4.m4.2b"><apply id="S3.SS2.p3.4.m4.2.2.cmml" xref="S3.SS2.p3.4.m4.2.2"><times id="S3.SS2.p3.4.m4.2.2.2.cmml" xref="S3.SS2.p3.4.m4.2.2.2"></times><apply id="S3.SS2.p3.4.m4.2.2.3.cmml" xref="S3.SS2.p3.4.m4.2.2.3"><apply id="S3.SS2.p3.4.m4.2.2.3.1.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1"><csymbol cd="ambiguous" id="S3.SS2.p3.4.m4.2.2.3.1.1.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1">subscript</csymbol><apply id="S3.SS2.p3.4.m4.2.2.3.1.2.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1.2"><times id="S3.SS2.p3.4.m4.2.2.3.1.2.1.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1.2.1"></times><ci id="S3.SS2.p3.4.m4.2.2.3.1.2.2.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1.2.2">arg</ci><ci id="S3.SS2.p3.4.m4.2.2.3.1.2.3.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1.2.3">min</ci></apply><ci id="S3.SS2.p3.4.m4.2.2.3.1.3.cmml" xref="S3.SS2.p3.4.m4.2.2.3.1.3">𝒚</ci></apply><apply id="S3.SS2.p3.4.m4.2.2.3.2.cmml" xref="S3.SS2.p3.4.m4.2.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p3.4.m4.2.2.3.2.1.cmml" xref="S3.SS2.p3.4.m4.2.2.3.2">subscript</csymbol><ci id="S3.SS2.p3.4.m4.2.2.3.2.2.cmml" xref="S3.SS2.p3.4.m4.2.2.3.2.2">𝐸</ci><ci id="S3.SS2.p3.4.m4.2.2.3.2.3.cmml" xref="S3.SS2.p3.4.m4.2.2.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p3.4.m4.2.2.1.2.cmml" xref="S3.SS2.p3.4.m4.2.2.1.1"><apply id="S3.SS2.p3.4.m4.2.2.1.1.1.cmml" xref="S3.SS2.p3.4.m4.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.4.m4.2.2.1.1.1.1.cmml" xref="S3.SS2.p3.4.m4.2.2.1.1.1">subscript</csymbol><ci id="S3.SS2.p3.4.m4.2.2.1.1.1.2.cmml" xref="S3.SS2.p3.4.m4.2.2.1.1.1.2">𝒙</ci><ci id="S3.SS2.p3.4.m4.2.2.1.1.1.3.cmml" xref="S3.SS2.p3.4.m4.2.2.1.1.1.3">𝑖</ci></apply><ci id="S3.SS2.p3.4.m4.1.1.cmml" xref="S3.SS2.p3.4.m4.1.1">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.4.m4.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})</annotation></semantics></math>. As a result, when running a larger number of iterative reasoning steps at test time to more precisely compute <math id="S3.SS2.p3.5.m5.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})" display="inline"><semantics id="S3.SS2.p3.5.m5.2a"><mrow id="S3.SS2.p3.5.m5.2.2" xref="S3.SS2.p3.5.m5.2.2.cmml"><mrow id="S3.SS2.p3.5.m5.2.2.3" xref="S3.SS2.p3.5.m5.2.2.3.cmml"><msub id="S3.SS2.p3.5.m5.2.2.3.1" xref="S3.SS2.p3.5.m5.2.2.3.1.cmml"><mrow id="S3.SS2.p3.5.m5.2.2.3.1.2" xref="S3.SS2.p3.5.m5.2.2.3.1.2.cmml"><mi id="S3.SS2.p3.5.m5.2.2.3.1.2.2" xref="S3.SS2.p3.5.m5.2.2.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p3.5.m5.2.2.3.1.2.1" xref="S3.SS2.p3.5.m5.2.2.3.1.2.1.cmml">​</mo><mi id="S3.SS2.p3.5.m5.2.2.3.1.2.3" xref="S3.SS2.p3.5.m5.2.2.3.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p3.5.m5.2.2.3.1.3" xref="S3.SS2.p3.5.m5.2.2.3.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p3.5.m5.2.2.3a" xref="S3.SS2.p3.5.m5.2.2.3.cmml">⁡</mo><msub id="S3.SS2.p3.5.m5.2.2.3.2" xref="S3.SS2.p3.5.m5.2.2.3.2.cmml"><mi id="S3.SS2.p3.5.m5.2.2.3.2.2" xref="S3.SS2.p3.5.m5.2.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p3.5.m5.2.2.3.2.3" xref="S3.SS2.p3.5.m5.2.2.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p3.5.m5.2.2.2" xref="S3.SS2.p3.5.m5.2.2.2.cmml">​</mo><mrow id="S3.SS2.p3.5.m5.2.2.1.1" xref="S3.SS2.p3.5.m5.2.2.1.2.cmml"><mo stretchy="false" id="S3.SS2.p3.5.m5.2.2.1.1.2" xref="S3.SS2.p3.5.m5.2.2.1.2.cmml">(</mo><msub id="S3.SS2.p3.5.m5.2.2.1.1.1" xref="S3.SS2.p3.5.m5.2.2.1.1.1.cmml"><mi id="S3.SS2.p3.5.m5.2.2.1.1.1.2" xref="S3.SS2.p3.5.m5.2.2.1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p3.5.m5.2.2.1.1.1.3" xref="S3.SS2.p3.5.m5.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S3.SS2.p3.5.m5.2.2.1.1.3" xref="S3.SS2.p3.5.m5.2.2.1.2.cmml">,</mo><mi id="S3.SS2.p3.5.m5.1.1" xref="S3.SS2.p3.5.m5.1.1.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p3.5.m5.2.2.1.1.4" xref="S3.SS2.p3.5.m5.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.5.m5.2b"><apply id="S3.SS2.p3.5.m5.2.2.cmml" xref="S3.SS2.p3.5.m5.2.2"><times id="S3.SS2.p3.5.m5.2.2.2.cmml" xref="S3.SS2.p3.5.m5.2.2.2"></times><apply id="S3.SS2.p3.5.m5.2.2.3.cmml" xref="S3.SS2.p3.5.m5.2.2.3"><apply id="S3.SS2.p3.5.m5.2.2.3.1.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1"><csymbol cd="ambiguous" id="S3.SS2.p3.5.m5.2.2.3.1.1.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1">subscript</csymbol><apply id="S3.SS2.p3.5.m5.2.2.3.1.2.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1.2"><times id="S3.SS2.p3.5.m5.2.2.3.1.2.1.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1.2.1"></times><ci id="S3.SS2.p3.5.m5.2.2.3.1.2.2.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1.2.2">arg</ci><ci id="S3.SS2.p3.5.m5.2.2.3.1.2.3.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1.2.3">min</ci></apply><ci id="S3.SS2.p3.5.m5.2.2.3.1.3.cmml" xref="S3.SS2.p3.5.m5.2.2.3.1.3">𝒚</ci></apply><apply id="S3.SS2.p3.5.m5.2.2.3.2.cmml" xref="S3.SS2.p3.5.m5.2.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p3.5.m5.2.2.3.2.1.cmml" xref="S3.SS2.p3.5.m5.2.2.3.2">subscript</csymbol><ci id="S3.SS2.p3.5.m5.2.2.3.2.2.cmml" xref="S3.SS2.p3.5.m5.2.2.3.2.2">𝐸</ci><ci id="S3.SS2.p3.5.m5.2.2.3.2.3.cmml" xref="S3.SS2.p3.5.m5.2.2.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p3.5.m5.2.2.1.2.cmml" xref="S3.SS2.p3.5.m5.2.2.1.1"><apply id="S3.SS2.p3.5.m5.2.2.1.1.1.cmml" xref="S3.SS2.p3.5.m5.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.5.m5.2.2.1.1.1.1.cmml" xref="S3.SS2.p3.5.m5.2.2.1.1.1">subscript</csymbol><ci id="S3.SS2.p3.5.m5.2.2.1.1.1.2.cmml" xref="S3.SS2.p3.5.m5.2.2.1.1.1.2">𝒙</ci><ci id="S3.SS2.p3.5.m5.2.2.1.1.1.3.cmml" xref="S3.SS2.p3.5.m5.2.2.1.1.1.3">𝑖</ci></apply><ci id="S3.SS2.p3.5.m5.1.1.cmml" xref="S3.SS2.p3.5.m5.1.1">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.5.m5.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})</annotation></semantics></math>, our underlying solution may degrade.</p>
</div>
<div id="S3.SS2.p4" class="ltx_para">
<p id="S3.SS2.p4.4" class="ltx_p">To remedy this issue, we maintain a replay buffer of previously optimized samples <math id="S3.SS2.p4.1.m1.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}" display="inline"><semantics id="S3.SS2.p4.1.m1.1a"><msubsup id="S3.SS2.p4.1.m1.1.1" xref="S3.SS2.p4.1.m1.1.1.cmml"><mi id="S3.SS2.p4.1.m1.1.1.2.2" xref="S3.SS2.p4.1.m1.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p4.1.m1.1.1.2.3" xref="S3.SS2.p4.1.m1.1.1.2.3.cmml">i</mi><mi id="S3.SS2.p4.1.m1.1.1.3" xref="S3.SS2.p4.1.m1.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.1.m1.1b"><apply id="S3.SS2.p4.1.m1.1.1.cmml" xref="S3.SS2.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.1.m1.1.1.1.cmml" xref="S3.SS2.p4.1.m1.1.1">superscript</csymbol><apply id="S3.SS2.p4.1.m1.1.1.2.cmml" xref="S3.SS2.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.1.m1.1.1.2.1.cmml" xref="S3.SS2.p4.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p4.1.m1.1.1.2.2.cmml" xref="S3.SS2.p4.1.m1.1.1.2.2">𝒚</ci><ci id="S3.SS2.p4.1.m1.1.1.2.3.cmml" xref="S3.SS2.p4.1.m1.1.1.2.3">𝑖</ci></apply><ci id="S3.SS2.p4.1.m1.1.1.3.cmml" xref="S3.SS2.p4.1.m1.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.1.m1.1c">{\bm{y}}_{i}^{N}</annotation></semantics></math>, and initialize <math id="S3.SS2.p4.2.m2.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{0}" display="inline"><semantics id="S3.SS2.p4.2.m2.1a"><msubsup id="S3.SS2.p4.2.m2.1.1" xref="S3.SS2.p4.2.m2.1.1.cmml"><mi id="S3.SS2.p4.2.m2.1.1.2.2" xref="S3.SS2.p4.2.m2.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p4.2.m2.1.1.2.3" xref="S3.SS2.p4.2.m2.1.1.2.3.cmml">i</mi><mn id="S3.SS2.p4.2.m2.1.1.3" xref="S3.SS2.p4.2.m2.1.1.3.cmml">0</mn></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.2.m2.1b"><apply id="S3.SS2.p4.2.m2.1.1.cmml" xref="S3.SS2.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.2.m2.1.1.1.cmml" xref="S3.SS2.p4.2.m2.1.1">superscript</csymbol><apply id="S3.SS2.p4.2.m2.1.1.2.cmml" xref="S3.SS2.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.2.m2.1.1.2.1.cmml" xref="S3.SS2.p4.2.m2.1.1">subscript</csymbol><ci id="S3.SS2.p4.2.m2.1.1.2.2.cmml" xref="S3.SS2.p4.2.m2.1.1.2.2">𝒚</ci><ci id="S3.SS2.p4.2.m2.1.1.2.3.cmml" xref="S3.SS2.p4.2.m2.1.1.2.3">𝑖</ci></apply><cn type="integer" id="S3.SS2.p4.2.m2.1.1.3.cmml" xref="S3.SS2.p4.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.2.m2.1c">{\bm{y}}_{i}^{0}</annotation></semantics></math> either from previously optimized values <math id="S3.SS2.p4.3.m3.1" class="ltx_Math" alttext="{\bm{y}}_{i}^{N}" display="inline"><semantics id="S3.SS2.p4.3.m3.1a"><msubsup id="S3.SS2.p4.3.m3.1.1" xref="S3.SS2.p4.3.m3.1.1.cmml"><mi id="S3.SS2.p4.3.m3.1.1.2.2" xref="S3.SS2.p4.3.m3.1.1.2.2.cmml">𝒚</mi><mi id="S3.SS2.p4.3.m3.1.1.2.3" xref="S3.SS2.p4.3.m3.1.1.2.3.cmml">i</mi><mi id="S3.SS2.p4.3.m3.1.1.3" xref="S3.SS2.p4.3.m3.1.1.3.cmml">N</mi></msubsup><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.3.m3.1b"><apply id="S3.SS2.p4.3.m3.1.1.cmml" xref="S3.SS2.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.3.m3.1.1.1.cmml" xref="S3.SS2.p4.3.m3.1.1">superscript</csymbol><apply id="S3.SS2.p4.3.m3.1.1.2.cmml" xref="S3.SS2.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.3.m3.1.1.2.1.cmml" xref="S3.SS2.p4.3.m3.1.1">subscript</csymbol><ci id="S3.SS2.p4.3.m3.1.1.2.2.cmml" xref="S3.SS2.p4.3.m3.1.1.2.2">𝒚</ci><ci id="S3.SS2.p4.3.m3.1.1.2.3.cmml" xref="S3.SS2.p4.3.m3.1.1.2.3">𝑖</ci></apply><ci id="S3.SS2.p4.3.m3.1.1.3.cmml" xref="S3.SS2.p4.3.m3.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.3.m3.1c">{\bm{y}}_{i}^{N}</annotation></semantics></math> or uniform noise. By initializing gradient descent optimization with previously optimized samples, we ensure that
these samples are closer to <math id="S3.SS2.p4.4.m4.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})" display="inline"><semantics id="S3.SS2.p4.4.m4.2a"><mrow id="S3.SS2.p4.4.m4.2.2" xref="S3.SS2.p4.4.m4.2.2.cmml"><mrow id="S3.SS2.p4.4.m4.2.2.3" xref="S3.SS2.p4.4.m4.2.2.3.cmml"><msub id="S3.SS2.p4.4.m4.2.2.3.1" xref="S3.SS2.p4.4.m4.2.2.3.1.cmml"><mrow id="S3.SS2.p4.4.m4.2.2.3.1.2" xref="S3.SS2.p4.4.m4.2.2.3.1.2.cmml"><mi id="S3.SS2.p4.4.m4.2.2.3.1.2.2" xref="S3.SS2.p4.4.m4.2.2.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS2.p4.4.m4.2.2.3.1.2.1" xref="S3.SS2.p4.4.m4.2.2.3.1.2.1.cmml">​</mo><mi id="S3.SS2.p4.4.m4.2.2.3.1.2.3" xref="S3.SS2.p4.4.m4.2.2.3.1.2.3.cmml">min</mi></mrow><mi id="S3.SS2.p4.4.m4.2.2.3.1.3" xref="S3.SS2.p4.4.m4.2.2.3.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS2.p4.4.m4.2.2.3a" xref="S3.SS2.p4.4.m4.2.2.3.cmml">⁡</mo><msub id="S3.SS2.p4.4.m4.2.2.3.2" xref="S3.SS2.p4.4.m4.2.2.3.2.cmml"><mi id="S3.SS2.p4.4.m4.2.2.3.2.2" xref="S3.SS2.p4.4.m4.2.2.3.2.2.cmml">E</mi><mi id="S3.SS2.p4.4.m4.2.2.3.2.3" xref="S3.SS2.p4.4.m4.2.2.3.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="S3.SS2.p4.4.m4.2.2.2" xref="S3.SS2.p4.4.m4.2.2.2.cmml">​</mo><mrow id="S3.SS2.p4.4.m4.2.2.1.1" xref="S3.SS2.p4.4.m4.2.2.1.2.cmml"><mo stretchy="false" id="S3.SS2.p4.4.m4.2.2.1.1.2" xref="S3.SS2.p4.4.m4.2.2.1.2.cmml">(</mo><msub id="S3.SS2.p4.4.m4.2.2.1.1.1" xref="S3.SS2.p4.4.m4.2.2.1.1.1.cmml"><mi id="S3.SS2.p4.4.m4.2.2.1.1.1.2" xref="S3.SS2.p4.4.m4.2.2.1.1.1.2.cmml">𝒙</mi><mi id="S3.SS2.p4.4.m4.2.2.1.1.1.3" xref="S3.SS2.p4.4.m4.2.2.1.1.1.3.cmml">i</mi></msub><mo id="S3.SS2.p4.4.m4.2.2.1.1.3" xref="S3.SS2.p4.4.m4.2.2.1.2.cmml">,</mo><mi id="S3.SS2.p4.4.m4.1.1" xref="S3.SS2.p4.4.m4.1.1.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p4.4.m4.2.2.1.1.4" xref="S3.SS2.p4.4.m4.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.4.m4.2b"><apply id="S3.SS2.p4.4.m4.2.2.cmml" xref="S3.SS2.p4.4.m4.2.2"><times id="S3.SS2.p4.4.m4.2.2.2.cmml" xref="S3.SS2.p4.4.m4.2.2.2"></times><apply id="S3.SS2.p4.4.m4.2.2.3.cmml" xref="S3.SS2.p4.4.m4.2.2.3"><apply id="S3.SS2.p4.4.m4.2.2.3.1.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1"><csymbol cd="ambiguous" id="S3.SS2.p4.4.m4.2.2.3.1.1.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1">subscript</csymbol><apply id="S3.SS2.p4.4.m4.2.2.3.1.2.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1.2"><times id="S3.SS2.p4.4.m4.2.2.3.1.2.1.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1.2.1"></times><ci id="S3.SS2.p4.4.m4.2.2.3.1.2.2.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1.2.2">arg</ci><ci id="S3.SS2.p4.4.m4.2.2.3.1.2.3.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1.2.3">min</ci></apply><ci id="S3.SS2.p4.4.m4.2.2.3.1.3.cmml" xref="S3.SS2.p4.4.m4.2.2.3.1.3">𝒚</ci></apply><apply id="S3.SS2.p4.4.m4.2.2.3.2.cmml" xref="S3.SS2.p4.4.m4.2.2.3.2"><csymbol cd="ambiguous" id="S3.SS2.p4.4.m4.2.2.3.2.1.cmml" xref="S3.SS2.p4.4.m4.2.2.3.2">subscript</csymbol><ci id="S3.SS2.p4.4.m4.2.2.3.2.2.cmml" xref="S3.SS2.p4.4.m4.2.2.3.2.2">𝐸</ci><ci id="S3.SS2.p4.4.m4.2.2.3.2.3.cmml" xref="S3.SS2.p4.4.m4.2.2.3.2.3">𝜃</ci></apply></apply><interval closure="open" id="S3.SS2.p4.4.m4.2.2.1.2.cmml" xref="S3.SS2.p4.4.m4.2.2.1.1"><apply id="S3.SS2.p4.4.m4.2.2.1.1.1.cmml" xref="S3.SS2.p4.4.m4.2.2.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.4.m4.2.2.1.1.1.1.cmml" xref="S3.SS2.p4.4.m4.2.2.1.1.1">subscript</csymbol><ci id="S3.SS2.p4.4.m4.2.2.1.1.1.2.cmml" xref="S3.SS2.p4.4.m4.2.2.1.1.1.2">𝒙</ci><ci id="S3.SS2.p4.4.m4.2.2.1.1.1.3.cmml" xref="S3.SS2.p4.4.m4.2.2.1.1.1.3">𝑖</ci></apply><ci id="S3.SS2.p4.4.m4.1.1.cmml" xref="S3.SS2.p4.4.m4.1.1">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.4.m4.2c">\operatorname*{arg\,min}_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}})</annotation></semantics></math>. A similar application of replay buffers has been utilized to train EBMs for consistent probability landscapes <cite class="ltx_cite ltx_citemacro_citep">(Du &amp; Mordatch, <a href="#bib.bib14" title="" class="ltx_ref">2019</a>)</cite>.</p>
</div>
<div id="S3.SS2.p5" class="ltx_para">
<p id="S3.SS2.p5.3" class="ltx_p">We provide pseudocode for training IREM in Algorithm <a href="#alg1" title="Algorithm 1 ‣ 3.2 Learning Energy Landscapes for Reasoning ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> and executing algorithmic reasoning with IREM in Algorithm <a href="#alg2" title="Algorithm 2 ‣ 3.3 Analysis ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, where an energy minimum is determined at test time once the energy value of a solution does not change for certain iterations. A fixed learning rate, <math id="S3.SS2.p5.1.m1.1" class="ltx_Math" alttext="\lambda=100" display="inline"><semantics id="S3.SS2.p5.1.m1.1a"><mrow id="S3.SS2.p5.1.m1.1.1" xref="S3.SS2.p5.1.m1.1.1.cmml"><mi id="S3.SS2.p5.1.m1.1.1.2" xref="S3.SS2.p5.1.m1.1.1.2.cmml">λ</mi><mo id="S3.SS2.p5.1.m1.1.1.1" xref="S3.SS2.p5.1.m1.1.1.1.cmml">=</mo><mn id="S3.SS2.p5.1.m1.1.1.3" xref="S3.SS2.p5.1.m1.1.1.3.cmml">100</mn></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p5.1.m1.1b"><apply id="S3.SS2.p5.1.m1.1.1.cmml" xref="S3.SS2.p5.1.m1.1.1"><eq id="S3.SS2.p5.1.m1.1.1.1.cmml" xref="S3.SS2.p5.1.m1.1.1.1"></eq><ci id="S3.SS2.p5.1.m1.1.1.2.cmml" xref="S3.SS2.p5.1.m1.1.1.2">𝜆</ci><cn type="integer" id="S3.SS2.p5.1.m1.1.1.3.cmml" xref="S3.SS2.p5.1.m1.1.1.3">100</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p5.1.m1.1c">\lambda=100</annotation></semantics></math>, is used for training IREM. At test time, <math id="S3.SS2.p5.2.m2.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="S3.SS2.p5.2.m2.1a"><mi id="S3.SS2.p5.2.m2.1.1" xref="S3.SS2.p5.2.m2.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.p5.2.m2.1b"><ci id="S3.SS2.p5.2.m2.1.1.cmml" xref="S3.SS2.p5.2.m2.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p5.2.m2.1c">\lambda</annotation></semantics></math> is empirically tuned so that energy values decrease smoothly across iterations of optimization (harder problems require smaller <math id="S3.SS2.p5.3.m3.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="S3.SS2.p5.3.m3.1a"><mi id="S3.SS2.p5.3.m3.1.1" xref="S3.SS2.p5.3.m3.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="S3.SS2.p5.3.m3.1b"><ci id="S3.SS2.p5.3.m3.1.1.cmml" xref="S3.SS2.p5.3.m3.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p5.3.m3.1c">\lambda</annotation></semantics></math> to optimize). In Appendix <a href="#A2" title="Appendix B Iterative Reasoning with a Scratchpad ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>, we discuss how we may utilize optimization to further incorporate an external scratchpad into iterative reasoning using IREM.</p>
</div>
<div id="S3.SS2.p6" class="ltx_para">
<p id="S3.SS2.p6.1" class="ltx_p">As IREM directly trains an energy landscape by optimizing samples to regress solutions, there is no guarantee that a smooth underlying energy landscape <math id="S3.SS2.p6.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS2.p6.1.m1.2a"><mrow id="S3.SS2.p6.1.m1.2.3" xref="S3.SS2.p6.1.m1.2.3.cmml"><mi id="S3.SS2.p6.1.m1.2.3.2" xref="S3.SS2.p6.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS2.p6.1.m1.2.3.1" xref="S3.SS2.p6.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS2.p6.1.m1.2.3.3.2" xref="S3.SS2.p6.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS2.p6.1.m1.2.3.3.2.1" xref="S3.SS2.p6.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS2.p6.1.m1.1.1" xref="S3.SS2.p6.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS2.p6.1.m1.2.3.3.2.2" xref="S3.SS2.p6.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS2.p6.1.m1.2.2" xref="S3.SS2.p6.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS2.p6.1.m1.2.3.3.2.3" xref="S3.SS2.p6.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p6.1.m1.2b"><apply id="S3.SS2.p6.1.m1.2.3.cmml" xref="S3.SS2.p6.1.m1.2.3"><times id="S3.SS2.p6.1.m1.2.3.1.cmml" xref="S3.SS2.p6.1.m1.2.3.1"></times><ci id="S3.SS2.p6.1.m1.2.3.2.cmml" xref="S3.SS2.p6.1.m1.2.3.2">𝐸</ci><interval closure="open" id="S3.SS2.p6.1.m1.2.3.3.1.cmml" xref="S3.SS2.p6.1.m1.2.3.3.2"><ci id="S3.SS2.p6.1.m1.1.1.cmml" xref="S3.SS2.p6.1.m1.1.1">𝒙</ci><ci id="S3.SS2.p6.1.m1.2.2.cmml" xref="S3.SS2.p6.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p6.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> is learned. Empirically, as seen in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> and Figure <a href="#S4.F5" title="Figure 5 ‣ 4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, we find that our objective does lead to consistent energy landscapes similar to past work <cite class="ltx_cite ltx_citemacro_citep">(Du et al., <a href="#bib.bib16" title="" class="ltx_ref">2021a</a>)</cite>.</p>
</div>
<figure id="alg1" class="ltx_float ltx_float_algorithm ltx_framed ltx_framed_top">
<div id="alg1.2" class="ltx_listing ltx_listing">
<div id="alg0.l1" class="ltx_listingline">
<span id="alg0.l1.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg0.l1.2" class="ltx_text ltx_font_bold" style="font-size:90%;">Input:</span><span id="alg0.l1.3" class="ltx_text" style="font-size:90%;"> Data Dist </span><math id="alg0.l1.m1.2" class="ltx_Math" alttext="p_{D}({\bm{x}},{\bm{y}})" display="inline"><semantics id="alg0.l1.m1.2a"><mrow id="alg0.l1.m1.2.3" xref="alg0.l1.m1.2.3.cmml"><msub id="alg0.l1.m1.2.3.2" xref="alg0.l1.m1.2.3.2.cmml"><mi mathsize="90%" id="alg0.l1.m1.2.3.2.2" xref="alg0.l1.m1.2.3.2.2.cmml">p</mi><mi mathsize="90%" id="alg0.l1.m1.2.3.2.3" xref="alg0.l1.m1.2.3.2.3.cmml">D</mi></msub><mo lspace="0em" rspace="0em" id="alg0.l1.m1.2.3.1" xref="alg0.l1.m1.2.3.1.cmml">​</mo><mrow id="alg0.l1.m1.2.3.3.2" xref="alg0.l1.m1.2.3.3.1.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l1.m1.2.3.3.2.1" xref="alg0.l1.m1.2.3.3.1.cmml">(</mo><mi mathsize="90%" id="alg0.l1.m1.1.1" xref="alg0.l1.m1.1.1.cmml">𝒙</mi><mo mathsize="90%" id="alg0.l1.m1.2.3.3.2.2" xref="alg0.l1.m1.2.3.3.1.cmml">,</mo><mi mathsize="90%" id="alg0.l1.m1.2.2" xref="alg0.l1.m1.2.2.cmml">𝒚</mi><mo maxsize="90%" minsize="90%" id="alg0.l1.m1.2.3.3.2.3" xref="alg0.l1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l1.m1.2b"><apply id="alg0.l1.m1.2.3.cmml" xref="alg0.l1.m1.2.3"><times id="alg0.l1.m1.2.3.1.cmml" xref="alg0.l1.m1.2.3.1"></times><apply id="alg0.l1.m1.2.3.2.cmml" xref="alg0.l1.m1.2.3.2"><csymbol cd="ambiguous" id="alg0.l1.m1.2.3.2.1.cmml" xref="alg0.l1.m1.2.3.2">subscript</csymbol><ci id="alg0.l1.m1.2.3.2.2.cmml" xref="alg0.l1.m1.2.3.2.2">𝑝</ci><ci id="alg0.l1.m1.2.3.2.3.cmml" xref="alg0.l1.m1.2.3.2.3">𝐷</ci></apply><interval closure="open" id="alg0.l1.m1.2.3.3.1.cmml" xref="alg0.l1.m1.2.3.3.2"><ci id="alg0.l1.m1.1.1.cmml" xref="alg0.l1.m1.1.1">𝒙</ci><ci id="alg0.l1.m1.2.2.cmml" xref="alg0.l1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m1.2c">p_{D}({\bm{x}},{\bm{y}})</annotation></semantics></math><span id="alg0.l1.4" class="ltx_text" style="font-size:90%;">, Replay Buffer </span><math id="alg0.l1.m2.1" class="ltx_Math" alttext="\mathcal{B}" display="inline"><semantics id="alg0.l1.m2.1a"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg0.l1.m2.1.1" xref="alg0.l1.m2.1.1.cmml">ℬ</mi><annotation-xml encoding="MathML-Content" id="alg0.l1.m2.1b"><ci id="alg0.l1.m2.1.1.cmml" xref="alg0.l1.m2.1.1">ℬ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m2.1c">\mathcal{B}</annotation></semantics></math><span id="alg0.l1.5" class="ltx_text" style="font-size:90%;">, Step Size </span><math id="alg0.l1.m3.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="alg0.l1.m3.1a"><mi mathsize="90%" id="alg0.l1.m3.1.1" xref="alg0.l1.m3.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="alg0.l1.m3.1b"><ci id="alg0.l1.m3.1.1.cmml" xref="alg0.l1.m3.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m3.1c">\lambda</annotation></semantics></math><span id="alg0.l1.6" class="ltx_text" style="font-size:90%;">, Number of Steps </span><math id="alg0.l1.m4.1" class="ltx_Math" alttext="N" display="inline"><semantics id="alg0.l1.m4.1a"><mi mathsize="90%" id="alg0.l1.m4.1.1" xref="alg0.l1.m4.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="alg0.l1.m4.1b"><ci id="alg0.l1.m4.1.1.cmml" xref="alg0.l1.m4.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m4.1c">N</annotation></semantics></math><span id="alg0.l1.7" class="ltx_text" style="font-size:90%;">, EBM </span><math id="alg0.l1.m5.1" class="ltx_Math" alttext="E_{\theta}(\cdot)" display="inline"><semantics id="alg0.l1.m5.1a"><mrow id="alg0.l1.m5.1.2" xref="alg0.l1.m5.1.2.cmml"><msub id="alg0.l1.m5.1.2.2" xref="alg0.l1.m5.1.2.2.cmml"><mi mathsize="90%" id="alg0.l1.m5.1.2.2.2" xref="alg0.l1.m5.1.2.2.2.cmml">E</mi><mi mathsize="90%" id="alg0.l1.m5.1.2.2.3" xref="alg0.l1.m5.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="alg0.l1.m5.1.2.1" xref="alg0.l1.m5.1.2.1.cmml">​</mo><mrow id="alg0.l1.m5.1.2.3.2" xref="alg0.l1.m5.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l1.m5.1.2.3.2.1" xref="alg0.l1.m5.1.2.cmml">(</mo><mo lspace="0em" mathsize="90%" rspace="0em" id="alg0.l1.m5.1.1" xref="alg0.l1.m5.1.1.cmml">⋅</mo><mo maxsize="90%" minsize="90%" id="alg0.l1.m5.1.2.3.2.2" xref="alg0.l1.m5.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l1.m5.1b"><apply id="alg0.l1.m5.1.2.cmml" xref="alg0.l1.m5.1.2"><times id="alg0.l1.m5.1.2.1.cmml" xref="alg0.l1.m5.1.2.1"></times><apply id="alg0.l1.m5.1.2.2.cmml" xref="alg0.l1.m5.1.2.2"><csymbol cd="ambiguous" id="alg0.l1.m5.1.2.2.1.cmml" xref="alg0.l1.m5.1.2.2">subscript</csymbol><ci id="alg0.l1.m5.1.2.2.2.cmml" xref="alg0.l1.m5.1.2.2.2">𝐸</ci><ci id="alg0.l1.m5.1.2.2.3.cmml" xref="alg0.l1.m5.1.2.2.3">𝜃</ci></apply><ci id="alg0.l1.m5.1.1.cmml" xref="alg0.l1.m5.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m5.1c">E_{\theta}(\cdot)</annotation></semantics></math><span id="alg0.l1.8" class="ltx_text" style="font-size:90%;">, Uniform Distribution </span><math id="alg0.l1.m6.2" class="ltx_Math" alttext="U(-1,1)" display="inline"><semantics id="alg0.l1.m6.2a"><mrow id="alg0.l1.m6.2.2" xref="alg0.l1.m6.2.2.cmml"><mi mathsize="90%" id="alg0.l1.m6.2.2.3" xref="alg0.l1.m6.2.2.3.cmml">U</mi><mo lspace="0em" rspace="0em" id="alg0.l1.m6.2.2.2" xref="alg0.l1.m6.2.2.2.cmml">​</mo><mrow id="alg0.l1.m6.2.2.1.1" xref="alg0.l1.m6.2.2.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l1.m6.2.2.1.1.2" xref="alg0.l1.m6.2.2.1.2.cmml">(</mo><mrow id="alg0.l1.m6.2.2.1.1.1" xref="alg0.l1.m6.2.2.1.1.1.cmml"><mo mathsize="90%" id="alg0.l1.m6.2.2.1.1.1a" xref="alg0.l1.m6.2.2.1.1.1.cmml">−</mo><mn mathsize="90%" id="alg0.l1.m6.2.2.1.1.1.2" xref="alg0.l1.m6.2.2.1.1.1.2.cmml">1</mn></mrow><mo mathsize="90%" id="alg0.l1.m6.2.2.1.1.3" xref="alg0.l1.m6.2.2.1.2.cmml">,</mo><mn mathsize="90%" id="alg0.l1.m6.1.1" xref="alg0.l1.m6.1.1.cmml">1</mn><mo maxsize="90%" minsize="90%" id="alg0.l1.m6.2.2.1.1.4" xref="alg0.l1.m6.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l1.m6.2b"><apply id="alg0.l1.m6.2.2.cmml" xref="alg0.l1.m6.2.2"><times id="alg0.l1.m6.2.2.2.cmml" xref="alg0.l1.m6.2.2.2"></times><ci id="alg0.l1.m6.2.2.3.cmml" xref="alg0.l1.m6.2.2.3">𝑈</ci><interval closure="open" id="alg0.l1.m6.2.2.1.2.cmml" xref="alg0.l1.m6.2.2.1.1"><apply id="alg0.l1.m6.2.2.1.1.1.cmml" xref="alg0.l1.m6.2.2.1.1.1"><minus id="alg0.l1.m6.2.2.1.1.1.1.cmml" xref="alg0.l1.m6.2.2.1.1.1"></minus><cn type="integer" id="alg0.l1.m6.2.2.1.1.1.2.cmml" xref="alg0.l1.m6.2.2.1.1.1.2">1</cn></apply><cn type="integer" id="alg0.l1.m6.1.1.cmml" xref="alg0.l1.m6.1.1">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m6.2c">U(-1,1)</annotation></semantics></math><span id="alg0.l1.9" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l2" class="ltx_listingline">
<span id="alg0.l2.1" class="ltx_text" style="font-size:90%;">  </span><math id="alg0.l2.m1.1" class="ltx_Math" alttext="\mathcal{B}\leftarrow\varnothing" display="inline"><semantics id="alg0.l2.m1.1a"><mrow id="alg0.l2.m1.1.1" xref="alg0.l2.m1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg0.l2.m1.1.1.2" xref="alg0.l2.m1.1.1.2.cmml">ℬ</mi><mo mathsize="90%" stretchy="false" id="alg0.l2.m1.1.1.1" xref="alg0.l2.m1.1.1.1.cmml">←</mo><mi mathsize="90%" mathvariant="normal" id="alg0.l2.m1.1.1.3" xref="alg0.l2.m1.1.1.3.cmml">∅</mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l2.m1.1b"><apply id="alg0.l2.m1.1.1.cmml" xref="alg0.l2.m1.1.1"><ci id="alg0.l2.m1.1.1.1.cmml" xref="alg0.l2.m1.1.1.1">←</ci><ci id="alg0.l2.m1.1.1.2.cmml" xref="alg0.l2.m1.1.1.2">ℬ</ci><emptyset id="alg0.l2.m1.1.1.3.cmml" xref="alg0.l2.m1.1.1.3"></emptyset></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l2.m1.1c">\mathcal{B}\leftarrow\varnothing</annotation></semantics></math><span id="alg0.l2.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l3" class="ltx_listingline">
<span id="alg0.l3.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg0.l3.2" class="ltx_text ltx_font_bold" style="font-size:90%;">while</span><span id="alg0.l3.3" class="ltx_text" style="font-size:90%;"> not converged </span><span id="alg0.l3.4" class="ltx_text ltx_font_bold" style="font-size:90%;">do</span><span id="alg0.l3.5" class="ltx_text" style="font-size:90%;">


</span>
</div>
<div id="alg0.l4" class="ltx_listingline">
<span id="alg0.l4.4" class="ltx_text" style="font-size:90%;">    </span><em id="alg0.l4.3" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg0.l4.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg0.l4.1.m1.1a"><mo id="alg0.l4.1.m1.1.1" xref="alg0.l4.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg0.l4.1.m1.1b"><ci id="alg0.l4.1.m1.1.1.cmml" xref="alg0.l4.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l4.1.m1.1c">\triangleright</annotation></semantics></math> Sample data and candidate solutions from <math id="alg0.l4.2.m2.1" class="ltx_Math" alttext="p_{d}" display="inline"><semantics id="alg0.l4.2.m2.1a"><msub id="alg0.l4.2.m2.1.1" xref="alg0.l4.2.m2.1.1.cmml"><mi id="alg0.l4.2.m2.1.1.2" xref="alg0.l4.2.m2.1.1.2.cmml">p</mi><mi id="alg0.l4.2.m2.1.1.3" xref="alg0.l4.2.m2.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l4.2.m2.1b"><apply id="alg0.l4.2.m2.1.1.cmml" xref="alg0.l4.2.m2.1.1"><csymbol cd="ambiguous" id="alg0.l4.2.m2.1.1.1.cmml" xref="alg0.l4.2.m2.1.1">subscript</csymbol><ci id="alg0.l4.2.m2.1.1.2.cmml" xref="alg0.l4.2.m2.1.1.2">𝑝</ci><ci id="alg0.l4.2.m2.1.1.3.cmml" xref="alg0.l4.2.m2.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l4.2.m2.1c">p_{d}</annotation></semantics></math> and replay buffer <math id="alg0.l4.3.m3.1" class="ltx_Math" alttext="\mathcal{B}" display="inline"><semantics id="alg0.l4.3.m3.1a"><mi class="ltx_font_mathcaligraphic" id="alg0.l4.3.m3.1.1" xref="alg0.l4.3.m3.1.1.cmml">ℬ</mi><annotation-xml encoding="MathML-Content" id="alg0.l4.3.m3.1b"><ci id="alg0.l4.3.m3.1.1.cmml" xref="alg0.l4.3.m3.1.1">ℬ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l4.3.m3.1c">\mathcal{B}</annotation></semantics></math></em><span id="alg0.l4.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l5" class="ltx_listingline">
<span id="alg0.l5.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg0.l5.m1.2" class="ltx_Math" alttext="{\bm{x}}_{i},{\bm{y}}_{i}\sim p_{D}" display="inline"><semantics id="alg0.l5.m1.2a"><mrow id="alg0.l5.m1.2.2" xref="alg0.l5.m1.2.2.cmml"><mrow id="alg0.l5.m1.2.2.2.2" xref="alg0.l5.m1.2.2.2.3.cmml"><msub id="alg0.l5.m1.1.1.1.1.1" xref="alg0.l5.m1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg0.l5.m1.1.1.1.1.1.2" xref="alg0.l5.m1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l5.m1.1.1.1.1.1.3" xref="alg0.l5.m1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l5.m1.2.2.2.2.3" xref="alg0.l5.m1.2.2.2.3.cmml">,</mo><msub id="alg0.l5.m1.2.2.2.2.2" xref="alg0.l5.m1.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg0.l5.m1.2.2.2.2.2.2" xref="alg0.l5.m1.2.2.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l5.m1.2.2.2.2.2.3" xref="alg0.l5.m1.2.2.2.2.2.3.cmml">i</mi></msub></mrow><mo mathsize="90%" id="alg0.l5.m1.2.2.3" xref="alg0.l5.m1.2.2.3.cmml">∼</mo><msub id="alg0.l5.m1.2.2.4" xref="alg0.l5.m1.2.2.4.cmml"><mi mathsize="90%" id="alg0.l5.m1.2.2.4.2" xref="alg0.l5.m1.2.2.4.2.cmml">p</mi><mi mathsize="90%" id="alg0.l5.m1.2.2.4.3" xref="alg0.l5.m1.2.2.4.3.cmml">D</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="alg0.l5.m1.2b"><apply id="alg0.l5.m1.2.2.cmml" xref="alg0.l5.m1.2.2"><csymbol cd="latexml" id="alg0.l5.m1.2.2.3.cmml" xref="alg0.l5.m1.2.2.3">similar-to</csymbol><list id="alg0.l5.m1.2.2.2.3.cmml" xref="alg0.l5.m1.2.2.2.2"><apply id="alg0.l5.m1.1.1.1.1.1.cmml" xref="alg0.l5.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l5.m1.1.1.1.1.1.1.cmml" xref="alg0.l5.m1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l5.m1.1.1.1.1.1.2.cmml" xref="alg0.l5.m1.1.1.1.1.1.2">𝒙</ci><ci id="alg0.l5.m1.1.1.1.1.1.3.cmml" xref="alg0.l5.m1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg0.l5.m1.2.2.2.2.2.cmml" xref="alg0.l5.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l5.m1.2.2.2.2.2.1.cmml" xref="alg0.l5.m1.2.2.2.2.2">subscript</csymbol><ci id="alg0.l5.m1.2.2.2.2.2.2.cmml" xref="alg0.l5.m1.2.2.2.2.2.2">𝒚</ci><ci id="alg0.l5.m1.2.2.2.2.2.3.cmml" xref="alg0.l5.m1.2.2.2.2.2.3">𝑖</ci></apply></list><apply id="alg0.l5.m1.2.2.4.cmml" xref="alg0.l5.m1.2.2.4"><csymbol cd="ambiguous" id="alg0.l5.m1.2.2.4.1.cmml" xref="alg0.l5.m1.2.2.4">subscript</csymbol><ci id="alg0.l5.m1.2.2.4.2.cmml" xref="alg0.l5.m1.2.2.4.2">𝑝</ci><ci id="alg0.l5.m1.2.2.4.3.cmml" xref="alg0.l5.m1.2.2.4.3">𝐷</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l5.m1.2c">{\bm{x}}_{i},{\bm{y}}_{i}\sim p_{D}</annotation></semantics></math><span id="alg0.l5.2" class="ltx_text" style="font-size:90%;">, </span><math id="alg0.l5.m2.2" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}^{0}\sim\mathcal{U}(-1,1)" display="inline"><semantics id="alg0.l5.m2.2a"><mrow id="alg0.l5.m2.2.2" xref="alg0.l5.m2.2.2.cmml"><msubsup id="alg0.l5.m2.2.2.3" xref="alg0.l5.m2.2.2.3.cmml"><mover accent="true" id="alg0.l5.m2.2.2.3.2.2" xref="alg0.l5.m2.2.2.3.2.2.cmml"><mi mathsize="90%" id="alg0.l5.m2.2.2.3.2.2.2" xref="alg0.l5.m2.2.2.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l5.m2.2.2.3.2.2.1" xref="alg0.l5.m2.2.2.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l5.m2.2.2.3.2.3" xref="alg0.l5.m2.2.2.3.2.3.cmml">i</mi><mn mathsize="90%" id="alg0.l5.m2.2.2.3.3" xref="alg0.l5.m2.2.2.3.3.cmml">0</mn></msubsup><mo mathsize="90%" id="alg0.l5.m2.2.2.2" xref="alg0.l5.m2.2.2.2.cmml">∼</mo><mrow id="alg0.l5.m2.2.2.1" xref="alg0.l5.m2.2.2.1.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg0.l5.m2.2.2.1.3" xref="alg0.l5.m2.2.2.1.3.cmml">𝒰</mi><mo lspace="0em" rspace="0em" id="alg0.l5.m2.2.2.1.2" xref="alg0.l5.m2.2.2.1.2.cmml">​</mo><mrow id="alg0.l5.m2.2.2.1.1.1" xref="alg0.l5.m2.2.2.1.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l5.m2.2.2.1.1.1.2" xref="alg0.l5.m2.2.2.1.1.2.cmml">(</mo><mrow id="alg0.l5.m2.2.2.1.1.1.1" xref="alg0.l5.m2.2.2.1.1.1.1.cmml"><mo mathsize="90%" id="alg0.l5.m2.2.2.1.1.1.1a" xref="alg0.l5.m2.2.2.1.1.1.1.cmml">−</mo><mn mathsize="90%" id="alg0.l5.m2.2.2.1.1.1.1.2" xref="alg0.l5.m2.2.2.1.1.1.1.2.cmml">1</mn></mrow><mo mathsize="90%" id="alg0.l5.m2.2.2.1.1.1.3" xref="alg0.l5.m2.2.2.1.1.2.cmml">,</mo><mn mathsize="90%" id="alg0.l5.m2.1.1" xref="alg0.l5.m2.1.1.cmml">1</mn><mo maxsize="90%" minsize="90%" id="alg0.l5.m2.2.2.1.1.1.4" xref="alg0.l5.m2.2.2.1.1.2.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l5.m2.2b"><apply id="alg0.l5.m2.2.2.cmml" xref="alg0.l5.m2.2.2"><csymbol cd="latexml" id="alg0.l5.m2.2.2.2.cmml" xref="alg0.l5.m2.2.2.2">similar-to</csymbol><apply id="alg0.l5.m2.2.2.3.cmml" xref="alg0.l5.m2.2.2.3"><csymbol cd="ambiguous" id="alg0.l5.m2.2.2.3.1.cmml" xref="alg0.l5.m2.2.2.3">superscript</csymbol><apply id="alg0.l5.m2.2.2.3.2.cmml" xref="alg0.l5.m2.2.2.3"><csymbol cd="ambiguous" id="alg0.l5.m2.2.2.3.2.1.cmml" xref="alg0.l5.m2.2.2.3">subscript</csymbol><apply id="alg0.l5.m2.2.2.3.2.2.cmml" xref="alg0.l5.m2.2.2.3.2.2"><ci id="alg0.l5.m2.2.2.3.2.2.1.cmml" xref="alg0.l5.m2.2.2.3.2.2.1">~</ci><ci id="alg0.l5.m2.2.2.3.2.2.2.cmml" xref="alg0.l5.m2.2.2.3.2.2.2">𝒚</ci></apply><ci id="alg0.l5.m2.2.2.3.2.3.cmml" xref="alg0.l5.m2.2.2.3.2.3">𝑖</ci></apply><cn type="integer" id="alg0.l5.m2.2.2.3.3.cmml" xref="alg0.l5.m2.2.2.3.3">0</cn></apply><apply id="alg0.l5.m2.2.2.1.cmml" xref="alg0.l5.m2.2.2.1"><times id="alg0.l5.m2.2.2.1.2.cmml" xref="alg0.l5.m2.2.2.1.2"></times><ci id="alg0.l5.m2.2.2.1.3.cmml" xref="alg0.l5.m2.2.2.1.3">𝒰</ci><interval closure="open" id="alg0.l5.m2.2.2.1.1.2.cmml" xref="alg0.l5.m2.2.2.1.1.1"><apply id="alg0.l5.m2.2.2.1.1.1.1.cmml" xref="alg0.l5.m2.2.2.1.1.1.1"><minus id="alg0.l5.m2.2.2.1.1.1.1.1.cmml" xref="alg0.l5.m2.2.2.1.1.1.1"></minus><cn type="integer" id="alg0.l5.m2.2.2.1.1.1.1.2.cmml" xref="alg0.l5.m2.2.2.1.1.1.1.2">1</cn></apply><cn type="integer" id="alg0.l5.m2.1.1.cmml" xref="alg0.l5.m2.1.1">1</cn></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l5.m2.2c">\tilde{{\bm{y}}}_{i}^{0}\sim\mathcal{U}(-1,1)</annotation></semantics></math><span id="alg0.l5.3" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l6" class="ltx_listingline">
<span id="alg0.l6.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg0.l6.m1.3" class="ltx_Math" alttext="{\bm{x}}_{i}^{b},{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{b}\sim B" display="inline"><semantics id="alg0.l6.m1.3a"><mrow id="alg0.l6.m1.3.3" xref="alg0.l6.m1.3.3.cmml"><mrow id="alg0.l6.m1.3.3.3.3" xref="alg0.l6.m1.3.3.3.4.cmml"><msubsup id="alg0.l6.m1.1.1.1.1.1" xref="alg0.l6.m1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg0.l6.m1.1.1.1.1.1.2.2" xref="alg0.l6.m1.1.1.1.1.1.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l6.m1.1.1.1.1.1.2.3" xref="alg0.l6.m1.1.1.1.1.1.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l6.m1.1.1.1.1.1.3" xref="alg0.l6.m1.1.1.1.1.1.3.cmml">b</mi></msubsup><mo mathsize="90%" id="alg0.l6.m1.3.3.3.3.4" xref="alg0.l6.m1.3.3.3.4.cmml">,</mo><msubsup id="alg0.l6.m1.2.2.2.2.2" xref="alg0.l6.m1.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg0.l6.m1.2.2.2.2.2.2.2" xref="alg0.l6.m1.2.2.2.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l6.m1.2.2.2.2.2.2.3" xref="alg0.l6.m1.2.2.2.2.2.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l6.m1.2.2.2.2.2.3" xref="alg0.l6.m1.2.2.2.2.2.3.cmml">b</mi></msubsup><mo mathsize="90%" id="alg0.l6.m1.3.3.3.3.5" xref="alg0.l6.m1.3.3.3.4.cmml">,</mo><msubsup id="alg0.l6.m1.3.3.3.3.3" xref="alg0.l6.m1.3.3.3.3.3.cmml"><mover accent="true" id="alg0.l6.m1.3.3.3.3.3.2.2" xref="alg0.l6.m1.3.3.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg0.l6.m1.3.3.3.3.3.2.2.2" xref="alg0.l6.m1.3.3.3.3.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l6.m1.3.3.3.3.3.2.2.1" xref="alg0.l6.m1.3.3.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l6.m1.3.3.3.3.3.2.3" xref="alg0.l6.m1.3.3.3.3.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l6.m1.3.3.3.3.3.3" xref="alg0.l6.m1.3.3.3.3.3.3.cmml">b</mi></msubsup></mrow><mo mathsize="90%" id="alg0.l6.m1.3.3.4" xref="alg0.l6.m1.3.3.4.cmml">∼</mo><mi mathsize="90%" id="alg0.l6.m1.3.3.5" xref="alg0.l6.m1.3.3.5.cmml">B</mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l6.m1.3b"><apply id="alg0.l6.m1.3.3.cmml" xref="alg0.l6.m1.3.3"><csymbol cd="latexml" id="alg0.l6.m1.3.3.4.cmml" xref="alg0.l6.m1.3.3.4">similar-to</csymbol><list id="alg0.l6.m1.3.3.3.4.cmml" xref="alg0.l6.m1.3.3.3.3"><apply id="alg0.l6.m1.1.1.1.1.1.cmml" xref="alg0.l6.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l6.m1.1.1.1.1.1.1.cmml" xref="alg0.l6.m1.1.1.1.1.1">superscript</csymbol><apply id="alg0.l6.m1.1.1.1.1.1.2.cmml" xref="alg0.l6.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l6.m1.1.1.1.1.1.2.1.cmml" xref="alg0.l6.m1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l6.m1.1.1.1.1.1.2.2.cmml" xref="alg0.l6.m1.1.1.1.1.1.2.2">𝒙</ci><ci id="alg0.l6.m1.1.1.1.1.1.2.3.cmml" xref="alg0.l6.m1.1.1.1.1.1.2.3">𝑖</ci></apply><ci id="alg0.l6.m1.1.1.1.1.1.3.cmml" xref="alg0.l6.m1.1.1.1.1.1.3">𝑏</ci></apply><apply id="alg0.l6.m1.2.2.2.2.2.cmml" xref="alg0.l6.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l6.m1.2.2.2.2.2.1.cmml" xref="alg0.l6.m1.2.2.2.2.2">superscript</csymbol><apply id="alg0.l6.m1.2.2.2.2.2.2.cmml" xref="alg0.l6.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l6.m1.2.2.2.2.2.2.1.cmml" xref="alg0.l6.m1.2.2.2.2.2">subscript</csymbol><ci id="alg0.l6.m1.2.2.2.2.2.2.2.cmml" xref="alg0.l6.m1.2.2.2.2.2.2.2">𝒚</ci><ci id="alg0.l6.m1.2.2.2.2.2.2.3.cmml" xref="alg0.l6.m1.2.2.2.2.2.2.3">𝑖</ci></apply><ci id="alg0.l6.m1.2.2.2.2.2.3.cmml" xref="alg0.l6.m1.2.2.2.2.2.3">𝑏</ci></apply><apply id="alg0.l6.m1.3.3.3.3.3.cmml" xref="alg0.l6.m1.3.3.3.3.3"><csymbol cd="ambiguous" id="alg0.l6.m1.3.3.3.3.3.1.cmml" xref="alg0.l6.m1.3.3.3.3.3">superscript</csymbol><apply id="alg0.l6.m1.3.3.3.3.3.2.cmml" xref="alg0.l6.m1.3.3.3.3.3"><csymbol cd="ambiguous" id="alg0.l6.m1.3.3.3.3.3.2.1.cmml" xref="alg0.l6.m1.3.3.3.3.3">subscript</csymbol><apply id="alg0.l6.m1.3.3.3.3.3.2.2.cmml" xref="alg0.l6.m1.3.3.3.3.3.2.2"><ci id="alg0.l6.m1.3.3.3.3.3.2.2.1.cmml" xref="alg0.l6.m1.3.3.3.3.3.2.2.1">~</ci><ci id="alg0.l6.m1.3.3.3.3.3.2.2.2.cmml" xref="alg0.l6.m1.3.3.3.3.3.2.2.2">𝒚</ci></apply><ci id="alg0.l6.m1.3.3.3.3.3.2.3.cmml" xref="alg0.l6.m1.3.3.3.3.3.2.3">𝑖</ci></apply><ci id="alg0.l6.m1.3.3.3.3.3.3.cmml" xref="alg0.l6.m1.3.3.3.3.3.3">𝑏</ci></apply></list><ci id="alg0.l6.m1.3.3.5.cmml" xref="alg0.l6.m1.3.3.5">𝐵</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l6.m1.3c">{\bm{x}}_{i}^{b},{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{b}\sim B</annotation></semantics></math><span id="alg0.l6.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l7" class="ltx_listingline">
<span id="alg0.l7.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg0.l7.m1.2" class="ltx_Math" alttext="{\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{0}\leftarrow{\bm{x}}_{i}\cup{\bm{x}}_{i}^{b},{\bm{y}}_{i}\cup{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{0}\cup\tilde{{\bm{y}}}_{i}^{b}" display="inline"><semantics id="alg0.l7.m1.2a"><mrow id="alg0.l7.m1.2.2.2" xref="alg0.l7.m1.2.2.3.cmml"><mrow id="alg0.l7.m1.1.1.1.1" xref="alg0.l7.m1.1.1.1.1.cmml"><mrow id="alg0.l7.m1.1.1.1.1.3.3" xref="alg0.l7.m1.1.1.1.1.3.4.cmml"><msub id="alg0.l7.m1.1.1.1.1.1.1.1" xref="alg0.l7.m1.1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.1.1.1.2" xref="alg0.l7.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.1.1.1.3" xref="alg0.l7.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l7.m1.1.1.1.1.3.3.4" xref="alg0.l7.m1.1.1.1.1.3.4.cmml">,</mo><msub id="alg0.l7.m1.1.1.1.1.2.2.2" xref="alg0.l7.m1.1.1.1.1.2.2.2.cmml"><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.2.2.2.2" xref="alg0.l7.m1.1.1.1.1.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.2.2.2.3" xref="alg0.l7.m1.1.1.1.1.2.2.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l7.m1.1.1.1.1.3.3.5" xref="alg0.l7.m1.1.1.1.1.3.4.cmml">,</mo><msubsup id="alg0.l7.m1.1.1.1.1.3.3.3" xref="alg0.l7.m1.1.1.1.1.3.3.3.cmml"><mover accent="true" id="alg0.l7.m1.1.1.1.1.3.3.3.2.2" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.3.3.3.2.2.2" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l7.m1.1.1.1.1.3.3.3.2.2.1" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.3.3.3.2.3" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.3.cmml">i</mi><mn mathsize="90%" id="alg0.l7.m1.1.1.1.1.3.3.3.3" xref="alg0.l7.m1.1.1.1.1.3.3.3.3.cmml">0</mn></msubsup></mrow><mo mathsize="90%" stretchy="false" id="alg0.l7.m1.1.1.1.1.4" xref="alg0.l7.m1.1.1.1.1.4.cmml">←</mo><mrow id="alg0.l7.m1.1.1.1.1.5" xref="alg0.l7.m1.1.1.1.1.5.cmml"><msub id="alg0.l7.m1.1.1.1.1.5.2" xref="alg0.l7.m1.1.1.1.1.5.2.cmml"><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.5.2.2" xref="alg0.l7.m1.1.1.1.1.5.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.5.2.3" xref="alg0.l7.m1.1.1.1.1.5.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l7.m1.1.1.1.1.5.1" xref="alg0.l7.m1.1.1.1.1.5.1.cmml">∪</mo><msubsup id="alg0.l7.m1.1.1.1.1.5.3" xref="alg0.l7.m1.1.1.1.1.5.3.cmml"><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.5.3.2.2" xref="alg0.l7.m1.1.1.1.1.5.3.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.5.3.2.3" xref="alg0.l7.m1.1.1.1.1.5.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l7.m1.1.1.1.1.5.3.3" xref="alg0.l7.m1.1.1.1.1.5.3.3.cmml">b</mi></msubsup></mrow></mrow><mo mathsize="90%" id="alg0.l7.m1.2.2.2.3" xref="alg0.l7.m1.2.2.3a.cmml">,</mo><mrow id="alg0.l7.m1.2.2.2.2.2" xref="alg0.l7.m1.2.2.2.2.3.cmml"><mrow id="alg0.l7.m1.2.2.2.2.1.1" xref="alg0.l7.m1.2.2.2.2.1.1.cmml"><msub id="alg0.l7.m1.2.2.2.2.1.1.2" xref="alg0.l7.m1.2.2.2.2.1.1.2.cmml"><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.1.1.2.2" xref="alg0.l7.m1.2.2.2.2.1.1.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.1.1.2.3" xref="alg0.l7.m1.2.2.2.2.1.1.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l7.m1.2.2.2.2.1.1.1" xref="alg0.l7.m1.2.2.2.2.1.1.1.cmml">∪</mo><msubsup id="alg0.l7.m1.2.2.2.2.1.1.3" xref="alg0.l7.m1.2.2.2.2.1.1.3.cmml"><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.1.1.3.2.2" xref="alg0.l7.m1.2.2.2.2.1.1.3.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.1.1.3.2.3" xref="alg0.l7.m1.2.2.2.2.1.1.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.1.1.3.3" xref="alg0.l7.m1.2.2.2.2.1.1.3.3.cmml">b</mi></msubsup></mrow><mo mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.3" xref="alg0.l7.m1.2.2.2.2.3.cmml">,</mo><mrow id="alg0.l7.m1.2.2.2.2.2.2" xref="alg0.l7.m1.2.2.2.2.2.2.cmml"><msubsup id="alg0.l7.m1.2.2.2.2.2.2.2" xref="alg0.l7.m1.2.2.2.2.2.2.2.cmml"><mover accent="true" id="alg0.l7.m1.2.2.2.2.2.2.2.2.2" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.2.2.2.2" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.2.2.2.1" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.2.2.3" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.3.cmml">i</mi><mn mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.2.3" xref="alg0.l7.m1.2.2.2.2.2.2.2.3.cmml">0</mn></msubsup><mo mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.1" xref="alg0.l7.m1.2.2.2.2.2.2.1.cmml">∪</mo><msubsup id="alg0.l7.m1.2.2.2.2.2.2.3" xref="alg0.l7.m1.2.2.2.2.2.2.3.cmml"><mover accent="true" id="alg0.l7.m1.2.2.2.2.2.2.3.2.2" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.2.cmml"><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.3.2.2.2" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.3.2.2.1" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.3.2.3" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l7.m1.2.2.2.2.2.2.3.3" xref="alg0.l7.m1.2.2.2.2.2.2.3.3.cmml">b</mi></msubsup></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l7.m1.2b"><apply id="alg0.l7.m1.2.2.3.cmml" xref="alg0.l7.m1.2.2.2"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.3a.cmml" xref="alg0.l7.m1.2.2.2.3">formulae-sequence</csymbol><apply id="alg0.l7.m1.1.1.1.1.cmml" xref="alg0.l7.m1.1.1.1.1"><ci id="alg0.l7.m1.1.1.1.1.4.cmml" xref="alg0.l7.m1.1.1.1.1.4">←</ci><list id="alg0.l7.m1.1.1.1.1.3.4.cmml" xref="alg0.l7.m1.1.1.1.1.3.3"><apply id="alg0.l7.m1.1.1.1.1.1.1.1.cmml" xref="alg0.l7.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.1.1.1.1.cmml" xref="alg0.l7.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l7.m1.1.1.1.1.1.1.1.2.cmml" xref="alg0.l7.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg0.l7.m1.1.1.1.1.1.1.1.3.cmml" xref="alg0.l7.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg0.l7.m1.1.1.1.1.2.2.2.cmml" xref="alg0.l7.m1.1.1.1.1.2.2.2"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.2.2.2.1.cmml" xref="alg0.l7.m1.1.1.1.1.2.2.2">subscript</csymbol><ci id="alg0.l7.m1.1.1.1.1.2.2.2.2.cmml" xref="alg0.l7.m1.1.1.1.1.2.2.2.2">𝒚</ci><ci id="alg0.l7.m1.1.1.1.1.2.2.2.3.cmml" xref="alg0.l7.m1.1.1.1.1.2.2.2.3">𝑖</ci></apply><apply id="alg0.l7.m1.1.1.1.1.3.3.3.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.3.3.3.1.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3">superscript</csymbol><apply id="alg0.l7.m1.1.1.1.1.3.3.3.2.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.3.3.3.2.1.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3">subscript</csymbol><apply id="alg0.l7.m1.1.1.1.1.3.3.3.2.2.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.2"><ci id="alg0.l7.m1.1.1.1.1.3.3.3.2.2.1.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.2.1">~</ci><ci id="alg0.l7.m1.1.1.1.1.3.3.3.2.2.2.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.2.2">𝒚</ci></apply><ci id="alg0.l7.m1.1.1.1.1.3.3.3.2.3.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3.2.3">𝑖</ci></apply><cn type="integer" id="alg0.l7.m1.1.1.1.1.3.3.3.3.cmml" xref="alg0.l7.m1.1.1.1.1.3.3.3.3">0</cn></apply></list><apply id="alg0.l7.m1.1.1.1.1.5.cmml" xref="alg0.l7.m1.1.1.1.1.5"><union id="alg0.l7.m1.1.1.1.1.5.1.cmml" xref="alg0.l7.m1.1.1.1.1.5.1"></union><apply id="alg0.l7.m1.1.1.1.1.5.2.cmml" xref="alg0.l7.m1.1.1.1.1.5.2"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.5.2.1.cmml" xref="alg0.l7.m1.1.1.1.1.5.2">subscript</csymbol><ci id="alg0.l7.m1.1.1.1.1.5.2.2.cmml" xref="alg0.l7.m1.1.1.1.1.5.2.2">𝒙</ci><ci id="alg0.l7.m1.1.1.1.1.5.2.3.cmml" xref="alg0.l7.m1.1.1.1.1.5.2.3">𝑖</ci></apply><apply id="alg0.l7.m1.1.1.1.1.5.3.cmml" xref="alg0.l7.m1.1.1.1.1.5.3"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.5.3.1.cmml" xref="alg0.l7.m1.1.1.1.1.5.3">superscript</csymbol><apply id="alg0.l7.m1.1.1.1.1.5.3.2.cmml" xref="alg0.l7.m1.1.1.1.1.5.3"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.1.1.5.3.2.1.cmml" xref="alg0.l7.m1.1.1.1.1.5.3">subscript</csymbol><ci id="alg0.l7.m1.1.1.1.1.5.3.2.2.cmml" xref="alg0.l7.m1.1.1.1.1.5.3.2.2">𝒙</ci><ci id="alg0.l7.m1.1.1.1.1.5.3.2.3.cmml" xref="alg0.l7.m1.1.1.1.1.5.3.2.3">𝑖</ci></apply><ci id="alg0.l7.m1.1.1.1.1.5.3.3.cmml" xref="alg0.l7.m1.1.1.1.1.5.3.3">𝑏</ci></apply></apply></apply><list id="alg0.l7.m1.2.2.2.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.2"><apply id="alg0.l7.m1.2.2.2.2.1.1.cmml" xref="alg0.l7.m1.2.2.2.2.1.1"><union id="alg0.l7.m1.2.2.2.2.1.1.1.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.1"></union><apply id="alg0.l7.m1.2.2.2.2.1.1.2.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.2"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.1.1.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.2">subscript</csymbol><ci id="alg0.l7.m1.2.2.2.2.1.1.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.2.2">𝒚</ci><ci id="alg0.l7.m1.2.2.2.2.1.1.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.2.3">𝑖</ci></apply><apply id="alg0.l7.m1.2.2.2.2.1.1.3.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.1.1.3.1.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3">superscript</csymbol><apply id="alg0.l7.m1.2.2.2.2.1.1.3.2.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.1.1.3.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3">subscript</csymbol><ci id="alg0.l7.m1.2.2.2.2.1.1.3.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3.2.2">𝒚</ci><ci id="alg0.l7.m1.2.2.2.2.1.1.3.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3.2.3">𝑖</ci></apply><ci id="alg0.l7.m1.2.2.2.2.1.1.3.3.cmml" xref="alg0.l7.m1.2.2.2.2.1.1.3.3">𝑏</ci></apply></apply><apply id="alg0.l7.m1.2.2.2.2.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2"><union id="alg0.l7.m1.2.2.2.2.2.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.1"></union><apply id="alg0.l7.m1.2.2.2.2.2.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.2.2.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2">superscript</csymbol><apply id="alg0.l7.m1.2.2.2.2.2.2.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.2.2.2.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2">subscript</csymbol><apply id="alg0.l7.m1.2.2.2.2.2.2.2.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.2"><ci id="alg0.l7.m1.2.2.2.2.2.2.2.2.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.2.1">~</ci><ci id="alg0.l7.m1.2.2.2.2.2.2.2.2.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.2.2">𝒚</ci></apply><ci id="alg0.l7.m1.2.2.2.2.2.2.2.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2.2.3">𝑖</ci></apply><cn type="integer" id="alg0.l7.m1.2.2.2.2.2.2.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.2.3">0</cn></apply><apply id="alg0.l7.m1.2.2.2.2.2.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.2.2.3.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3">superscript</csymbol><apply id="alg0.l7.m1.2.2.2.2.2.2.3.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3"><csymbol cd="ambiguous" id="alg0.l7.m1.2.2.2.2.2.2.3.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3">subscript</csymbol><apply id="alg0.l7.m1.2.2.2.2.2.2.3.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.2"><ci id="alg0.l7.m1.2.2.2.2.2.2.3.2.2.1.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.2.1">~</ci><ci id="alg0.l7.m1.2.2.2.2.2.2.3.2.2.2.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.2.2">𝒚</ci></apply><ci id="alg0.l7.m1.2.2.2.2.2.2.3.2.3.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3.2.3">𝑖</ci></apply><ci id="alg0.l7.m1.2.2.2.2.2.2.3.3.cmml" xref="alg0.l7.m1.2.2.2.2.2.2.3.3">𝑏</ci></apply></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l7.m1.2c">{\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{0}\leftarrow{\bm{x}}_{i}\cup{\bm{x}}_{i}^{b},{\bm{y}}_{i}\cup{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{0}\cup\tilde{{\bm{y}}}_{i}^{b}</annotation></semantics></math><span id="alg0.l7.2" class="ltx_text" style="font-size:90%;">

</span>
</div>
<div id="alg0.l8" class="ltx_listingline">
<span id="alg0.l8.2" class="ltx_text" style="font-size:90%;">    </span><em id="alg0.l8.1" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg0.l8.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg0.l8.1.m1.1a"><mo id="alg0.l8.1.m1.1.1" xref="alg0.l8.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg0.l8.1.m1.1b"><ci id="alg0.l8.1.m1.1.1.cmml" xref="alg0.l8.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l8.1.m1.1c">\triangleright</annotation></semantics></math> Generate low energy solutions through optimization:</em><span id="alg0.l8.3" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l9" class="ltx_listingline">
<span id="alg0.l9.1" class="ltx_text" style="font-size:90%;">    </span><span id="alg0.l9.2" class="ltx_text ltx_font_bold" style="font-size:90%;">for</span><span id="alg0.l9.3" class="ltx_text" style="font-size:90%;"> sample step </span><math id="alg0.l9.m1.1" class="ltx_Math" alttext="n=1" display="inline"><semantics id="alg0.l9.m1.1a"><mrow id="alg0.l9.m1.1.1" xref="alg0.l9.m1.1.1.cmml"><mi mathsize="90%" id="alg0.l9.m1.1.1.2" xref="alg0.l9.m1.1.1.2.cmml">n</mi><mo mathsize="90%" id="alg0.l9.m1.1.1.1" xref="alg0.l9.m1.1.1.1.cmml">=</mo><mn mathsize="90%" id="alg0.l9.m1.1.1.3" xref="alg0.l9.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="alg0.l9.m1.1b"><apply id="alg0.l9.m1.1.1.cmml" xref="alg0.l9.m1.1.1"><eq id="alg0.l9.m1.1.1.1.cmml" xref="alg0.l9.m1.1.1.1"></eq><ci id="alg0.l9.m1.1.1.2.cmml" xref="alg0.l9.m1.1.1.2">𝑛</ci><cn type="integer" id="alg0.l9.m1.1.1.3.cmml" xref="alg0.l9.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m1.1c">n=1</annotation></semantics></math><span id="alg0.l9.4" class="ltx_text" style="font-size:90%;"> to </span><math id="alg0.l9.m2.1" class="ltx_Math" alttext="N" display="inline"><semantics id="alg0.l9.m2.1a"><mi mathsize="90%" id="alg0.l9.m2.1.1" xref="alg0.l9.m2.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="alg0.l9.m2.1b"><ci id="alg0.l9.m2.1.1.cmml" xref="alg0.l9.m2.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m2.1c">N</annotation></semantics></math><span id="alg0.l9.5" class="ltx_text" style="font-size:90%;"> </span><span id="alg0.l9.6" class="ltx_text ltx_font_bold" style="font-size:90%;">do</span><span id="alg0.l9.7" class="ltx_text" style="font-size:90%;">


</span>
</div>
<div id="alg0.l10" class="ltx_listingline">
<span id="alg0.l10.1" class="ltx_text" style="font-size:90%;">       </span><math id="alg0.l10.m1.2" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}^{n}\leftarrow\tilde{{\bm{y}}}_{i}^{n-1}-\lambda\nabla_{\bm{y}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i}^{n-1})" display="inline"><semantics id="alg0.l10.m1.2a"><mrow id="alg0.l10.m1.2.2" xref="alg0.l10.m1.2.2.cmml"><msubsup id="alg0.l10.m1.2.2.4" xref="alg0.l10.m1.2.2.4.cmml"><mover accent="true" id="alg0.l10.m1.2.2.4.2.2" xref="alg0.l10.m1.2.2.4.2.2.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.4.2.2.2" xref="alg0.l10.m1.2.2.4.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l10.m1.2.2.4.2.2.1" xref="alg0.l10.m1.2.2.4.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l10.m1.2.2.4.2.3" xref="alg0.l10.m1.2.2.4.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l10.m1.2.2.4.3" xref="alg0.l10.m1.2.2.4.3.cmml">n</mi></msubsup><mo mathsize="90%" stretchy="false" id="alg0.l10.m1.2.2.3" xref="alg0.l10.m1.2.2.3.cmml">←</mo><mrow id="alg0.l10.m1.2.2.2" xref="alg0.l10.m1.2.2.2.cmml"><msubsup id="alg0.l10.m1.2.2.2.4" xref="alg0.l10.m1.2.2.2.4.cmml"><mover accent="true" id="alg0.l10.m1.2.2.2.4.2.2" xref="alg0.l10.m1.2.2.2.4.2.2.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.2.4.2.2.2" xref="alg0.l10.m1.2.2.2.4.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l10.m1.2.2.2.4.2.2.1" xref="alg0.l10.m1.2.2.2.4.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l10.m1.2.2.2.4.2.3" xref="alg0.l10.m1.2.2.2.4.2.3.cmml">i</mi><mrow id="alg0.l10.m1.2.2.2.4.3" xref="alg0.l10.m1.2.2.2.4.3.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.2.4.3.2" xref="alg0.l10.m1.2.2.2.4.3.2.cmml">n</mi><mo mathsize="90%" id="alg0.l10.m1.2.2.2.4.3.1" xref="alg0.l10.m1.2.2.2.4.3.1.cmml">−</mo><mn mathsize="90%" id="alg0.l10.m1.2.2.2.4.3.3" xref="alg0.l10.m1.2.2.2.4.3.3.cmml">1</mn></mrow></msubsup><mo mathsize="90%" id="alg0.l10.m1.2.2.2.3" xref="alg0.l10.m1.2.2.2.3.cmml">−</mo><mrow id="alg0.l10.m1.2.2.2.2" xref="alg0.l10.m1.2.2.2.2.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.4" xref="alg0.l10.m1.2.2.2.2.4.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="alg0.l10.m1.2.2.2.2.3" xref="alg0.l10.m1.2.2.2.2.3.cmml">​</mo><mrow id="alg0.l10.m1.2.2.2.2.5" xref="alg0.l10.m1.2.2.2.2.5.cmml"><msub id="alg0.l10.m1.2.2.2.2.5.1" xref="alg0.l10.m1.2.2.2.2.5.1.cmml"><mo mathsize="90%" rspace="0.167em" id="alg0.l10.m1.2.2.2.2.5.1.2" xref="alg0.l10.m1.2.2.2.2.5.1.2.cmml">∇</mo><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.5.1.3" xref="alg0.l10.m1.2.2.2.2.5.1.3.cmml">𝒚</mi></msub><msub id="alg0.l10.m1.2.2.2.2.5.2" xref="alg0.l10.m1.2.2.2.2.5.2.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.5.2.2" xref="alg0.l10.m1.2.2.2.2.5.2.2.cmml">E</mi><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.5.2.3" xref="alg0.l10.m1.2.2.2.2.5.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="alg0.l10.m1.2.2.2.2.3a" xref="alg0.l10.m1.2.2.2.2.3.cmml">​</mo><mrow id="alg0.l10.m1.2.2.2.2.2.2" xref="alg0.l10.m1.2.2.2.2.2.3.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.3" xref="alg0.l10.m1.2.2.2.2.2.3.cmml">(</mo><msub id="alg0.l10.m1.1.1.1.1.1.1.1" xref="alg0.l10.m1.1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg0.l10.m1.1.1.1.1.1.1.1.2" xref="alg0.l10.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l10.m1.1.1.1.1.1.1.1.3" xref="alg0.l10.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.4" xref="alg0.l10.m1.2.2.2.2.2.3.cmml">,</mo><msubsup id="alg0.l10.m1.2.2.2.2.2.2.2" xref="alg0.l10.m1.2.2.2.2.2.2.2.cmml"><mover accent="true" id="alg0.l10.m1.2.2.2.2.2.2.2.2.2" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.2.2.2.2" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.2.2.2.1" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.2.2.3" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.3.cmml">i</mi><mrow id="alg0.l10.m1.2.2.2.2.2.2.2.3" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.cmml"><mi mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.2.3.2" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.2.cmml">n</mi><mo mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.2.3.1" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.1.cmml">−</mo><mn mathsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.2.3.3" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.3.cmml">1</mn></mrow></msubsup><mo maxsize="90%" minsize="90%" id="alg0.l10.m1.2.2.2.2.2.2.5" xref="alg0.l10.m1.2.2.2.2.2.3.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l10.m1.2b"><apply id="alg0.l10.m1.2.2.cmml" xref="alg0.l10.m1.2.2"><ci id="alg0.l10.m1.2.2.3.cmml" xref="alg0.l10.m1.2.2.3">←</ci><apply id="alg0.l10.m1.2.2.4.cmml" xref="alg0.l10.m1.2.2.4"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.4.1.cmml" xref="alg0.l10.m1.2.2.4">superscript</csymbol><apply id="alg0.l10.m1.2.2.4.2.cmml" xref="alg0.l10.m1.2.2.4"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.4.2.1.cmml" xref="alg0.l10.m1.2.2.4">subscript</csymbol><apply id="alg0.l10.m1.2.2.4.2.2.cmml" xref="alg0.l10.m1.2.2.4.2.2"><ci id="alg0.l10.m1.2.2.4.2.2.1.cmml" xref="alg0.l10.m1.2.2.4.2.2.1">~</ci><ci id="alg0.l10.m1.2.2.4.2.2.2.cmml" xref="alg0.l10.m1.2.2.4.2.2.2">𝒚</ci></apply><ci id="alg0.l10.m1.2.2.4.2.3.cmml" xref="alg0.l10.m1.2.2.4.2.3">𝑖</ci></apply><ci id="alg0.l10.m1.2.2.4.3.cmml" xref="alg0.l10.m1.2.2.4.3">𝑛</ci></apply><apply id="alg0.l10.m1.2.2.2.cmml" xref="alg0.l10.m1.2.2.2"><minus id="alg0.l10.m1.2.2.2.3.cmml" xref="alg0.l10.m1.2.2.2.3"></minus><apply id="alg0.l10.m1.2.2.2.4.cmml" xref="alg0.l10.m1.2.2.2.4"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.2.4.1.cmml" xref="alg0.l10.m1.2.2.2.4">superscript</csymbol><apply id="alg0.l10.m1.2.2.2.4.2.cmml" xref="alg0.l10.m1.2.2.2.4"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.2.4.2.1.cmml" xref="alg0.l10.m1.2.2.2.4">subscript</csymbol><apply id="alg0.l10.m1.2.2.2.4.2.2.cmml" xref="alg0.l10.m1.2.2.2.4.2.2"><ci id="alg0.l10.m1.2.2.2.4.2.2.1.cmml" xref="alg0.l10.m1.2.2.2.4.2.2.1">~</ci><ci id="alg0.l10.m1.2.2.2.4.2.2.2.cmml" xref="alg0.l10.m1.2.2.2.4.2.2.2">𝒚</ci></apply><ci id="alg0.l10.m1.2.2.2.4.2.3.cmml" xref="alg0.l10.m1.2.2.2.4.2.3">𝑖</ci></apply><apply id="alg0.l10.m1.2.2.2.4.3.cmml" xref="alg0.l10.m1.2.2.2.4.3"><minus id="alg0.l10.m1.2.2.2.4.3.1.cmml" xref="alg0.l10.m1.2.2.2.4.3.1"></minus><ci id="alg0.l10.m1.2.2.2.4.3.2.cmml" xref="alg0.l10.m1.2.2.2.4.3.2">𝑛</ci><cn type="integer" id="alg0.l10.m1.2.2.2.4.3.3.cmml" xref="alg0.l10.m1.2.2.2.4.3.3">1</cn></apply></apply><apply id="alg0.l10.m1.2.2.2.2.cmml" xref="alg0.l10.m1.2.2.2.2"><times id="alg0.l10.m1.2.2.2.2.3.cmml" xref="alg0.l10.m1.2.2.2.2.3"></times><ci id="alg0.l10.m1.2.2.2.2.4.cmml" xref="alg0.l10.m1.2.2.2.2.4">𝜆</ci><apply id="alg0.l10.m1.2.2.2.2.5.cmml" xref="alg0.l10.m1.2.2.2.2.5"><apply id="alg0.l10.m1.2.2.2.2.5.1.cmml" xref="alg0.l10.m1.2.2.2.2.5.1"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.2.2.5.1.1.cmml" xref="alg0.l10.m1.2.2.2.2.5.1">subscript</csymbol><ci id="alg0.l10.m1.2.2.2.2.5.1.2.cmml" xref="alg0.l10.m1.2.2.2.2.5.1.2">∇</ci><ci id="alg0.l10.m1.2.2.2.2.5.1.3.cmml" xref="alg0.l10.m1.2.2.2.2.5.1.3">𝒚</ci></apply><apply id="alg0.l10.m1.2.2.2.2.5.2.cmml" xref="alg0.l10.m1.2.2.2.2.5.2"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.2.2.5.2.1.cmml" xref="alg0.l10.m1.2.2.2.2.5.2">subscript</csymbol><ci id="alg0.l10.m1.2.2.2.2.5.2.2.cmml" xref="alg0.l10.m1.2.2.2.2.5.2.2">𝐸</ci><ci id="alg0.l10.m1.2.2.2.2.5.2.3.cmml" xref="alg0.l10.m1.2.2.2.2.5.2.3">𝜃</ci></apply></apply><interval closure="open" id="alg0.l10.m1.2.2.2.2.2.3.cmml" xref="alg0.l10.m1.2.2.2.2.2.2"><apply id="alg0.l10.m1.1.1.1.1.1.1.1.cmml" xref="alg0.l10.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l10.m1.1.1.1.1.1.1.1.1.cmml" xref="alg0.l10.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l10.m1.1.1.1.1.1.1.1.2.cmml" xref="alg0.l10.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg0.l10.m1.1.1.1.1.1.1.1.3.cmml" xref="alg0.l10.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg0.l10.m1.2.2.2.2.2.2.2.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.2.2.2.2.2.1.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2">superscript</csymbol><apply id="alg0.l10.m1.2.2.2.2.2.2.2.2.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l10.m1.2.2.2.2.2.2.2.2.1.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2">subscript</csymbol><apply id="alg0.l10.m1.2.2.2.2.2.2.2.2.2.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.2"><ci id="alg0.l10.m1.2.2.2.2.2.2.2.2.2.1.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.2.1">~</ci><ci id="alg0.l10.m1.2.2.2.2.2.2.2.2.2.2.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.2.2">𝒚</ci></apply><ci id="alg0.l10.m1.2.2.2.2.2.2.2.2.3.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="alg0.l10.m1.2.2.2.2.2.2.2.3.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.3"><minus id="alg0.l10.m1.2.2.2.2.2.2.2.3.1.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.1"></minus><ci id="alg0.l10.m1.2.2.2.2.2.2.2.3.2.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.2">𝑛</ci><cn type="integer" id="alg0.l10.m1.2.2.2.2.2.2.2.3.3.cmml" xref="alg0.l10.m1.2.2.2.2.2.2.2.3.3">1</cn></apply></apply></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l10.m1.2c">\tilde{{\bm{y}}}_{i}^{n}\leftarrow\tilde{{\bm{y}}}_{i}^{n-1}-\lambda\nabla_{\bm{y}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i}^{n-1})</annotation></semantics></math><span id="alg0.l10.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l11" class="ltx_listingline">
<span id="alg0.l11.1" class="ltx_text" style="font-size:90%;">    </span><span id="alg0.l11.2" class="ltx_text ltx_font_bold" style="font-size:90%;">end</span><span id="alg0.l11.3" class="ltx_text" style="font-size:90%;"> </span><span id="alg0.l11.4" class="ltx_text ltx_font_bold" style="font-size:90%;">for</span><span id="alg0.l11.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l12" class="ltx_listingline">
<span id="alg0.l12.4" class="ltx_text" style="font-size:90%;">    </span><em id="alg0.l12.3" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg0.l12.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg0.l12.1.m1.1a"><mo id="alg0.l12.1.m1.1.1" xref="alg0.l12.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg0.l12.1.m1.1b"><ci id="alg0.l12.1.m1.1.1.cmml" xref="alg0.l12.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l12.1.m1.1c">\triangleright</annotation></semantics></math> Optimize objective <math id="alg0.l12.2.m2.1" class="ltx_Math" alttext="\mathcal{L}_{\text{MSE}}" display="inline"><semantics id="alg0.l12.2.m2.1a"><msub id="alg0.l12.2.m2.1.1" xref="alg0.l12.2.m2.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l12.2.m2.1.1.2" xref="alg0.l12.2.m2.1.1.2.cmml">ℒ</mi><mtext class="ltx_mathvariant_italic" id="alg0.l12.2.m2.1.1.3" xref="alg0.l12.2.m2.1.1.3b.cmml"><em id="alg0.l12.2.m2.1.1.3.1nest" class="ltx_emph" style="font-size:70%;">MSE</em></mtext></msub><annotation-xml encoding="MathML-Content" id="alg0.l12.2.m2.1b"><apply id="alg0.l12.2.m2.1.1.cmml" xref="alg0.l12.2.m2.1.1"><csymbol cd="ambiguous" id="alg0.l12.2.m2.1.1.1.cmml" xref="alg0.l12.2.m2.1.1">subscript</csymbol><ci id="alg0.l12.2.m2.1.1.2.cmml" xref="alg0.l12.2.m2.1.1.2">ℒ</ci><ci id="alg0.l12.2.m2.1.1.3b.cmml" xref="alg0.l12.2.m2.1.1.3"><mtext class="ltx_mathvariant_italic" mathsize="70%" id="alg0.l12.2.m2.1.1.3.cmml" xref="alg0.l12.2.m2.1.1.3"><em id="alg0.l12.2.m2.1.1.3.1anest" class="ltx_emph" style="font-size:70%;">MSE</em></mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l12.2.m2.1c">\mathcal{L}_{\text{MSE}}</annotation></semantics></math> wrt <math id="alg0.l12.3.m3.1" class="ltx_Math" alttext="\theta" display="inline"><semantics id="alg0.l12.3.m3.1a"><mi id="alg0.l12.3.m3.1.1" xref="alg0.l12.3.m3.1.1.cmml">θ</mi><annotation-xml encoding="MathML-Content" id="alg0.l12.3.m3.1b"><ci id="alg0.l12.3.m3.1.1.cmml" xref="alg0.l12.3.m3.1.1">𝜃</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l12.3.m3.1c">\theta</annotation></semantics></math>:</em><span id="alg0.l12.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l13" class="ltx_listingline">
<span id="alg0.l13.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg0.l13.m1.1" class="ltx_Math" alttext="\Delta\theta\leftarrow\nabla_{\theta}\sum_{n=1}^{N}\|\tilde{{\bm{y}}}^{N}_{i}-{\bm{y}}_{i}\|^{2}" display="inline"><semantics id="alg0.l13.m1.1a"><mrow id="alg0.l13.m1.1.1" xref="alg0.l13.m1.1.1.cmml"><mrow id="alg0.l13.m1.1.1.3" xref="alg0.l13.m1.1.1.3.cmml"><mi mathsize="90%" mathvariant="normal" id="alg0.l13.m1.1.1.3.2" xref="alg0.l13.m1.1.1.3.2.cmml">Δ</mi><mo lspace="0em" rspace="0em" id="alg0.l13.m1.1.1.3.1" xref="alg0.l13.m1.1.1.3.1.cmml">​</mo><mi mathsize="90%" id="alg0.l13.m1.1.1.3.3" xref="alg0.l13.m1.1.1.3.3.cmml">θ</mi></mrow><mo mathsize="90%" stretchy="false" id="alg0.l13.m1.1.1.2" xref="alg0.l13.m1.1.1.2.cmml">←</mo><mrow id="alg0.l13.m1.1.1.1" xref="alg0.l13.m1.1.1.1.cmml"><msub id="alg0.l13.m1.1.1.1.3" xref="alg0.l13.m1.1.1.1.3.cmml"><mo mathsize="90%" id="alg0.l13.m1.1.1.1.3.2" xref="alg0.l13.m1.1.1.1.3.2.cmml">∇</mo><mi mathsize="90%" id="alg0.l13.m1.1.1.1.3.3" xref="alg0.l13.m1.1.1.1.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="alg0.l13.m1.1.1.1.2" xref="alg0.l13.m1.1.1.1.2.cmml">​</mo><mrow id="alg0.l13.m1.1.1.1.1" xref="alg0.l13.m1.1.1.1.1.cmml"><msubsup id="alg0.l13.m1.1.1.1.1.2" xref="alg0.l13.m1.1.1.1.1.2.cmml"><mo maxsize="90%" minsize="90%" rspace="0em" stretchy="true" id="alg0.l13.m1.1.1.1.1.2.2.2" xref="alg0.l13.m1.1.1.1.1.2.2.2.cmml">∑</mo><mrow id="alg0.l13.m1.1.1.1.1.2.2.3" xref="alg0.l13.m1.1.1.1.1.2.2.3.cmml"><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.2.2.3.2" xref="alg0.l13.m1.1.1.1.1.2.2.3.2.cmml">n</mi><mo mathsize="90%" id="alg0.l13.m1.1.1.1.1.2.2.3.1" xref="alg0.l13.m1.1.1.1.1.2.2.3.1.cmml">=</mo><mn mathsize="90%" id="alg0.l13.m1.1.1.1.1.2.2.3.3" xref="alg0.l13.m1.1.1.1.1.2.2.3.3.cmml">1</mn></mrow><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.2.3" xref="alg0.l13.m1.1.1.1.1.2.3.cmml">N</mi></msubsup><msup id="alg0.l13.m1.1.1.1.1.1" xref="alg0.l13.m1.1.1.1.1.1.cmml"><mrow id="alg0.l13.m1.1.1.1.1.1.1.1" xref="alg0.l13.m1.1.1.1.1.1.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.2" xref="alg0.l13.m1.1.1.1.1.1.1.2.1.cmml">‖</mo><mrow id="alg0.l13.m1.1.1.1.1.1.1.1.1" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.cmml"><msubsup id="alg0.l13.m1.1.1.1.1.1.1.1.1.2" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.cmml"><mover accent="true" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.cmml"><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.2" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.1" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.3" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.3" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.3.cmml">N</mi></msubsup><mo mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.1" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.1.cmml">−</mo><msub id="alg0.l13.m1.1.1.1.1.1.1.1.1.3" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3.cmml"><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.3.2" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.1.3.3" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo maxsize="90%" minsize="90%" id="alg0.l13.m1.1.1.1.1.1.1.1.3" xref="alg0.l13.m1.1.1.1.1.1.1.2.1.cmml">‖</mo></mrow><mn mathsize="90%" id="alg0.l13.m1.1.1.1.1.1.3" xref="alg0.l13.m1.1.1.1.1.1.3.cmml">2</mn></msup></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l13.m1.1b"><apply id="alg0.l13.m1.1.1.cmml" xref="alg0.l13.m1.1.1"><ci id="alg0.l13.m1.1.1.2.cmml" xref="alg0.l13.m1.1.1.2">←</ci><apply id="alg0.l13.m1.1.1.3.cmml" xref="alg0.l13.m1.1.1.3"><times id="alg0.l13.m1.1.1.3.1.cmml" xref="alg0.l13.m1.1.1.3.1"></times><ci id="alg0.l13.m1.1.1.3.2.cmml" xref="alg0.l13.m1.1.1.3.2">Δ</ci><ci id="alg0.l13.m1.1.1.3.3.cmml" xref="alg0.l13.m1.1.1.3.3">𝜃</ci></apply><apply id="alg0.l13.m1.1.1.1.cmml" xref="alg0.l13.m1.1.1.1"><times id="alg0.l13.m1.1.1.1.2.cmml" xref="alg0.l13.m1.1.1.1.2"></times><apply id="alg0.l13.m1.1.1.1.3.cmml" xref="alg0.l13.m1.1.1.1.3"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.3.1.cmml" xref="alg0.l13.m1.1.1.1.3">subscript</csymbol><ci id="alg0.l13.m1.1.1.1.3.2.cmml" xref="alg0.l13.m1.1.1.1.3.2">∇</ci><ci id="alg0.l13.m1.1.1.1.3.3.cmml" xref="alg0.l13.m1.1.1.1.3.3">𝜃</ci></apply><apply id="alg0.l13.m1.1.1.1.1.cmml" xref="alg0.l13.m1.1.1.1.1"><apply id="alg0.l13.m1.1.1.1.1.2.cmml" xref="alg0.l13.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.1.2.1.cmml" xref="alg0.l13.m1.1.1.1.1.2">superscript</csymbol><apply id="alg0.l13.m1.1.1.1.1.2.2.cmml" xref="alg0.l13.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.1.2.2.1.cmml" xref="alg0.l13.m1.1.1.1.1.2">subscript</csymbol><sum id="alg0.l13.m1.1.1.1.1.2.2.2.cmml" xref="alg0.l13.m1.1.1.1.1.2.2.2"></sum><apply id="alg0.l13.m1.1.1.1.1.2.2.3.cmml" xref="alg0.l13.m1.1.1.1.1.2.2.3"><eq id="alg0.l13.m1.1.1.1.1.2.2.3.1.cmml" xref="alg0.l13.m1.1.1.1.1.2.2.3.1"></eq><ci id="alg0.l13.m1.1.1.1.1.2.2.3.2.cmml" xref="alg0.l13.m1.1.1.1.1.2.2.3.2">𝑛</ci><cn type="integer" id="alg0.l13.m1.1.1.1.1.2.2.3.3.cmml" xref="alg0.l13.m1.1.1.1.1.2.2.3.3">1</cn></apply></apply><ci id="alg0.l13.m1.1.1.1.1.2.3.cmml" xref="alg0.l13.m1.1.1.1.1.2.3">𝑁</ci></apply><apply id="alg0.l13.m1.1.1.1.1.1.cmml" xref="alg0.l13.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.1.1.2.cmml" xref="alg0.l13.m1.1.1.1.1.1">superscript</csymbol><apply id="alg0.l13.m1.1.1.1.1.1.1.2.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1"><csymbol cd="latexml" id="alg0.l13.m1.1.1.1.1.1.1.2.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.2">norm</csymbol><apply id="alg0.l13.m1.1.1.1.1.1.1.1.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1"><minus id="alg0.l13.m1.1.1.1.1.1.1.1.1.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.1"></minus><apply id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2">subscript</csymbol><apply id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2">superscript</csymbol><apply id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2"><ci id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.1">~</ci><ci id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.2.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.2.2">𝒚</ci></apply><ci id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.3.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.2.3">𝑁</ci></apply><ci id="alg0.l13.m1.1.1.1.1.1.1.1.1.2.3.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="alg0.l13.m1.1.1.1.1.1.1.1.1.3.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="alg0.l13.m1.1.1.1.1.1.1.1.1.3.1.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3">subscript</csymbol><ci id="alg0.l13.m1.1.1.1.1.1.1.1.1.3.2.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3.2">𝒚</ci><ci id="alg0.l13.m1.1.1.1.1.1.1.1.1.3.3.cmml" xref="alg0.l13.m1.1.1.1.1.1.1.1.1.3.3">𝑖</ci></apply></apply></apply><cn type="integer" id="alg0.l13.m1.1.1.1.1.1.3.cmml" xref="alg0.l13.m1.1.1.1.1.1.3">2</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l13.m1.1c">\Delta\theta\leftarrow\nabla_{\theta}\sum_{n=1}^{N}\|\tilde{{\bm{y}}}^{N}_{i}-{\bm{y}}_{i}\|^{2}</annotation></semantics></math><span id="alg0.l13.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l14" class="ltx_listingline">
<span id="alg0.l14.1" class="ltx_text" style="font-size:90%;">    Update </span><math id="alg0.l14.m1.1" class="ltx_Math" alttext="\theta" display="inline"><semantics id="alg0.l14.m1.1a"><mi mathsize="90%" id="alg0.l14.m1.1.1" xref="alg0.l14.m1.1.1.cmml">θ</mi><annotation-xml encoding="MathML-Content" id="alg0.l14.m1.1b"><ci id="alg0.l14.m1.1.1.cmml" xref="alg0.l14.m1.1.1">𝜃</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l14.m1.1c">\theta</annotation></semantics></math><span id="alg0.l14.2" class="ltx_text" style="font-size:90%;"> based on </span><math id="alg0.l14.m2.1" class="ltx_Math" alttext="\Delta\theta" display="inline"><semantics id="alg0.l14.m2.1a"><mrow id="alg0.l14.m2.1.1" xref="alg0.l14.m2.1.1.cmml"><mi mathsize="90%" mathvariant="normal" id="alg0.l14.m2.1.1.2" xref="alg0.l14.m2.1.1.2.cmml">Δ</mi><mo lspace="0em" rspace="0em" id="alg0.l14.m2.1.1.1" xref="alg0.l14.m2.1.1.1.cmml">​</mo><mi mathsize="90%" id="alg0.l14.m2.1.1.3" xref="alg0.l14.m2.1.1.3.cmml">θ</mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l14.m2.1b"><apply id="alg0.l14.m2.1.1.cmml" xref="alg0.l14.m2.1.1"><times id="alg0.l14.m2.1.1.1.cmml" xref="alg0.l14.m2.1.1.1"></times><ci id="alg0.l14.m2.1.1.2.cmml" xref="alg0.l14.m2.1.1.2">Δ</ci><ci id="alg0.l14.m2.1.1.3.cmml" xref="alg0.l14.m2.1.1.3">𝜃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l14.m2.1c">\Delta\theta</annotation></semantics></math><span id="alg0.l14.3" class="ltx_text" style="font-size:90%;"> using Adam optimizer

</span>
</div>
<div id="alg0.l15" class="ltx_listingline">
<span id="alg0.l15.3" class="ltx_text" style="font-size:90%;">    </span><em id="alg0.l15.2" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg0.l15.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg0.l15.1.m1.1a"><mo id="alg0.l15.1.m1.1.1" xref="alg0.l15.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg0.l15.1.m1.1b"><ci id="alg0.l15.1.m1.1.1.cmml" xref="alg0.l15.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l15.1.m1.1c">\triangleright</annotation></semantics></math> Update replay buffer <math id="alg0.l15.2.m2.1" class="ltx_Math" alttext="\mathcal{B}" display="inline"><semantics id="alg0.l15.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="alg0.l15.2.m2.1.1" xref="alg0.l15.2.m2.1.1.cmml">ℬ</mi><annotation-xml encoding="MathML-Content" id="alg0.l15.2.m2.1b"><ci id="alg0.l15.2.m2.1.1.cmml" xref="alg0.l15.2.m2.1.1">ℬ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l15.2.m2.1c">\mathcal{B}</annotation></semantics></math></em><span id="alg0.l15.4" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l16" class="ltx_listingline">
<span id="alg0.l16.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg0.l16.m1.3" class="ltx_Math" alttext="\mathcal{B}\leftarrow\mathcal{B}\cup({\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{N})" display="inline"><semantics id="alg0.l16.m1.3a"><mrow id="alg0.l16.m1.3.3" xref="alg0.l16.m1.3.3.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg0.l16.m1.3.3.5" xref="alg0.l16.m1.3.3.5.cmml">ℬ</mi><mo mathsize="90%" stretchy="false" id="alg0.l16.m1.3.3.4" xref="alg0.l16.m1.3.3.4.cmml">←</mo><mrow id="alg0.l16.m1.3.3.3" xref="alg0.l16.m1.3.3.3.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg0.l16.m1.3.3.3.5" xref="alg0.l16.m1.3.3.3.5.cmml">ℬ</mi><mo mathsize="90%" id="alg0.l16.m1.3.3.3.4" xref="alg0.l16.m1.3.3.3.4.cmml">∪</mo><mrow id="alg0.l16.m1.3.3.3.3.3" xref="alg0.l16.m1.3.3.3.3.4.cmml"><mo maxsize="90%" minsize="90%" id="alg0.l16.m1.3.3.3.3.3.4" xref="alg0.l16.m1.3.3.3.3.4.cmml">(</mo><msub id="alg0.l16.m1.1.1.1.1.1.1" xref="alg0.l16.m1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg0.l16.m1.1.1.1.1.1.1.2" xref="alg0.l16.m1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg0.l16.m1.1.1.1.1.1.1.3" xref="alg0.l16.m1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l16.m1.3.3.3.3.3.5" xref="alg0.l16.m1.3.3.3.3.4.cmml">,</mo><msub id="alg0.l16.m1.2.2.2.2.2.2" xref="alg0.l16.m1.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg0.l16.m1.2.2.2.2.2.2.2" xref="alg0.l16.m1.2.2.2.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg0.l16.m1.2.2.2.2.2.2.3" xref="alg0.l16.m1.2.2.2.2.2.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg0.l16.m1.3.3.3.3.3.6" xref="alg0.l16.m1.3.3.3.3.4.cmml">,</mo><msubsup id="alg0.l16.m1.3.3.3.3.3.3" xref="alg0.l16.m1.3.3.3.3.3.3.cmml"><mover accent="true" id="alg0.l16.m1.3.3.3.3.3.3.2.2" xref="alg0.l16.m1.3.3.3.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg0.l16.m1.3.3.3.3.3.3.2.2.2" xref="alg0.l16.m1.3.3.3.3.3.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg0.l16.m1.3.3.3.3.3.3.2.2.1" xref="alg0.l16.m1.3.3.3.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg0.l16.m1.3.3.3.3.3.3.2.3" xref="alg0.l16.m1.3.3.3.3.3.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg0.l16.m1.3.3.3.3.3.3.3" xref="alg0.l16.m1.3.3.3.3.3.3.3.cmml">N</mi></msubsup><mo maxsize="90%" minsize="90%" id="alg0.l16.m1.3.3.3.3.3.7" xref="alg0.l16.m1.3.3.3.3.4.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l16.m1.3b"><apply id="alg0.l16.m1.3.3.cmml" xref="alg0.l16.m1.3.3"><ci id="alg0.l16.m1.3.3.4.cmml" xref="alg0.l16.m1.3.3.4">←</ci><ci id="alg0.l16.m1.3.3.5.cmml" xref="alg0.l16.m1.3.3.5">ℬ</ci><apply id="alg0.l16.m1.3.3.3.cmml" xref="alg0.l16.m1.3.3.3"><union id="alg0.l16.m1.3.3.3.4.cmml" xref="alg0.l16.m1.3.3.3.4"></union><ci id="alg0.l16.m1.3.3.3.5.cmml" xref="alg0.l16.m1.3.3.3.5">ℬ</ci><vector id="alg0.l16.m1.3.3.3.3.4.cmml" xref="alg0.l16.m1.3.3.3.3.3"><apply id="alg0.l16.m1.1.1.1.1.1.1.cmml" xref="alg0.l16.m1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l16.m1.1.1.1.1.1.1.1.cmml" xref="alg0.l16.m1.1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l16.m1.1.1.1.1.1.1.2.cmml" xref="alg0.l16.m1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg0.l16.m1.1.1.1.1.1.1.3.cmml" xref="alg0.l16.m1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg0.l16.m1.2.2.2.2.2.2.cmml" xref="alg0.l16.m1.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l16.m1.2.2.2.2.2.2.1.cmml" xref="alg0.l16.m1.2.2.2.2.2.2">subscript</csymbol><ci id="alg0.l16.m1.2.2.2.2.2.2.2.cmml" xref="alg0.l16.m1.2.2.2.2.2.2.2">𝒚</ci><ci id="alg0.l16.m1.2.2.2.2.2.2.3.cmml" xref="alg0.l16.m1.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="alg0.l16.m1.3.3.3.3.3.3.cmml" xref="alg0.l16.m1.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg0.l16.m1.3.3.3.3.3.3.1.cmml" xref="alg0.l16.m1.3.3.3.3.3.3">superscript</csymbol><apply id="alg0.l16.m1.3.3.3.3.3.3.2.cmml" xref="alg0.l16.m1.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg0.l16.m1.3.3.3.3.3.3.2.1.cmml" xref="alg0.l16.m1.3.3.3.3.3.3">subscript</csymbol><apply id="alg0.l16.m1.3.3.3.3.3.3.2.2.cmml" xref="alg0.l16.m1.3.3.3.3.3.3.2.2"><ci id="alg0.l16.m1.3.3.3.3.3.3.2.2.1.cmml" xref="alg0.l16.m1.3.3.3.3.3.3.2.2.1">~</ci><ci id="alg0.l16.m1.3.3.3.3.3.3.2.2.2.cmml" xref="alg0.l16.m1.3.3.3.3.3.3.2.2.2">𝒚</ci></apply><ci id="alg0.l16.m1.3.3.3.3.3.3.2.3.cmml" xref="alg0.l16.m1.3.3.3.3.3.3.2.3">𝑖</ci></apply><ci id="alg0.l16.m1.3.3.3.3.3.3.3.cmml" xref="alg0.l16.m1.3.3.3.3.3.3.3">𝑁</ci></apply></vector></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l16.m1.3c">\mathcal{B}\leftarrow\mathcal{B}\cup({\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{N})</annotation></semantics></math><span id="alg0.l16.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg0.l17" class="ltx_listingline">
<span id="alg0.l17.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg0.l17.2" class="ltx_text ltx_font_bold" style="font-size:90%;">end</span><span id="alg0.l17.3" class="ltx_text" style="font-size:90%;"> </span><span id="alg0.l17.4" class="ltx_text ltx_font_bold" style="font-size:90%;">while</span>
</div>
</div>
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_float"><span id="alg1.5.1.1" class="ltx_text ltx_font_bold">Algorithm 1</span> </span> IREM training algorithm</figcaption>
</figure>
</section>
<section id="S3.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.3 </span>Analysis</h3>

<div id="S3.SS3.p1" class="ltx_para">
<p id="S3.SS3.p1.1" class="ltx_p">In this section, we provide complexity-theoretic motivation for representing iterative reasoning as an energy minimization is advantageous. We consider two approaches to represent reasoning:</p>
</div>
<div id="S3.SS3.p2" class="ltx_para">
<p id="S3.SS3.p2.5" class="ltx_p"><span id="S3.SS3.p2.5.1" class="ltx_text ltx_font_bold">Feedforward Computation.</span>  We consider reasoning as a feedforward function <math id="S3.SS3.p2.1.m1.1" class="ltx_Math" alttext="f" display="inline"><semantics id="S3.SS3.p2.1.m1.1a"><mi id="S3.SS3.p2.1.m1.1.1" xref="S3.SS3.p2.1.m1.1.1.cmml">f</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p2.1.m1.1b"><ci id="S3.SS3.p2.1.m1.1.1.cmml" xref="S3.SS3.p2.1.m1.1.1">𝑓</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p2.1.m1.1c">f</annotation></semantics></math> of the form <math id="S3.SS3.p2.2.m2.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p2.2.m2.1a"><mrow id="S3.SS3.p2.2.m2.1.2" xref="S3.SS3.p2.2.m2.1.2.cmml"><mi id="S3.SS3.p2.2.m2.1.2.2" xref="S3.SS3.p2.2.m2.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p2.2.m2.1.2.1" xref="S3.SS3.p2.2.m2.1.2.1.cmml">​</mo><mrow id="S3.SS3.p2.2.m2.1.2.3.2" xref="S3.SS3.p2.2.m2.1.2.cmml"><mo stretchy="false" id="S3.SS3.p2.2.m2.1.2.3.2.1" xref="S3.SS3.p2.2.m2.1.2.cmml">(</mo><mi id="S3.SS3.p2.2.m2.1.1" xref="S3.SS3.p2.2.m2.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p2.2.m2.1.2.3.2.2" xref="S3.SS3.p2.2.m2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p2.2.m2.1b"><apply id="S3.SS3.p2.2.m2.1.2.cmml" xref="S3.SS3.p2.2.m2.1.2"><times id="S3.SS3.p2.2.m2.1.2.1.cmml" xref="S3.SS3.p2.2.m2.1.2.1"></times><ci id="S3.SS3.p2.2.m2.1.2.2.cmml" xref="S3.SS3.p2.2.m2.1.2.2">𝑓</ci><ci id="S3.SS3.p2.2.m2.1.1.cmml" xref="S3.SS3.p2.2.m2.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p2.2.m2.1c">f({\bm{x}})</annotation></semantics></math> where <math id="S3.SS3.p2.3.m3.1" class="ltx_Math" alttext="f({\bm{x}}):\mathbb{R}^{O}\rightarrow\mathbb{R}^{M}" display="inline"><semantics id="S3.SS3.p2.3.m3.1a"><mrow id="S3.SS3.p2.3.m3.1.2" xref="S3.SS3.p2.3.m3.1.2.cmml"><mrow id="S3.SS3.p2.3.m3.1.2.2" xref="S3.SS3.p2.3.m3.1.2.2.cmml"><mi id="S3.SS3.p2.3.m3.1.2.2.2" xref="S3.SS3.p2.3.m3.1.2.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p2.3.m3.1.2.2.1" xref="S3.SS3.p2.3.m3.1.2.2.1.cmml">​</mo><mrow id="S3.SS3.p2.3.m3.1.2.2.3.2" xref="S3.SS3.p2.3.m3.1.2.2.cmml"><mo stretchy="false" id="S3.SS3.p2.3.m3.1.2.2.3.2.1" xref="S3.SS3.p2.3.m3.1.2.2.cmml">(</mo><mi id="S3.SS3.p2.3.m3.1.1" xref="S3.SS3.p2.3.m3.1.1.cmml">𝒙</mi><mo rspace="0.278em" stretchy="false" id="S3.SS3.p2.3.m3.1.2.2.3.2.2" xref="S3.SS3.p2.3.m3.1.2.2.cmml">)</mo></mrow></mrow><mo rspace="0.278em" id="S3.SS3.p2.3.m3.1.2.1" xref="S3.SS3.p2.3.m3.1.2.1.cmml">:</mo><mrow id="S3.SS3.p2.3.m3.1.2.3" xref="S3.SS3.p2.3.m3.1.2.3.cmml"><msup id="S3.SS3.p2.3.m3.1.2.3.2" xref="S3.SS3.p2.3.m3.1.2.3.2.cmml"><mi id="S3.SS3.p2.3.m3.1.2.3.2.2" xref="S3.SS3.p2.3.m3.1.2.3.2.2.cmml">ℝ</mi><mi id="S3.SS3.p2.3.m3.1.2.3.2.3" xref="S3.SS3.p2.3.m3.1.2.3.2.3.cmml">O</mi></msup><mo stretchy="false" id="S3.SS3.p2.3.m3.1.2.3.1" xref="S3.SS3.p2.3.m3.1.2.3.1.cmml">→</mo><msup id="S3.SS3.p2.3.m3.1.2.3.3" xref="S3.SS3.p2.3.m3.1.2.3.3.cmml"><mi id="S3.SS3.p2.3.m3.1.2.3.3.2" xref="S3.SS3.p2.3.m3.1.2.3.3.2.cmml">ℝ</mi><mi id="S3.SS3.p2.3.m3.1.2.3.3.3" xref="S3.SS3.p2.3.m3.1.2.3.3.3.cmml">M</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p2.3.m3.1b"><apply id="S3.SS3.p2.3.m3.1.2.cmml" xref="S3.SS3.p2.3.m3.1.2"><ci id="S3.SS3.p2.3.m3.1.2.1.cmml" xref="S3.SS3.p2.3.m3.1.2.1">:</ci><apply id="S3.SS3.p2.3.m3.1.2.2.cmml" xref="S3.SS3.p2.3.m3.1.2.2"><times id="S3.SS3.p2.3.m3.1.2.2.1.cmml" xref="S3.SS3.p2.3.m3.1.2.2.1"></times><ci id="S3.SS3.p2.3.m3.1.2.2.2.cmml" xref="S3.SS3.p2.3.m3.1.2.2.2">𝑓</ci><ci id="S3.SS3.p2.3.m3.1.1.cmml" xref="S3.SS3.p2.3.m3.1.1">𝒙</ci></apply><apply id="S3.SS3.p2.3.m3.1.2.3.cmml" xref="S3.SS3.p2.3.m3.1.2.3"><ci id="S3.SS3.p2.3.m3.1.2.3.1.cmml" xref="S3.SS3.p2.3.m3.1.2.3.1">→</ci><apply id="S3.SS3.p2.3.m3.1.2.3.2.cmml" xref="S3.SS3.p2.3.m3.1.2.3.2"><csymbol cd="ambiguous" id="S3.SS3.p2.3.m3.1.2.3.2.1.cmml" xref="S3.SS3.p2.3.m3.1.2.3.2">superscript</csymbol><ci id="S3.SS3.p2.3.m3.1.2.3.2.2.cmml" xref="S3.SS3.p2.3.m3.1.2.3.2.2">ℝ</ci><ci id="S3.SS3.p2.3.m3.1.2.3.2.3.cmml" xref="S3.SS3.p2.3.m3.1.2.3.2.3">𝑂</ci></apply><apply id="S3.SS3.p2.3.m3.1.2.3.3.cmml" xref="S3.SS3.p2.3.m3.1.2.3.3"><csymbol cd="ambiguous" id="S3.SS3.p2.3.m3.1.2.3.3.1.cmml" xref="S3.SS3.p2.3.m3.1.2.3.3">superscript</csymbol><ci id="S3.SS3.p2.3.m3.1.2.3.3.2.cmml" xref="S3.SS3.p2.3.m3.1.2.3.3.2">ℝ</ci><ci id="S3.SS3.p2.3.m3.1.2.3.3.3.cmml" xref="S3.SS3.p2.3.m3.1.2.3.3.3">𝑀</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p2.3.m3.1c">f({\bm{x}}):\mathbb{R}^{O}\rightarrow\mathbb{R}^{M}</annotation></semantics></math> maps a input <math id="S3.SS3.p2.4.m4.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.p2.4.m4.1a"><mi id="S3.SS3.p2.4.m4.1.1" xref="S3.SS3.p2.4.m4.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p2.4.m4.1b"><ci id="S3.SS3.p2.4.m4.1.1.cmml" xref="S3.SS3.p2.4.m4.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p2.4.m4.1c">{\bm{x}}</annotation></semantics></math> to a predicted solution <math id="S3.SS3.p2.5.m5.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S3.SS3.p2.5.m5.1a"><mi id="S3.SS3.p2.5.m5.1.1" xref="S3.SS3.p2.5.m5.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p2.5.m5.1b"><ci id="S3.SS3.p2.5.m5.1.1.cmml" xref="S3.SS3.p2.5.m5.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p2.5.m5.1c">{\bm{y}}</annotation></semantics></math>.</p>
</div>
<div id="S3.SS3.p3" class="ltx_para">
<p id="S3.SS3.p3.5" class="ltx_p"><span id="S3.SS3.p3.5.1" class="ltx_text ltx_font_bold">Energy Minimization.</span>  We next consider reasoning as a energy minimization problem, <math id="S3.SS3.p3.1.m1.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{\bm{y}}E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p3.1.m1.2a"><mrow id="S3.SS3.p3.1.m1.2.3" xref="S3.SS3.p3.1.m1.2.3.cmml"><mrow id="S3.SS3.p3.1.m1.2.3.2" xref="S3.SS3.p3.1.m1.2.3.2.cmml"><msub id="S3.SS3.p3.1.m1.2.3.2.1" xref="S3.SS3.p3.1.m1.2.3.2.1.cmml"><mrow id="S3.SS3.p3.1.m1.2.3.2.1.2" xref="S3.SS3.p3.1.m1.2.3.2.1.2.cmml"><mi id="S3.SS3.p3.1.m1.2.3.2.1.2.2" xref="S3.SS3.p3.1.m1.2.3.2.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS3.p3.1.m1.2.3.2.1.2.1" xref="S3.SS3.p3.1.m1.2.3.2.1.2.1.cmml">​</mo><mi id="S3.SS3.p3.1.m1.2.3.2.1.2.3" xref="S3.SS3.p3.1.m1.2.3.2.1.2.3.cmml">min</mi></mrow><mi id="S3.SS3.p3.1.m1.2.3.2.1.3" xref="S3.SS3.p3.1.m1.2.3.2.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS3.p3.1.m1.2.3.2a" xref="S3.SS3.p3.1.m1.2.3.2.cmml">⁡</mo><mi id="S3.SS3.p3.1.m1.2.3.2.2" xref="S3.SS3.p3.1.m1.2.3.2.2.cmml">E</mi></mrow><mo lspace="0em" rspace="0em" id="S3.SS3.p3.1.m1.2.3.1" xref="S3.SS3.p3.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS3.p3.1.m1.2.3.3.2" xref="S3.SS3.p3.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p3.1.m1.2.3.3.2.1" xref="S3.SS3.p3.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p3.1.m1.1.1" xref="S3.SS3.p3.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS3.p3.1.m1.2.3.3.2.2" xref="S3.SS3.p3.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p3.1.m1.2.2" xref="S3.SS3.p3.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p3.1.m1.2.3.3.2.3" xref="S3.SS3.p3.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p3.1.m1.2b"><apply id="S3.SS3.p3.1.m1.2.3.cmml" xref="S3.SS3.p3.1.m1.2.3"><times id="S3.SS3.p3.1.m1.2.3.1.cmml" xref="S3.SS3.p3.1.m1.2.3.1"></times><apply id="S3.SS3.p3.1.m1.2.3.2.cmml" xref="S3.SS3.p3.1.m1.2.3.2"><apply id="S3.SS3.p3.1.m1.2.3.2.1.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1"><csymbol cd="ambiguous" id="S3.SS3.p3.1.m1.2.3.2.1.1.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1">subscript</csymbol><apply id="S3.SS3.p3.1.m1.2.3.2.1.2.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1.2"><times id="S3.SS3.p3.1.m1.2.3.2.1.2.1.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1.2.1"></times><ci id="S3.SS3.p3.1.m1.2.3.2.1.2.2.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1.2.2">arg</ci><ci id="S3.SS3.p3.1.m1.2.3.2.1.2.3.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1.2.3">min</ci></apply><ci id="S3.SS3.p3.1.m1.2.3.2.1.3.cmml" xref="S3.SS3.p3.1.m1.2.3.2.1.3">𝒚</ci></apply><ci id="S3.SS3.p3.1.m1.2.3.2.2.cmml" xref="S3.SS3.p3.1.m1.2.3.2.2">𝐸</ci></apply><interval closure="open" id="S3.SS3.p3.1.m1.2.3.3.1.cmml" xref="S3.SS3.p3.1.m1.2.3.3.2"><ci id="S3.SS3.p3.1.m1.1.1.cmml" xref="S3.SS3.p3.1.m1.1.1">𝒙</ci><ci id="S3.SS3.p3.1.m1.2.2.cmml" xref="S3.SS3.p3.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p3.1.m1.2c">\operatorname*{arg\,min}_{\bm{y}}E({\bm{x}},{\bm{y}})</annotation></semantics></math>, where <math id="S3.SS3.p3.2.m2.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p3.2.m2.2a"><mrow id="S3.SS3.p3.2.m2.2.3" xref="S3.SS3.p3.2.m2.2.3.cmml"><mi id="S3.SS3.p3.2.m2.2.3.2" xref="S3.SS3.p3.2.m2.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p3.2.m2.2.3.1" xref="S3.SS3.p3.2.m2.2.3.1.cmml">​</mo><mrow id="S3.SS3.p3.2.m2.2.3.3.2" xref="S3.SS3.p3.2.m2.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p3.2.m2.2.3.3.2.1" xref="S3.SS3.p3.2.m2.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p3.2.m2.1.1" xref="S3.SS3.p3.2.m2.1.1.cmml">𝒙</mi><mo id="S3.SS3.p3.2.m2.2.3.3.2.2" xref="S3.SS3.p3.2.m2.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p3.2.m2.2.2" xref="S3.SS3.p3.2.m2.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p3.2.m2.2.3.3.2.3" xref="S3.SS3.p3.2.m2.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p3.2.m2.2b"><apply id="S3.SS3.p3.2.m2.2.3.cmml" xref="S3.SS3.p3.2.m2.2.3"><times id="S3.SS3.p3.2.m2.2.3.1.cmml" xref="S3.SS3.p3.2.m2.2.3.1"></times><ci id="S3.SS3.p3.2.m2.2.3.2.cmml" xref="S3.SS3.p3.2.m2.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p3.2.m2.2.3.3.1.cmml" xref="S3.SS3.p3.2.m2.2.3.3.2"><ci id="S3.SS3.p3.2.m2.1.1.cmml" xref="S3.SS3.p3.2.m2.1.1">𝒙</ci><ci id="S3.SS3.p3.2.m2.2.2.cmml" xref="S3.SS3.p3.2.m2.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p3.2.m2.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> is a function from <math id="S3.SS3.p3.3.m3.1" class="ltx_Math" alttext="\mathbb{R}^{O}\times\mathbb{R}^{M}\rightarrow\mathbb{R}" display="inline"><semantics id="S3.SS3.p3.3.m3.1a"><mrow id="S3.SS3.p3.3.m3.1.1" xref="S3.SS3.p3.3.m3.1.1.cmml"><mrow id="S3.SS3.p3.3.m3.1.1.2" xref="S3.SS3.p3.3.m3.1.1.2.cmml"><msup id="S3.SS3.p3.3.m3.1.1.2.2" xref="S3.SS3.p3.3.m3.1.1.2.2.cmml"><mi id="S3.SS3.p3.3.m3.1.1.2.2.2" xref="S3.SS3.p3.3.m3.1.1.2.2.2.cmml">ℝ</mi><mi id="S3.SS3.p3.3.m3.1.1.2.2.3" xref="S3.SS3.p3.3.m3.1.1.2.2.3.cmml">O</mi></msup><mo lspace="0.222em" rspace="0.222em" id="S3.SS3.p3.3.m3.1.1.2.1" xref="S3.SS3.p3.3.m3.1.1.2.1.cmml">×</mo><msup id="S3.SS3.p3.3.m3.1.1.2.3" xref="S3.SS3.p3.3.m3.1.1.2.3.cmml"><mi id="S3.SS3.p3.3.m3.1.1.2.3.2" xref="S3.SS3.p3.3.m3.1.1.2.3.2.cmml">ℝ</mi><mi id="S3.SS3.p3.3.m3.1.1.2.3.3" xref="S3.SS3.p3.3.m3.1.1.2.3.3.cmml">M</mi></msup></mrow><mo stretchy="false" id="S3.SS3.p3.3.m3.1.1.1" xref="S3.SS3.p3.3.m3.1.1.1.cmml">→</mo><mi id="S3.SS3.p3.3.m3.1.1.3" xref="S3.SS3.p3.3.m3.1.1.3.cmml">ℝ</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p3.3.m3.1b"><apply id="S3.SS3.p3.3.m3.1.1.cmml" xref="S3.SS3.p3.3.m3.1.1"><ci id="S3.SS3.p3.3.m3.1.1.1.cmml" xref="S3.SS3.p3.3.m3.1.1.1">→</ci><apply id="S3.SS3.p3.3.m3.1.1.2.cmml" xref="S3.SS3.p3.3.m3.1.1.2"><times id="S3.SS3.p3.3.m3.1.1.2.1.cmml" xref="S3.SS3.p3.3.m3.1.1.2.1"></times><apply id="S3.SS3.p3.3.m3.1.1.2.2.cmml" xref="S3.SS3.p3.3.m3.1.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p3.3.m3.1.1.2.2.1.cmml" xref="S3.SS3.p3.3.m3.1.1.2.2">superscript</csymbol><ci id="S3.SS3.p3.3.m3.1.1.2.2.2.cmml" xref="S3.SS3.p3.3.m3.1.1.2.2.2">ℝ</ci><ci id="S3.SS3.p3.3.m3.1.1.2.2.3.cmml" xref="S3.SS3.p3.3.m3.1.1.2.2.3">𝑂</ci></apply><apply id="S3.SS3.p3.3.m3.1.1.2.3.cmml" xref="S3.SS3.p3.3.m3.1.1.2.3"><csymbol cd="ambiguous" id="S3.SS3.p3.3.m3.1.1.2.3.1.cmml" xref="S3.SS3.p3.3.m3.1.1.2.3">superscript</csymbol><ci id="S3.SS3.p3.3.m3.1.1.2.3.2.cmml" xref="S3.SS3.p3.3.m3.1.1.2.3.2">ℝ</ci><ci id="S3.SS3.p3.3.m3.1.1.2.3.3.cmml" xref="S3.SS3.p3.3.m3.1.1.2.3.3">𝑀</ci></apply></apply><ci id="S3.SS3.p3.3.m3.1.1.3.cmml" xref="S3.SS3.p3.3.m3.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p3.3.m3.1c">\mathbb{R}^{O}\times\mathbb{R}^{M}\rightarrow\mathbb{R}</annotation></semantics></math>, mapping a input <math id="S3.SS3.p3.4.m4.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.p3.4.m4.1a"><mi id="S3.SS3.p3.4.m4.1.1" xref="S3.SS3.p3.4.m4.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p3.4.m4.1b"><ci id="S3.SS3.p3.4.m4.1.1.cmml" xref="S3.SS3.p3.4.m4.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p3.4.m4.1c">{\bm{x}}</annotation></semantics></math> and solution <math id="S3.SS3.p3.5.m5.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S3.SS3.p3.5.m5.1a"><mi id="S3.SS3.p3.5.m5.1.1" xref="S3.SS3.p3.5.m5.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p3.5.m5.1b"><ci id="S3.SS3.p3.5.m5.1.1.cmml" xref="S3.SS3.p3.5.m5.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p3.5.m5.1c">{\bm{y}}</annotation></semantics></math> into an energy. This corresponds to the computation represented by IREM.</p>
</div>
<div id="S3.SS3.p4" class="ltx_para">
<p id="S3.SS3.p4.4" class="ltx_p">We construct a class of algorithmic reasoning tasks that are easier to learn using an energy function <math id="S3.SS3.p4.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p4.1.m1.2a"><mrow id="S3.SS3.p4.1.m1.2.3" xref="S3.SS3.p4.1.m1.2.3.cmml"><mi id="S3.SS3.p4.1.m1.2.3.2" xref="S3.SS3.p4.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p4.1.m1.2.3.1" xref="S3.SS3.p4.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS3.p4.1.m1.2.3.3.2" xref="S3.SS3.p4.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p4.1.m1.2.3.3.2.1" xref="S3.SS3.p4.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p4.1.m1.1.1" xref="S3.SS3.p4.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS3.p4.1.m1.2.3.3.2.2" xref="S3.SS3.p4.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p4.1.m1.2.2" xref="S3.SS3.p4.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p4.1.m1.2.3.3.2.3" xref="S3.SS3.p4.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p4.1.m1.2b"><apply id="S3.SS3.p4.1.m1.2.3.cmml" xref="S3.SS3.p4.1.m1.2.3"><times id="S3.SS3.p4.1.m1.2.3.1.cmml" xref="S3.SS3.p4.1.m1.2.3.1"></times><ci id="S3.SS3.p4.1.m1.2.3.2.cmml" xref="S3.SS3.p4.1.m1.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p4.1.m1.2.3.3.1.cmml" xref="S3.SS3.p4.1.m1.2.3.3.2"><ci id="S3.SS3.p4.1.m1.1.1.cmml" xref="S3.SS3.p4.1.m1.1.1">𝒙</ci><ci id="S3.SS3.p4.1.m1.2.2.cmml" xref="S3.SS3.p4.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p4.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> as opposed to the feedforward function <math id="S3.SS3.p4.2.m2.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p4.2.m2.1a"><mrow id="S3.SS3.p4.2.m2.1.2" xref="S3.SS3.p4.2.m2.1.2.cmml"><mi id="S3.SS3.p4.2.m2.1.2.2" xref="S3.SS3.p4.2.m2.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p4.2.m2.1.2.1" xref="S3.SS3.p4.2.m2.1.2.1.cmml">​</mo><mrow id="S3.SS3.p4.2.m2.1.2.3.2" xref="S3.SS3.p4.2.m2.1.2.cmml"><mo stretchy="false" id="S3.SS3.p4.2.m2.1.2.3.2.1" xref="S3.SS3.p4.2.m2.1.2.cmml">(</mo><mi id="S3.SS3.p4.2.m2.1.1" xref="S3.SS3.p4.2.m2.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p4.2.m2.1.2.3.2.2" xref="S3.SS3.p4.2.m2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p4.2.m2.1b"><apply id="S3.SS3.p4.2.m2.1.2.cmml" xref="S3.SS3.p4.2.m2.1.2"><times id="S3.SS3.p4.2.m2.1.2.1.cmml" xref="S3.SS3.p4.2.m2.1.2.1"></times><ci id="S3.SS3.p4.2.m2.1.2.2.cmml" xref="S3.SS3.p4.2.m2.1.2.2">𝑓</ci><ci id="S3.SS3.p4.2.m2.1.1.cmml" xref="S3.SS3.p4.2.m2.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p4.2.m2.1c">f({\bm{x}})</annotation></semantics></math>. Our result is based on the intuition that learning the energy function <math id="S3.SS3.p4.3.m3.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p4.3.m3.2a"><mrow id="S3.SS3.p4.3.m3.2.3" xref="S3.SS3.p4.3.m3.2.3.cmml"><mi id="S3.SS3.p4.3.m3.2.3.2" xref="S3.SS3.p4.3.m3.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p4.3.m3.2.3.1" xref="S3.SS3.p4.3.m3.2.3.1.cmml">​</mo><mrow id="S3.SS3.p4.3.m3.2.3.3.2" xref="S3.SS3.p4.3.m3.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p4.3.m3.2.3.3.2.1" xref="S3.SS3.p4.3.m3.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p4.3.m3.1.1" xref="S3.SS3.p4.3.m3.1.1.cmml">𝒙</mi><mo id="S3.SS3.p4.3.m3.2.3.3.2.2" xref="S3.SS3.p4.3.m3.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p4.3.m3.2.2" xref="S3.SS3.p4.3.m3.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p4.3.m3.2.3.3.2.3" xref="S3.SS3.p4.3.m3.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p4.3.m3.2b"><apply id="S3.SS3.p4.3.m3.2.3.cmml" xref="S3.SS3.p4.3.m3.2.3"><times id="S3.SS3.p4.3.m3.2.3.1.cmml" xref="S3.SS3.p4.3.m3.2.3.1"></times><ci id="S3.SS3.p4.3.m3.2.3.2.cmml" xref="S3.SS3.p4.3.m3.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p4.3.m3.2.3.3.1.cmml" xref="S3.SS3.p4.3.m3.2.3.3.2"><ci id="S3.SS3.p4.3.m3.1.1.cmml" xref="S3.SS3.p4.3.m3.1.1">𝒙</ci><ci id="S3.SS3.p4.3.m3.2.2.cmml" xref="S3.SS3.p4.3.m3.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p4.3.m3.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> corresponds to learning a solution verifier, which assigns minimal energy to the correct solution, and high energy to all the other solutions. In contrast, learning a function <math id="S3.SS3.p4.4.m4.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p4.4.m4.1a"><mrow id="S3.SS3.p4.4.m4.1.2" xref="S3.SS3.p4.4.m4.1.2.cmml"><mi id="S3.SS3.p4.4.m4.1.2.2" xref="S3.SS3.p4.4.m4.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p4.4.m4.1.2.1" xref="S3.SS3.p4.4.m4.1.2.1.cmml">​</mo><mrow id="S3.SS3.p4.4.m4.1.2.3.2" xref="S3.SS3.p4.4.m4.1.2.cmml"><mo stretchy="false" id="S3.SS3.p4.4.m4.1.2.3.2.1" xref="S3.SS3.p4.4.m4.1.2.cmml">(</mo><mi id="S3.SS3.p4.4.m4.1.1" xref="S3.SS3.p4.4.m4.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p4.4.m4.1.2.3.2.2" xref="S3.SS3.p4.4.m4.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p4.4.m4.1b"><apply id="S3.SS3.p4.4.m4.1.2.cmml" xref="S3.SS3.p4.4.m4.1.2"><times id="S3.SS3.p4.4.m4.1.2.1.cmml" xref="S3.SS3.p4.4.m4.1.2.1"></times><ci id="S3.SS3.p4.4.m4.1.2.2.cmml" xref="S3.SS3.p4.4.m4.1.2.2">𝑓</ci><ci id="S3.SS3.p4.4.m4.1.1.cmml" xref="S3.SS3.p4.4.m4.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p4.4.m4.1c">f({\bm{x}})</annotation></semantics></math> corresponds to explicitly generating a solution. We rely on the well-known theorem in complexity theory that constructing a solution verifier is easier than a solution generator to show that learning energy minimization is easier than feedforward computation.</p>
</div>
<figure id="alg2" class="ltx_float ltx_float_algorithm ltx_framed ltx_framed_top">
<div id="alg2.2" class="ltx_listing ltx_listing">
<div id="alg1.l1" class="ltx_listingline">
<span id="alg1.l1.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg1.l1.2" class="ltx_text ltx_font_bold" style="font-size:90%;">Input:</span><span id="alg1.l1.3" class="ltx_text" style="font-size:90%;"> Data Dist </span><math id="alg1.l1.m1.1" class="ltx_Math" alttext="p_{D}({\bm{x}})" display="inline"><semantics id="alg1.l1.m1.1a"><mrow id="alg1.l1.m1.1.2" xref="alg1.l1.m1.1.2.cmml"><msub id="alg1.l1.m1.1.2.2" xref="alg1.l1.m1.1.2.2.cmml"><mi mathsize="90%" id="alg1.l1.m1.1.2.2.2" xref="alg1.l1.m1.1.2.2.2.cmml">p</mi><mi mathsize="90%" id="alg1.l1.m1.1.2.2.3" xref="alg1.l1.m1.1.2.2.3.cmml">D</mi></msub><mo lspace="0em" rspace="0em" id="alg1.l1.m1.1.2.1" xref="alg1.l1.m1.1.2.1.cmml">​</mo><mrow id="alg1.l1.m1.1.2.3.2" xref="alg1.l1.m1.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg1.l1.m1.1.2.3.2.1" xref="alg1.l1.m1.1.2.cmml">(</mo><mi mathsize="90%" id="alg1.l1.m1.1.1" xref="alg1.l1.m1.1.1.cmml">𝒙</mi><mo maxsize="90%" minsize="90%" id="alg1.l1.m1.1.2.3.2.2" xref="alg1.l1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l1.m1.1b"><apply id="alg1.l1.m1.1.2.cmml" xref="alg1.l1.m1.1.2"><times id="alg1.l1.m1.1.2.1.cmml" xref="alg1.l1.m1.1.2.1"></times><apply id="alg1.l1.m1.1.2.2.cmml" xref="alg1.l1.m1.1.2.2"><csymbol cd="ambiguous" id="alg1.l1.m1.1.2.2.1.cmml" xref="alg1.l1.m1.1.2.2">subscript</csymbol><ci id="alg1.l1.m1.1.2.2.2.cmml" xref="alg1.l1.m1.1.2.2.2">𝑝</ci><ci id="alg1.l1.m1.1.2.2.3.cmml" xref="alg1.l1.m1.1.2.2.3">𝐷</ci></apply><ci id="alg1.l1.m1.1.1.cmml" xref="alg1.l1.m1.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l1.m1.1c">p_{D}({\bm{x}})</annotation></semantics></math><span id="alg1.l1.4" class="ltx_text" style="font-size:90%;">, Step Size </span><math id="alg1.l1.m2.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="alg1.l1.m2.1a"><mi mathsize="90%" id="alg1.l1.m2.1.1" xref="alg1.l1.m2.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="alg1.l1.m2.1b"><ci id="alg1.l1.m2.1.1.cmml" xref="alg1.l1.m2.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l1.m2.1c">\lambda</annotation></semantics></math><span id="alg1.l1.5" class="ltx_text" style="font-size:90%;">, Number of Steps </span><math id="alg1.l1.m3.1" class="ltx_Math" alttext="K" display="inline"><semantics id="alg1.l1.m3.1a"><mi mathsize="90%" id="alg1.l1.m3.1.1" xref="alg1.l1.m3.1.1.cmml">K</mi><annotation-xml encoding="MathML-Content" id="alg1.l1.m3.1b"><ci id="alg1.l1.m3.1.1.cmml" xref="alg1.l1.m3.1.1">𝐾</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l1.m3.1c">K</annotation></semantics></math><span id="alg1.l1.6" class="ltx_text" style="font-size:90%;">, EBM </span><math id="alg1.l1.m4.1" class="ltx_Math" alttext="E_{\theta}(\cdot)" display="inline"><semantics id="alg1.l1.m4.1a"><mrow id="alg1.l1.m4.1.2" xref="alg1.l1.m4.1.2.cmml"><msub id="alg1.l1.m4.1.2.2" xref="alg1.l1.m4.1.2.2.cmml"><mi mathsize="90%" id="alg1.l1.m4.1.2.2.2" xref="alg1.l1.m4.1.2.2.2.cmml">E</mi><mi mathsize="90%" id="alg1.l1.m4.1.2.2.3" xref="alg1.l1.m4.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="alg1.l1.m4.1.2.1" xref="alg1.l1.m4.1.2.1.cmml">​</mo><mrow id="alg1.l1.m4.1.2.3.2" xref="alg1.l1.m4.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg1.l1.m4.1.2.3.2.1" xref="alg1.l1.m4.1.2.cmml">(</mo><mo lspace="0em" mathsize="90%" rspace="0em" id="alg1.l1.m4.1.1" xref="alg1.l1.m4.1.1.cmml">⋅</mo><mo maxsize="90%" minsize="90%" id="alg1.l1.m4.1.2.3.2.2" xref="alg1.l1.m4.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l1.m4.1b"><apply id="alg1.l1.m4.1.2.cmml" xref="alg1.l1.m4.1.2"><times id="alg1.l1.m4.1.2.1.cmml" xref="alg1.l1.m4.1.2.1"></times><apply id="alg1.l1.m4.1.2.2.cmml" xref="alg1.l1.m4.1.2.2"><csymbol cd="ambiguous" id="alg1.l1.m4.1.2.2.1.cmml" xref="alg1.l1.m4.1.2.2">subscript</csymbol><ci id="alg1.l1.m4.1.2.2.2.cmml" xref="alg1.l1.m4.1.2.2.2">𝐸</ci><ci id="alg1.l1.m4.1.2.2.3.cmml" xref="alg1.l1.m4.1.2.2.3">𝜃</ci></apply><ci id="alg1.l1.m4.1.1.cmml" xref="alg1.l1.m4.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l1.m4.1c">E_{\theta}(\cdot)</annotation></semantics></math><span id="alg1.l1.7" class="ltx_text" style="font-size:90%;">, Uniform Distribution </span><math id="alg1.l1.m5.2" class="ltx_Math" alttext="U(-1,1)" display="inline"><semantics id="alg1.l1.m5.2a"><mrow id="alg1.l1.m5.2.2" xref="alg1.l1.m5.2.2.cmml"><mi mathsize="90%" id="alg1.l1.m5.2.2.3" xref="alg1.l1.m5.2.2.3.cmml">U</mi><mo lspace="0em" rspace="0em" id="alg1.l1.m5.2.2.2" xref="alg1.l1.m5.2.2.2.cmml">​</mo><mrow id="alg1.l1.m5.2.2.1.1" xref="alg1.l1.m5.2.2.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg1.l1.m5.2.2.1.1.2" xref="alg1.l1.m5.2.2.1.2.cmml">(</mo><mrow id="alg1.l1.m5.2.2.1.1.1" xref="alg1.l1.m5.2.2.1.1.1.cmml"><mo mathsize="90%" id="alg1.l1.m5.2.2.1.1.1a" xref="alg1.l1.m5.2.2.1.1.1.cmml">−</mo><mn mathsize="90%" id="alg1.l1.m5.2.2.1.1.1.2" xref="alg1.l1.m5.2.2.1.1.1.2.cmml">1</mn></mrow><mo mathsize="90%" id="alg1.l1.m5.2.2.1.1.3" xref="alg1.l1.m5.2.2.1.2.cmml">,</mo><mn mathsize="90%" id="alg1.l1.m5.1.1" xref="alg1.l1.m5.1.1.cmml">1</mn><mo maxsize="90%" minsize="90%" id="alg1.l1.m5.2.2.1.1.4" xref="alg1.l1.m5.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l1.m5.2b"><apply id="alg1.l1.m5.2.2.cmml" xref="alg1.l1.m5.2.2"><times id="alg1.l1.m5.2.2.2.cmml" xref="alg1.l1.m5.2.2.2"></times><ci id="alg1.l1.m5.2.2.3.cmml" xref="alg1.l1.m5.2.2.3">𝑈</ci><interval closure="open" id="alg1.l1.m5.2.2.1.2.cmml" xref="alg1.l1.m5.2.2.1.1"><apply id="alg1.l1.m5.2.2.1.1.1.cmml" xref="alg1.l1.m5.2.2.1.1.1"><minus id="alg1.l1.m5.2.2.1.1.1.1.cmml" xref="alg1.l1.m5.2.2.1.1.1"></minus><cn type="integer" id="alg1.l1.m5.2.2.1.1.1.2.cmml" xref="alg1.l1.m5.2.2.1.1.1.2">1</cn></apply><cn type="integer" id="alg1.l1.m5.1.1.cmml" xref="alg1.l1.m5.1.1">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l1.m5.2c">U(-1,1)</annotation></semantics></math><span id="alg1.l1.8" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l2" class="ltx_listingline">
<span id="alg1.l2.3" class="ltx_text" style="font-size:90%;">  </span><em id="alg1.l2.2" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg1.l2.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg1.l2.1.m1.1a"><mo id="alg1.l2.1.m1.1.1" xref="alg1.l2.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg1.l2.1.m1.1b"><ci id="alg1.l2.1.m1.1.1.cmml" xref="alg1.l2.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l2.1.m1.1c">\triangleright</annotation></semantics></math> Sample input from <math id="alg1.l2.2.m2.1" class="ltx_Math" alttext="p_{d}" display="inline"><semantics id="alg1.l2.2.m2.1a"><msub id="alg1.l2.2.m2.1.1" xref="alg1.l2.2.m2.1.1.cmml"><mi id="alg1.l2.2.m2.1.1.2" xref="alg1.l2.2.m2.1.1.2.cmml">p</mi><mi id="alg1.l2.2.m2.1.1.3" xref="alg1.l2.2.m2.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="alg1.l2.2.m2.1b"><apply id="alg1.l2.2.m2.1.1.cmml" xref="alg1.l2.2.m2.1.1"><csymbol cd="ambiguous" id="alg1.l2.2.m2.1.1.1.cmml" xref="alg1.l2.2.m2.1.1">subscript</csymbol><ci id="alg1.l2.2.m2.1.1.2.cmml" xref="alg1.l2.2.m2.1.1.2">𝑝</ci><ci id="alg1.l2.2.m2.1.1.3.cmml" xref="alg1.l2.2.m2.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l2.2.m2.1c">p_{d}</annotation></semantics></math> and initialize candidate solution</em><span id="alg1.l2.4" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l3" class="ltx_listingline">
<span id="alg1.l3.1" class="ltx_text" style="font-size:90%;">  </span><math id="alg1.l3.m1.1" class="ltx_Math" alttext="{\bm{x}}_{i}\sim p_{D}" display="inline"><semantics id="alg1.l3.m1.1a"><mrow id="alg1.l3.m1.1.1" xref="alg1.l3.m1.1.1.cmml"><msub id="alg1.l3.m1.1.1.2" xref="alg1.l3.m1.1.1.2.cmml"><mi mathsize="90%" id="alg1.l3.m1.1.1.2.2" xref="alg1.l3.m1.1.1.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg1.l3.m1.1.1.2.3" xref="alg1.l3.m1.1.1.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg1.l3.m1.1.1.1" xref="alg1.l3.m1.1.1.1.cmml">∼</mo><msub id="alg1.l3.m1.1.1.3" xref="alg1.l3.m1.1.1.3.cmml"><mi mathsize="90%" id="alg1.l3.m1.1.1.3.2" xref="alg1.l3.m1.1.1.3.2.cmml">p</mi><mi mathsize="90%" id="alg1.l3.m1.1.1.3.3" xref="alg1.l3.m1.1.1.3.3.cmml">D</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="alg1.l3.m1.1b"><apply id="alg1.l3.m1.1.1.cmml" xref="alg1.l3.m1.1.1"><csymbol cd="latexml" id="alg1.l3.m1.1.1.1.cmml" xref="alg1.l3.m1.1.1.1">similar-to</csymbol><apply id="alg1.l3.m1.1.1.2.cmml" xref="alg1.l3.m1.1.1.2"><csymbol cd="ambiguous" id="alg1.l3.m1.1.1.2.1.cmml" xref="alg1.l3.m1.1.1.2">subscript</csymbol><ci id="alg1.l3.m1.1.1.2.2.cmml" xref="alg1.l3.m1.1.1.2.2">𝒙</ci><ci id="alg1.l3.m1.1.1.2.3.cmml" xref="alg1.l3.m1.1.1.2.3">𝑖</ci></apply><apply id="alg1.l3.m1.1.1.3.cmml" xref="alg1.l3.m1.1.1.3"><csymbol cd="ambiguous" id="alg1.l3.m1.1.1.3.1.cmml" xref="alg1.l3.m1.1.1.3">subscript</csymbol><ci id="alg1.l3.m1.1.1.3.2.cmml" xref="alg1.l3.m1.1.1.3.2">𝑝</ci><ci id="alg1.l3.m1.1.1.3.3.cmml" xref="alg1.l3.m1.1.1.3.3">𝐷</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l3.m1.1c">{\bm{x}}_{i}\sim p_{D}</annotation></semantics></math><span id="alg1.l3.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l4" class="ltx_listingline">
<span id="alg1.l4.1" class="ltx_text" style="font-size:90%;">  </span><math id="alg1.l4.m1.2" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}\sim\mathcal{U}(-1,1)" display="inline"><semantics id="alg1.l4.m1.2a"><mrow id="alg1.l4.m1.2.2" xref="alg1.l4.m1.2.2.cmml"><msub id="alg1.l4.m1.2.2.3" xref="alg1.l4.m1.2.2.3.cmml"><mover accent="true" id="alg1.l4.m1.2.2.3.2" xref="alg1.l4.m1.2.2.3.2.cmml"><mi mathsize="90%" id="alg1.l4.m1.2.2.3.2.2" xref="alg1.l4.m1.2.2.3.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg1.l4.m1.2.2.3.2.1" xref="alg1.l4.m1.2.2.3.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg1.l4.m1.2.2.3.3" xref="alg1.l4.m1.2.2.3.3.cmml">i</mi></msub><mo mathsize="90%" id="alg1.l4.m1.2.2.2" xref="alg1.l4.m1.2.2.2.cmml">∼</mo><mrow id="alg1.l4.m1.2.2.1" xref="alg1.l4.m1.2.2.1.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg1.l4.m1.2.2.1.3" xref="alg1.l4.m1.2.2.1.3.cmml">𝒰</mi><mo lspace="0em" rspace="0em" id="alg1.l4.m1.2.2.1.2" xref="alg1.l4.m1.2.2.1.2.cmml">​</mo><mrow id="alg1.l4.m1.2.2.1.1.1" xref="alg1.l4.m1.2.2.1.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg1.l4.m1.2.2.1.1.1.2" xref="alg1.l4.m1.2.2.1.1.2.cmml">(</mo><mrow id="alg1.l4.m1.2.2.1.1.1.1" xref="alg1.l4.m1.2.2.1.1.1.1.cmml"><mo mathsize="90%" id="alg1.l4.m1.2.2.1.1.1.1a" xref="alg1.l4.m1.2.2.1.1.1.1.cmml">−</mo><mn mathsize="90%" id="alg1.l4.m1.2.2.1.1.1.1.2" xref="alg1.l4.m1.2.2.1.1.1.1.2.cmml">1</mn></mrow><mo mathsize="90%" id="alg1.l4.m1.2.2.1.1.1.3" xref="alg1.l4.m1.2.2.1.1.2.cmml">,</mo><mn mathsize="90%" id="alg1.l4.m1.1.1" xref="alg1.l4.m1.1.1.cmml">1</mn><mo maxsize="90%" minsize="90%" id="alg1.l4.m1.2.2.1.1.1.4" xref="alg1.l4.m1.2.2.1.1.2.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l4.m1.2b"><apply id="alg1.l4.m1.2.2.cmml" xref="alg1.l4.m1.2.2"><csymbol cd="latexml" id="alg1.l4.m1.2.2.2.cmml" xref="alg1.l4.m1.2.2.2">similar-to</csymbol><apply id="alg1.l4.m1.2.2.3.cmml" xref="alg1.l4.m1.2.2.3"><csymbol cd="ambiguous" id="alg1.l4.m1.2.2.3.1.cmml" xref="alg1.l4.m1.2.2.3">subscript</csymbol><apply id="alg1.l4.m1.2.2.3.2.cmml" xref="alg1.l4.m1.2.2.3.2"><ci id="alg1.l4.m1.2.2.3.2.1.cmml" xref="alg1.l4.m1.2.2.3.2.1">~</ci><ci id="alg1.l4.m1.2.2.3.2.2.cmml" xref="alg1.l4.m1.2.2.3.2.2">𝒚</ci></apply><ci id="alg1.l4.m1.2.2.3.3.cmml" xref="alg1.l4.m1.2.2.3.3">𝑖</ci></apply><apply id="alg1.l4.m1.2.2.1.cmml" xref="alg1.l4.m1.2.2.1"><times id="alg1.l4.m1.2.2.1.2.cmml" xref="alg1.l4.m1.2.2.1.2"></times><ci id="alg1.l4.m1.2.2.1.3.cmml" xref="alg1.l4.m1.2.2.1.3">𝒰</ci><interval closure="open" id="alg1.l4.m1.2.2.1.1.2.cmml" xref="alg1.l4.m1.2.2.1.1.1"><apply id="alg1.l4.m1.2.2.1.1.1.1.cmml" xref="alg1.l4.m1.2.2.1.1.1.1"><minus id="alg1.l4.m1.2.2.1.1.1.1.1.cmml" xref="alg1.l4.m1.2.2.1.1.1.1"></minus><cn type="integer" id="alg1.l4.m1.2.2.1.1.1.1.2.cmml" xref="alg1.l4.m1.2.2.1.1.1.1.2">1</cn></apply><cn type="integer" id="alg1.l4.m1.1.1.cmml" xref="alg1.l4.m1.1.1">1</cn></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l4.m1.2c">\tilde{{\bm{y}}}_{i}\sim\mathcal{U}(-1,1)</annotation></semantics></math><span id="alg1.l4.2" class="ltx_text" style="font-size:90%;">

</span>
</div>
<div id="alg1.l5" class="ltx_listingline">
<span id="alg1.l5.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg1.l5.2" class="ltx_text ltx_font_bold" style="font-size:90%;">while</span><span id="alg1.l5.3" class="ltx_text" style="font-size:90%;"> Not at Energy Minima </span><span id="alg1.l5.4" class="ltx_text ltx_font_bold" style="font-size:90%;">do</span><span id="alg1.l5.5" class="ltx_text" style="font-size:90%;">


</span>
</div>
<div id="alg1.l6" class="ltx_listingline">
<span id="alg1.l6.3" class="ltx_text" style="font-size:90%;">    </span><em id="alg1.l6.2" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg1.l6.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg1.l6.1.m1.1a"><mo id="alg1.l6.1.m1.1.1" xref="alg1.l6.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg1.l6.1.m1.1b"><ci id="alg1.l6.1.m1.1.1.cmml" xref="alg1.l6.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l6.1.m1.1c">\triangleright</annotation></semantics></math> Optimize candidate solution <math id="alg1.l6.2.m2.1" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}" display="inline"><semantics id="alg1.l6.2.m2.1a"><msub id="alg1.l6.2.m2.1.1" xref="alg1.l6.2.m2.1.1.cmml"><mover accent="true" id="alg1.l6.2.m2.1.1.2" xref="alg1.l6.2.m2.1.1.2.cmml"><mi id="alg1.l6.2.m2.1.1.2.2" xref="alg1.l6.2.m2.1.1.2.2.cmml">𝐲</mi><mo id="alg1.l6.2.m2.1.1.2.1" xref="alg1.l6.2.m2.1.1.2.1.cmml">~</mo></mover><mi id="alg1.l6.2.m2.1.1.3" xref="alg1.l6.2.m2.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="alg1.l6.2.m2.1b"><apply id="alg1.l6.2.m2.1.1.cmml" xref="alg1.l6.2.m2.1.1"><csymbol cd="ambiguous" id="alg1.l6.2.m2.1.1.1.cmml" xref="alg1.l6.2.m2.1.1">subscript</csymbol><apply id="alg1.l6.2.m2.1.1.2.cmml" xref="alg1.l6.2.m2.1.1.2"><ci id="alg1.l6.2.m2.1.1.2.1.cmml" xref="alg1.l6.2.m2.1.1.2.1">~</ci><ci id="alg1.l6.2.m2.1.1.2.2.cmml" xref="alg1.l6.2.m2.1.1.2.2">𝐲</ci></apply><ci id="alg1.l6.2.m2.1.1.3.cmml" xref="alg1.l6.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l6.2.m2.1c">\tilde{{\bm{y}}}_{i}</annotation></semantics></math> with gradient descent:</em><span id="alg1.l6.4" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l7" class="ltx_listingline">
<span id="alg1.l7.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg1.l7.m1.2" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}\leftarrow\tilde{{\bm{y}}}_{i}-\lambda\nabla_{\bm{y}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i})" display="inline"><semantics id="alg1.l7.m1.2a"><mrow id="alg1.l7.m1.2.2" xref="alg1.l7.m1.2.2.cmml"><msub id="alg1.l7.m1.2.2.4" xref="alg1.l7.m1.2.2.4.cmml"><mover accent="true" id="alg1.l7.m1.2.2.4.2" xref="alg1.l7.m1.2.2.4.2.cmml"><mi mathsize="90%" id="alg1.l7.m1.2.2.4.2.2" xref="alg1.l7.m1.2.2.4.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg1.l7.m1.2.2.4.2.1" xref="alg1.l7.m1.2.2.4.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg1.l7.m1.2.2.4.3" xref="alg1.l7.m1.2.2.4.3.cmml">i</mi></msub><mo mathsize="90%" stretchy="false" id="alg1.l7.m1.2.2.3" xref="alg1.l7.m1.2.2.3.cmml">←</mo><mrow id="alg1.l7.m1.2.2.2" xref="alg1.l7.m1.2.2.2.cmml"><msub id="alg1.l7.m1.2.2.2.4" xref="alg1.l7.m1.2.2.2.4.cmml"><mover accent="true" id="alg1.l7.m1.2.2.2.4.2" xref="alg1.l7.m1.2.2.2.4.2.cmml"><mi mathsize="90%" id="alg1.l7.m1.2.2.2.4.2.2" xref="alg1.l7.m1.2.2.2.4.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg1.l7.m1.2.2.2.4.2.1" xref="alg1.l7.m1.2.2.2.4.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg1.l7.m1.2.2.2.4.3" xref="alg1.l7.m1.2.2.2.4.3.cmml">i</mi></msub><mo mathsize="90%" id="alg1.l7.m1.2.2.2.3" xref="alg1.l7.m1.2.2.2.3.cmml">−</mo><mrow id="alg1.l7.m1.2.2.2.2" xref="alg1.l7.m1.2.2.2.2.cmml"><mi mathsize="90%" id="alg1.l7.m1.2.2.2.2.4" xref="alg1.l7.m1.2.2.2.2.4.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="alg1.l7.m1.2.2.2.2.3" xref="alg1.l7.m1.2.2.2.2.3.cmml">​</mo><mrow id="alg1.l7.m1.2.2.2.2.5" xref="alg1.l7.m1.2.2.2.2.5.cmml"><msub id="alg1.l7.m1.2.2.2.2.5.1" xref="alg1.l7.m1.2.2.2.2.5.1.cmml"><mo mathsize="90%" rspace="0.167em" id="alg1.l7.m1.2.2.2.2.5.1.2" xref="alg1.l7.m1.2.2.2.2.5.1.2.cmml">∇</mo><mi mathsize="90%" id="alg1.l7.m1.2.2.2.2.5.1.3" xref="alg1.l7.m1.2.2.2.2.5.1.3.cmml">𝒚</mi></msub><msub id="alg1.l7.m1.2.2.2.2.5.2" xref="alg1.l7.m1.2.2.2.2.5.2.cmml"><mi mathsize="90%" id="alg1.l7.m1.2.2.2.2.5.2.2" xref="alg1.l7.m1.2.2.2.2.5.2.2.cmml">E</mi><mi mathsize="90%" id="alg1.l7.m1.2.2.2.2.5.2.3" xref="alg1.l7.m1.2.2.2.2.5.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="alg1.l7.m1.2.2.2.2.3a" xref="alg1.l7.m1.2.2.2.2.3.cmml">​</mo><mrow id="alg1.l7.m1.2.2.2.2.2.2" xref="alg1.l7.m1.2.2.2.2.2.3.cmml"><mo maxsize="90%" minsize="90%" id="alg1.l7.m1.2.2.2.2.2.2.3" xref="alg1.l7.m1.2.2.2.2.2.3.cmml">(</mo><msub id="alg1.l7.m1.1.1.1.1.1.1.1" xref="alg1.l7.m1.1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg1.l7.m1.1.1.1.1.1.1.1.2" xref="alg1.l7.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg1.l7.m1.1.1.1.1.1.1.1.3" xref="alg1.l7.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg1.l7.m1.2.2.2.2.2.2.4" xref="alg1.l7.m1.2.2.2.2.2.3.cmml">,</mo><msub id="alg1.l7.m1.2.2.2.2.2.2.2" xref="alg1.l7.m1.2.2.2.2.2.2.2.cmml"><mover accent="true" id="alg1.l7.m1.2.2.2.2.2.2.2.2" xref="alg1.l7.m1.2.2.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg1.l7.m1.2.2.2.2.2.2.2.2.2" xref="alg1.l7.m1.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg1.l7.m1.2.2.2.2.2.2.2.2.1" xref="alg1.l7.m1.2.2.2.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg1.l7.m1.2.2.2.2.2.2.2.3" xref="alg1.l7.m1.2.2.2.2.2.2.2.3.cmml">i</mi></msub><mo maxsize="90%" minsize="90%" id="alg1.l7.m1.2.2.2.2.2.2.5" xref="alg1.l7.m1.2.2.2.2.2.3.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg1.l7.m1.2b"><apply id="alg1.l7.m1.2.2.cmml" xref="alg1.l7.m1.2.2"><ci id="alg1.l7.m1.2.2.3.cmml" xref="alg1.l7.m1.2.2.3">←</ci><apply id="alg1.l7.m1.2.2.4.cmml" xref="alg1.l7.m1.2.2.4"><csymbol cd="ambiguous" id="alg1.l7.m1.2.2.4.1.cmml" xref="alg1.l7.m1.2.2.4">subscript</csymbol><apply id="alg1.l7.m1.2.2.4.2.cmml" xref="alg1.l7.m1.2.2.4.2"><ci id="alg1.l7.m1.2.2.4.2.1.cmml" xref="alg1.l7.m1.2.2.4.2.1">~</ci><ci id="alg1.l7.m1.2.2.4.2.2.cmml" xref="alg1.l7.m1.2.2.4.2.2">𝒚</ci></apply><ci id="alg1.l7.m1.2.2.4.3.cmml" xref="alg1.l7.m1.2.2.4.3">𝑖</ci></apply><apply id="alg1.l7.m1.2.2.2.cmml" xref="alg1.l7.m1.2.2.2"><minus id="alg1.l7.m1.2.2.2.3.cmml" xref="alg1.l7.m1.2.2.2.3"></minus><apply id="alg1.l7.m1.2.2.2.4.cmml" xref="alg1.l7.m1.2.2.2.4"><csymbol cd="ambiguous" id="alg1.l7.m1.2.2.2.4.1.cmml" xref="alg1.l7.m1.2.2.2.4">subscript</csymbol><apply id="alg1.l7.m1.2.2.2.4.2.cmml" xref="alg1.l7.m1.2.2.2.4.2"><ci id="alg1.l7.m1.2.2.2.4.2.1.cmml" xref="alg1.l7.m1.2.2.2.4.2.1">~</ci><ci id="alg1.l7.m1.2.2.2.4.2.2.cmml" xref="alg1.l7.m1.2.2.2.4.2.2">𝒚</ci></apply><ci id="alg1.l7.m1.2.2.2.4.3.cmml" xref="alg1.l7.m1.2.2.2.4.3">𝑖</ci></apply><apply id="alg1.l7.m1.2.2.2.2.cmml" xref="alg1.l7.m1.2.2.2.2"><times id="alg1.l7.m1.2.2.2.2.3.cmml" xref="alg1.l7.m1.2.2.2.2.3"></times><ci id="alg1.l7.m1.2.2.2.2.4.cmml" xref="alg1.l7.m1.2.2.2.2.4">𝜆</ci><apply id="alg1.l7.m1.2.2.2.2.5.cmml" xref="alg1.l7.m1.2.2.2.2.5"><apply id="alg1.l7.m1.2.2.2.2.5.1.cmml" xref="alg1.l7.m1.2.2.2.2.5.1"><csymbol cd="ambiguous" id="alg1.l7.m1.2.2.2.2.5.1.1.cmml" xref="alg1.l7.m1.2.2.2.2.5.1">subscript</csymbol><ci id="alg1.l7.m1.2.2.2.2.5.1.2.cmml" xref="alg1.l7.m1.2.2.2.2.5.1.2">∇</ci><ci id="alg1.l7.m1.2.2.2.2.5.1.3.cmml" xref="alg1.l7.m1.2.2.2.2.5.1.3">𝒚</ci></apply><apply id="alg1.l7.m1.2.2.2.2.5.2.cmml" xref="alg1.l7.m1.2.2.2.2.5.2"><csymbol cd="ambiguous" id="alg1.l7.m1.2.2.2.2.5.2.1.cmml" xref="alg1.l7.m1.2.2.2.2.5.2">subscript</csymbol><ci id="alg1.l7.m1.2.2.2.2.5.2.2.cmml" xref="alg1.l7.m1.2.2.2.2.5.2.2">𝐸</ci><ci id="alg1.l7.m1.2.2.2.2.5.2.3.cmml" xref="alg1.l7.m1.2.2.2.2.5.2.3">𝜃</ci></apply></apply><interval closure="open" id="alg1.l7.m1.2.2.2.2.2.3.cmml" xref="alg1.l7.m1.2.2.2.2.2.2"><apply id="alg1.l7.m1.1.1.1.1.1.1.1.cmml" xref="alg1.l7.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg1.l7.m1.1.1.1.1.1.1.1.1.cmml" xref="alg1.l7.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="alg1.l7.m1.1.1.1.1.1.1.1.2.cmml" xref="alg1.l7.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg1.l7.m1.1.1.1.1.1.1.1.3.cmml" xref="alg1.l7.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg1.l7.m1.2.2.2.2.2.2.2.cmml" xref="alg1.l7.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg1.l7.m1.2.2.2.2.2.2.2.1.cmml" xref="alg1.l7.m1.2.2.2.2.2.2.2">subscript</csymbol><apply id="alg1.l7.m1.2.2.2.2.2.2.2.2.cmml" xref="alg1.l7.m1.2.2.2.2.2.2.2.2"><ci id="alg1.l7.m1.2.2.2.2.2.2.2.2.1.cmml" xref="alg1.l7.m1.2.2.2.2.2.2.2.2.1">~</ci><ci id="alg1.l7.m1.2.2.2.2.2.2.2.2.2.cmml" xref="alg1.l7.m1.2.2.2.2.2.2.2.2.2">𝒚</ci></apply><ci id="alg1.l7.m1.2.2.2.2.2.2.2.3.cmml" xref="alg1.l7.m1.2.2.2.2.2.2.2.3">𝑖</ci></apply></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l7.m1.2c">\tilde{{\bm{y}}}_{i}\leftarrow\tilde{{\bm{y}}}_{i}-\lambda\nabla_{\bm{y}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i})</annotation></semantics></math><span id="alg1.l7.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l8" class="ltx_listingline">
<span id="alg1.l8.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg1.l8.2" class="ltx_text ltx_font_bold" style="font-size:90%;">end</span><span id="alg1.l8.3" class="ltx_text" style="font-size:90%;"> </span><span id="alg1.l8.4" class="ltx_text ltx_font_bold" style="font-size:90%;">while</span><span id="alg1.l8.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l9" class="ltx_listingline">
<span id="alg1.l9.2" class="ltx_text" style="font-size:90%;">  </span><em id="alg1.l9.1" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg1.l9.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg1.l9.1.m1.1a"><mo id="alg1.l9.1.m1.1.1" xref="alg1.l9.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg1.l9.1.m1.1b"><ci id="alg1.l9.1.m1.1.1.cmml" xref="alg1.l9.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg1.l9.1.m1.1c">\triangleright</annotation></semantics></math> Final predicted solution:</em><span id="alg1.l9.3" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg1.l10" class="ltx_listingline">
<span id="alg1.l10.1" class="ltx_text" style="font-size:90%;">  </span><math id="alg1.l10.m1.1" class="ltx_Math" alttext="{\bm{y}}=\tilde{{\bm{y}}}" display="inline"><semantics id="alg1.l10.m1.1a"><mrow id="alg1.l10.m1.1.1" xref="alg1.l10.m1.1.1.cmml"><mi mathsize="90%" id="alg1.l10.m1.1.1.2" xref="alg1.l10.m1.1.1.2.cmml">𝒚</mi><mo mathsize="90%" id="alg1.l10.m1.1.1.1" xref="alg1.l10.m1.1.1.1.cmml">=</mo><mover accent="true" id="alg1.l10.m1.1.1.3" xref="alg1.l10.m1.1.1.3.cmml"><mi mathsize="90%" id="alg1.l10.m1.1.1.3.2" xref="alg1.l10.m1.1.1.3.2.cmml">𝒚</mi><mo mathsize="90%" id="alg1.l10.m1.1.1.3.1" xref="alg1.l10.m1.1.1.3.1.cmml">~</mo></mover></mrow><annotation-xml encoding="MathML-Content" id="alg1.l10.m1.1b"><apply id="alg1.l10.m1.1.1.cmml" xref="alg1.l10.m1.1.1"><eq id="alg1.l10.m1.1.1.1.cmml" xref="alg1.l10.m1.1.1.1"></eq><ci id="alg1.l10.m1.1.1.2.cmml" xref="alg1.l10.m1.1.1.2">𝒚</ci><apply id="alg1.l10.m1.1.1.3.cmml" xref="alg1.l10.m1.1.1.3"><ci id="alg1.l10.m1.1.1.3.1.cmml" xref="alg1.l10.m1.1.1.3.1">~</ci><ci id="alg1.l10.m1.1.1.3.2.cmml" xref="alg1.l10.m1.1.1.3.2">𝒚</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg1.l10.m1.1c">{\bm{y}}=\tilde{{\bm{y}}}</annotation></semantics></math><span id="alg1.l10.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
</div>
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_float"><span id="alg2.5.1.1" class="ltx_text ltx_font_bold">Algorithm 2</span> </span> IREM prediction algorithm</figcaption>
</figure>
<figure id="S3.T1" class="ltx_table">
<div id="S3.T1.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:506.5pt;height:274.5pt;vertical-align:-0.9pt;"><span class="ltx_transformed_inner" style="transform:translate(-13.3pt,7.2pt) scale(0.95,0.95) ;">
<table id="S3.T1.1.1" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S3.T1.1.1.1.1" class="ltx_tr">
<th id="S3.T1.1.1.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.1.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Task</span></th>
<th id="S3.T1.1.1.1.1.2" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Method</span></th>
<th id="S3.T1.1.1.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Same Size</span></th>
<th id="S3.T1.1.1.1.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_r ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.4.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Larger Size</span></th>
<th id="S3.T1.1.1.1.1.5" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.5.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Task</span></th>
<th id="S3.T1.1.1.1.1.6" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.6.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Method</span></th>
<th id="S3.T1.1.1.1.1.7" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.7.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Same Diff.</span></th>
<th id="S3.T1.1.1.1.1.8" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.1.1.8.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Harder Diff.</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S3.T1.1.1.2.1" class="ltx_tr">
<th id="S3.T1.1.1.2.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.1.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Edge</span></th>
<th id="S3.T1.1.1.2.1.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.2.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S3.T1.1.1.2.1.3" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.3.1" class="ltx_text" style="font-size:90%;">0.3016</span></td>
<td id="S3.T1.1.1.2.1.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.4.1" class="ltx_text" style="font-size:90%;">0.3124</span></td>
<th id="S3.T1.1.1.2.1.5" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;" rowspan="5"><span id="S3.T1.1.1.2.1.5.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Add</span></th>
<th id="S3.T1.1.1.2.1.6" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.6.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S3.T1.1.1.2.1.7" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.7.1" class="ltx_text" style="font-size:90%;">0.0448</span></td>
<td id="S3.T1.1.1.2.1.8" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.2.1.8.1" class="ltx_text" style="font-size:90%;">0.7029</span></td>
</tr>
<tr id="S3.T1.1.1.3.2" class="ltx_tr">
<th id="S3.T1.1.1.3.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;" rowspan="4"><span id="S3.T1.1.1.3.2.1.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Copy</span></th>
<th id="S3.T1.1.1.3.2.2" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.3.2.2.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S3.T1.1.1.3.2.3" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.3.2.3.1" class="ltx_text" style="font-size:90%;">0.3015</span></td>
<td id="S3.T1.1.1.3.2.4" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.3.2.4.1" class="ltx_text" style="font-size:90%;">0.3113</span></td>
<th id="S3.T1.1.1.3.2.5" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.3.2.5.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S3.T1.1.1.3.2.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.3.2.6.1" class="ltx_text" style="font-size:90%;">0.3610</span></td>
<td id="S3.T1.1.1.3.2.7" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.3.2.7.1" class="ltx_text" style="font-size:90%;">2.6133</span></td>
</tr>
<tr id="S3.T1.1.1.4.3" class="ltx_tr">
<th id="S3.T1.1.1.4.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.4.3.1.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S3.T1.1.1.4.3.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.4.3.2.1" class="ltx_text" style="font-size:90%;">0.3053</span></td>
<td id="S3.T1.1.1.4.3.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.4.3.3.1" class="ltx_text" style="font-size:90%;">0.4409</span></td>
<th id="S3.T1.1.1.4.3.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.4.3.4.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S3.T1.1.1.4.3.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.4.3.5.1" class="ltx_text" style="font-size:90%;">0.0111</span></td>
<td id="S3.T1.1.1.4.3.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.4.3.6.1" class="ltx_text" style="font-size:90%;">0.3446</span></td>
</tr>
<tr id="S3.T1.1.1.5.4" class="ltx_tr">
<th id="S3.T1.1.1.5.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.5.4.1.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S3.T1.1.1.5.4.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.5.4.2.1" class="ltx_text" style="font-size:90%;">0.6163</span></td>
<td id="S3.T1.1.1.5.4.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.5.4.3.1" class="ltx_text" style="font-size:90%;">0.6498</span></td>
<th id="S3.T1.1.1.5.4.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.5.4.4.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S3.T1.1.1.5.4.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.5.4.5.1" class="ltx_text" style="font-size:90%;">0.0144</span></td>
<td id="S3.T1.1.1.5.4.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.5.4.6.1" class="ltx_text" style="font-size:90%;">0.1577</span></td>
</tr>
<tr id="S3.T1.1.1.6.5" class="ltx_tr">
<th id="S3.T1.1.1.6.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.6.5.1.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S3.T1.1.1.6.5.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.6.5.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0019</span></td>
<td id="S3.T1.1.1.6.5.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.6.5.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0019</span></td>
<th id="S3.T1.1.1.6.5.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.6.5.4.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S3.T1.1.1.6.5.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.6.5.5.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0003</span></td>
<td id="S3.T1.1.1.6.5.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.6.5.6.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0021</span></td>
</tr>
<tr id="S3.T1.1.1.7.6" class="ltx_tr">
<th id="S3.T1.1.1.7.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.1.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.7.6.1.1.1" class="ltx_text ltx_font_bold">Connected</span></span></th>
<th id="S3.T1.1.1.7.6.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.2.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S3.T1.1.1.7.6.3" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.3.1" class="ltx_text" style="font-size:90%;">0.1796</span></td>
<td id="S3.T1.1.1.7.6.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.4.1" class="ltx_text" style="font-size:90%;">0.3460</span></td>
<th id="S3.T1.1.1.7.6.5" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.5.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.7.6.5.1.1" class="ltx_text ltx_font_bold">Matrix</span></span></th>
<th id="S3.T1.1.1.7.6.6" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.6.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S3.T1.1.1.7.6.7" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.7.1" class="ltx_text" style="font-size:90%;">0.0203</span></td>
<td id="S3.T1.1.1.7.6.8" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.7.6.8.1" class="ltx_text" style="font-size:90%;">0.2720</span></td>
</tr>
<tr id="S3.T1.1.1.8.7" class="ltx_tr">
<th id="S3.T1.1.1.8.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;" rowspan="4"><span id="S3.T1.1.1.8.7.1.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.8.7.1.1.1" class="ltx_text ltx_font_bold">Components</span></span></th>
<th id="S3.T1.1.1.8.7.2" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.8.7.2.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S3.T1.1.1.8.7.3" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.8.7.3.1" class="ltx_text" style="font-size:90%;">0.1794</span></td>
<td id="S3.T1.1.1.8.7.4" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.8.7.4.1" class="ltx_text" style="font-size:90%;">0.2766</span></td>
<th id="S3.T1.1.1.8.7.5" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;" rowspan="4"><span id="S3.T1.1.1.8.7.5.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.8.7.5.1.1" class="ltx_text ltx_font_bold">Completion</span></span></th>
<th id="S3.T1.1.1.8.7.6" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.8.7.6.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S3.T1.1.1.8.7.7" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.8.7.7.1" class="ltx_text" style="font-size:90%;">0.0266</span></td>
<td id="S3.T1.1.1.8.7.8" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.8.7.8.1" class="ltx_text" style="font-size:90%;">0.3285</span></td>
</tr>
<tr id="S3.T1.1.1.9.8" class="ltx_tr">
<th id="S3.T1.1.1.9.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.9.8.1.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S3.T1.1.1.9.8.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.9.8.2.1" class="ltx_text" style="font-size:90%;">0.2338</span></td>
<td id="S3.T1.1.1.9.8.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.9.8.3.1" class="ltx_text" style="font-size:90%;">3.1381</span></td>
<th id="S3.T1.1.1.9.8.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.9.8.4.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S3.T1.1.1.9.8.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.9.8.5.1" class="ltx_text" style="font-size:90%;">0.0203</span></td>
<td id="S3.T1.1.1.9.8.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.9.8.6.1" class="ltx_text" style="font-size:90%;">0.2637</span></td>
</tr>
<tr id="S3.T1.1.1.10.9" class="ltx_tr">
<th id="S3.T1.1.1.10.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.10.9.1.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S3.T1.1.1.10.9.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.10.9.2.1" class="ltx_text" style="font-size:90%;">0.4908</span></td>
<td id="S3.T1.1.1.10.9.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.10.9.3.1" class="ltx_text" style="font-size:90%;">1.2064</span></td>
<th id="S3.T1.1.1.10.9.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.10.9.4.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S3.T1.1.1.10.9.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.10.9.5.1" class="ltx_text" style="font-size:90%;">0.0253</span></td>
<td id="S3.T1.1.1.10.9.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.10.9.6.1" class="ltx_text" style="font-size:90%;">0.2102</span></td>
</tr>
<tr id="S3.T1.1.1.11.10" class="ltx_tr">
<th id="S3.T1.1.1.11.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.11.10.1.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S3.T1.1.1.11.10.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.11.10.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.1424</span></td>
<td id="S3.T1.1.1.11.10.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.11.10.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.2171</span></td>
<th id="S3.T1.1.1.11.10.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.11.10.4.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S3.T1.1.1.11.10.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.11.10.5.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0183</span></td>
<td id="S3.T1.1.1.11.10.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.11.10.6.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.2074</span></td>
</tr>
<tr id="S3.T1.1.1.12.11" class="ltx_tr">
<th id="S3.T1.1.1.12.11.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.1.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.12.11.1.1.1" class="ltx_text ltx_font_bold">Shortest</span></span></th>
<th id="S3.T1.1.1.12.11.2" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.2.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S3.T1.1.1.12.11.3" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.3.1" class="ltx_text" style="font-size:90%;">0.1233</span></td>
<td id="S3.T1.1.1.12.11.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.4.1" class="ltx_text" style="font-size:90%;">1.4089</span></td>
<th id="S3.T1.1.1.12.11.5" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.5.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.12.11.5.1.1" class="ltx_text ltx_font_bold">Matrix</span></span></th>
<th id="S3.T1.1.1.12.11.6" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.6.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S3.T1.1.1.12.11.7" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.7.1" class="ltx_text" style="font-size:90%;">0.0112</span></td>
<td id="S3.T1.1.1.12.11.8" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.12.11.8.1" class="ltx_text" style="font-size:90%;">0.2150</span></td>
</tr>
<tr id="S3.T1.1.1.13.12" class="ltx_tr">
<th id="S3.T1.1.1.13.12.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;" rowspan="4"><span id="S3.T1.1.1.13.12.1.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.13.12.1.1.1" class="ltx_text ltx_font_bold">Path</span></span></th>
<th id="S3.T1.1.1.13.12.2" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.13.12.2.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S3.T1.1.1.13.12.3" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.13.12.3.1" class="ltx_text" style="font-size:90%;">0.1259</span></td>
<td id="S3.T1.1.1.13.12.4" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.13.12.4.1" class="ltx_text" style="font-size:90%;">0.1083</span></td>
<th id="S3.T1.1.1.13.12.5" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;" rowspan="4"><span id="S3.T1.1.1.13.12.5.1" class="ltx_text" style="font-size:90%;"><span id="S3.T1.1.1.13.12.5.1.1" class="ltx_text ltx_font_bold">Inverse</span></span></th>
<th id="S3.T1.1.1.13.12.6" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.13.12.6.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S3.T1.1.1.13.12.7" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.13.12.7.1" class="ltx_text" style="font-size:90%;">0.0109</span></td>
<td id="S3.T1.1.1.13.12.8" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.13.12.8.1" class="ltx_text" style="font-size:90%;">0.2123</span></td>
</tr>
<tr id="S3.T1.1.1.14.13" class="ltx_tr">
<th id="S3.T1.1.1.14.13.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.14.13.1.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S3.T1.1.1.14.13.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.14.13.2.1" class="ltx_text" style="font-size:90%;">0.1375</span></td>
<td id="S3.T1.1.1.14.13.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.14.13.3.1" class="ltx_text" style="font-size:90%;">0.1290</span></td>
<th id="S3.T1.1.1.14.13.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.14.13.4.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S3.T1.1.1.14.13.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.14.13.5.1" class="ltx_text" style="font-size:90%;">0.0124</span></td>
<td id="S3.T1.1.1.14.13.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.14.13.6.1" class="ltx_text" style="font-size:90%;">0.2209</span></td>
</tr>
<tr id="S3.T1.1.1.15.14" class="ltx_tr">
<th id="S3.T1.1.1.15.14.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.15.14.1.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S3.T1.1.1.15.14.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.15.14.2.1" class="ltx_text" style="font-size:90%;">0.4588</span></td>
<td id="S3.T1.1.1.15.14.3" class="ltx_td ltx_align_center ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.15.14.3.1" class="ltx_text" style="font-size:90%;">0.7688</span></td>
<th id="S3.T1.1.1.15.14.4" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.15.14.4.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S3.T1.1.1.15.14.5" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.15.14.5.1" class="ltx_text" style="font-size:90%;">0.0270</span></td>
<td id="S3.T1.1.1.15.14.6" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.15.14.6.1" class="ltx_text" style="font-size:90%;">0.5250</span></td>
</tr>
<tr id="S3.T1.1.1.16.15" class="ltx_tr">
<th id="S3.T1.1.1.16.15.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.16.15.1.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S3.T1.1.1.16.15.2" class="ltx_td ltx_align_center ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.16.15.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0274</span></td>
<td id="S3.T1.1.1.16.15.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.16.15.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0464</span></td>
<th id="S3.T1.1.1.16.15.4" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.16.15.4.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S3.T1.1.1.16.15.5" class="ltx_td ltx_align_center ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.16.15.5.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0108</span></td>
<td id="S3.T1.1.1.16.15.6" class="ltx_td ltx_align_center ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S3.T1.1.1.16.15.6.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.2083</span></td>
</tr>
</tbody>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 1: </span><span id="S3.T1.9.1" class="ltx_text ltx_font_bold">Algorithmic Reasoning with IREM–</span> IREM is general framework for iterative reasoning which can learn algorithmic computation on both graph (<span id="S3.T1.10.2" class="ltx_text ltx_font_bold">left</span>) and continuous (<span id="S3.T1.11.3" class="ltx_text ltx_font_bold">right</span>) inputs. IREM generalizes at test time to both larger (left) and harder (right) algorithmic problems through iterative computation. Error reported on each task using elementwise mean square error. Approaches on the left are trained on graphs with ten nodes and evaluated on larger graphs with fifteen nodes. IREM significantly outperforms comparisons.</figcaption>
</figure>
<div id="S3.SS3.p5" class="ltx_para">
<p id="S3.SS3.p5.19" class="ltx_p">To analyze <math id="S3.SS3.p5.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p5.1.m1.2a"><mrow id="S3.SS3.p5.1.m1.2.3" xref="S3.SS3.p5.1.m1.2.3.cmml"><mi id="S3.SS3.p5.1.m1.2.3.2" xref="S3.SS3.p5.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p5.1.m1.2.3.1" xref="S3.SS3.p5.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS3.p5.1.m1.2.3.3.2" xref="S3.SS3.p5.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p5.1.m1.2.3.3.2.1" xref="S3.SS3.p5.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p5.1.m1.1.1" xref="S3.SS3.p5.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS3.p5.1.m1.2.3.3.2.2" xref="S3.SS3.p5.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p5.1.m1.2.2" xref="S3.SS3.p5.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p5.1.m1.2.3.3.2.3" xref="S3.SS3.p5.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.1.m1.2b"><apply id="S3.SS3.p5.1.m1.2.3.cmml" xref="S3.SS3.p5.1.m1.2.3"><times id="S3.SS3.p5.1.m1.2.3.1.cmml" xref="S3.SS3.p5.1.m1.2.3.1"></times><ci id="S3.SS3.p5.1.m1.2.3.2.cmml" xref="S3.SS3.p5.1.m1.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p5.1.m1.2.3.3.1.cmml" xref="S3.SS3.p5.1.m1.2.3.3.2"><ci id="S3.SS3.p5.1.m1.1.1.cmml" xref="S3.SS3.p5.1.m1.1.1">𝒙</ci><ci id="S3.SS3.p5.1.m1.2.2.cmml" xref="S3.SS3.p5.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math>, we consider solving the 3-SAT <cite class="ltx_cite ltx_citemacro_citep">(Impagliazzo &amp; Paturi, <a href="#bib.bib24" title="" class="ltx_ref">1999</a>)</cite> problem. We construct a energy function <math id="S3.SS3.p5.2.m2.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p5.2.m2.2a"><mrow id="S3.SS3.p5.2.m2.2.3" xref="S3.SS3.p5.2.m2.2.3.cmml"><mi id="S3.SS3.p5.2.m2.2.3.2" xref="S3.SS3.p5.2.m2.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p5.2.m2.2.3.1" xref="S3.SS3.p5.2.m2.2.3.1.cmml">​</mo><mrow id="S3.SS3.p5.2.m2.2.3.3.2" xref="S3.SS3.p5.2.m2.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p5.2.m2.2.3.3.2.1" xref="S3.SS3.p5.2.m2.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p5.2.m2.1.1" xref="S3.SS3.p5.2.m2.1.1.cmml">𝒙</mi><mo id="S3.SS3.p5.2.m2.2.3.3.2.2" xref="S3.SS3.p5.2.m2.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p5.2.m2.2.2" xref="S3.SS3.p5.2.m2.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p5.2.m2.2.3.3.2.3" xref="S3.SS3.p5.2.m2.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.2.m2.2b"><apply id="S3.SS3.p5.2.m2.2.3.cmml" xref="S3.SS3.p5.2.m2.2.3"><times id="S3.SS3.p5.2.m2.2.3.1.cmml" xref="S3.SS3.p5.2.m2.2.3.1"></times><ci id="S3.SS3.p5.2.m2.2.3.2.cmml" xref="S3.SS3.p5.2.m2.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p5.2.m2.2.3.3.1.cmml" xref="S3.SS3.p5.2.m2.2.3.3.2"><ci id="S3.SS3.p5.2.m2.1.1.cmml" xref="S3.SS3.p5.2.m2.1.1">𝒙</ci><ci id="S3.SS3.p5.2.m2.2.2.cmml" xref="S3.SS3.p5.2.m2.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.2.m2.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> to verify the 3-SAT formula. Given a 3-SAT formula <math id="S3.SS3.p5.3.m3.1" class="ltx_Math" alttext="\phi" display="inline"><semantics id="S3.SS3.p5.3.m3.1a"><mi id="S3.SS3.p5.3.m3.1.1" xref="S3.SS3.p5.3.m3.1.1.cmml">ϕ</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.3.m3.1b"><ci id="S3.SS3.p5.3.m3.1.1.cmml" xref="S3.SS3.p5.3.m3.1.1">italic-ϕ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.3.m3.1c">\phi</annotation></semantics></math> with <math id="S3.SS3.p5.4.m4.1" class="ltx_Math" alttext="D" display="inline"><semantics id="S3.SS3.p5.4.m4.1a"><mi id="S3.SS3.p5.4.m4.1.1" xref="S3.SS3.p5.4.m4.1.1.cmml">D</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.4.m4.1b"><ci id="S3.SS3.p5.4.m4.1.1.cmml" xref="S3.SS3.p5.4.m4.1.1">𝐷</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.4.m4.1c">D</annotation></semantics></math> variables and <math id="S3.SS3.p5.5.m5.1" class="ltx_Math" alttext="K" display="inline"><semantics id="S3.SS3.p5.5.m5.1a"><mi id="S3.SS3.p5.5.m5.1.1" xref="S3.SS3.p5.5.m5.1.1.cmml">K</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.5.m5.1b"><ci id="S3.SS3.p5.5.m5.1.1.cmml" xref="S3.SS3.p5.5.m5.1.1">𝐾</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.5.m5.1c">K</annotation></semantics></math> clauses, we construct an energy function to represent <math id="S3.SS3.p5.6.m6.1" class="ltx_Math" alttext="\phi" display="inline"><semantics id="S3.SS3.p5.6.m6.1a"><mi id="S3.SS3.p5.6.m6.1.1" xref="S3.SS3.p5.6.m6.1.1.cmml">ϕ</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.6.m6.1b"><ci id="S3.SS3.p5.6.m6.1.1.cmml" xref="S3.SS3.p5.6.m6.1.1">italic-ϕ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.6.m6.1c">\phi</annotation></semantics></math> as <math id="S3.SS3.p5.7.m7.4" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}}):=\sum_{1\leq k\leq K}e_{k}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p5.7.m7.4a"><mrow id="S3.SS3.p5.7.m7.4.5" xref="S3.SS3.p5.7.m7.4.5.cmml"><mrow id="S3.SS3.p5.7.m7.4.5.2" xref="S3.SS3.p5.7.m7.4.5.2.cmml"><mi id="S3.SS3.p5.7.m7.4.5.2.2" xref="S3.SS3.p5.7.m7.4.5.2.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p5.7.m7.4.5.2.1" xref="S3.SS3.p5.7.m7.4.5.2.1.cmml">​</mo><mrow id="S3.SS3.p5.7.m7.4.5.2.3.2" xref="S3.SS3.p5.7.m7.4.5.2.3.1.cmml"><mo stretchy="false" id="S3.SS3.p5.7.m7.4.5.2.3.2.1" xref="S3.SS3.p5.7.m7.4.5.2.3.1.cmml">(</mo><mi id="S3.SS3.p5.7.m7.1.1" xref="S3.SS3.p5.7.m7.1.1.cmml">𝒙</mi><mo id="S3.SS3.p5.7.m7.4.5.2.3.2.2" xref="S3.SS3.p5.7.m7.4.5.2.3.1.cmml">,</mo><mi id="S3.SS3.p5.7.m7.2.2" xref="S3.SS3.p5.7.m7.2.2.cmml">𝒚</mi><mo rspace="0.278em" stretchy="false" id="S3.SS3.p5.7.m7.4.5.2.3.2.3" xref="S3.SS3.p5.7.m7.4.5.2.3.1.cmml">)</mo></mrow></mrow><mo rspace="0.111em" id="S3.SS3.p5.7.m7.4.5.1" xref="S3.SS3.p5.7.m7.4.5.1.cmml">:=</mo><mrow id="S3.SS3.p5.7.m7.4.5.3" xref="S3.SS3.p5.7.m7.4.5.3.cmml"><msub id="S3.SS3.p5.7.m7.4.5.3.1" xref="S3.SS3.p5.7.m7.4.5.3.1.cmml"><mo id="S3.SS3.p5.7.m7.4.5.3.1.2" xref="S3.SS3.p5.7.m7.4.5.3.1.2.cmml">∑</mo><mrow id="S3.SS3.p5.7.m7.4.5.3.1.3" xref="S3.SS3.p5.7.m7.4.5.3.1.3.cmml"><mn id="S3.SS3.p5.7.m7.4.5.3.1.3.2" xref="S3.SS3.p5.7.m7.4.5.3.1.3.2.cmml">1</mn><mo id="S3.SS3.p5.7.m7.4.5.3.1.3.3" xref="S3.SS3.p5.7.m7.4.5.3.1.3.3.cmml">≤</mo><mi id="S3.SS3.p5.7.m7.4.5.3.1.3.4" xref="S3.SS3.p5.7.m7.4.5.3.1.3.4.cmml">k</mi><mo id="S3.SS3.p5.7.m7.4.5.3.1.3.5" xref="S3.SS3.p5.7.m7.4.5.3.1.3.5.cmml">≤</mo><mi id="S3.SS3.p5.7.m7.4.5.3.1.3.6" xref="S3.SS3.p5.7.m7.4.5.3.1.3.6.cmml">K</mi></mrow></msub><mrow id="S3.SS3.p5.7.m7.4.5.3.2" xref="S3.SS3.p5.7.m7.4.5.3.2.cmml"><msub id="S3.SS3.p5.7.m7.4.5.3.2.2" xref="S3.SS3.p5.7.m7.4.5.3.2.2.cmml"><mi id="S3.SS3.p5.7.m7.4.5.3.2.2.2" xref="S3.SS3.p5.7.m7.4.5.3.2.2.2.cmml">e</mi><mi id="S3.SS3.p5.7.m7.4.5.3.2.2.3" xref="S3.SS3.p5.7.m7.4.5.3.2.2.3.cmml">k</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p5.7.m7.4.5.3.2.1" xref="S3.SS3.p5.7.m7.4.5.3.2.1.cmml">​</mo><mrow id="S3.SS3.p5.7.m7.4.5.3.2.3.2" xref="S3.SS3.p5.7.m7.4.5.3.2.3.1.cmml"><mo stretchy="false" id="S3.SS3.p5.7.m7.4.5.3.2.3.2.1" xref="S3.SS3.p5.7.m7.4.5.3.2.3.1.cmml">(</mo><mi id="S3.SS3.p5.7.m7.3.3" xref="S3.SS3.p5.7.m7.3.3.cmml">𝒙</mi><mo id="S3.SS3.p5.7.m7.4.5.3.2.3.2.2" xref="S3.SS3.p5.7.m7.4.5.3.2.3.1.cmml">,</mo><mi id="S3.SS3.p5.7.m7.4.4" xref="S3.SS3.p5.7.m7.4.4.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p5.7.m7.4.5.3.2.3.2.3" xref="S3.SS3.p5.7.m7.4.5.3.2.3.1.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.7.m7.4b"><apply id="S3.SS3.p5.7.m7.4.5.cmml" xref="S3.SS3.p5.7.m7.4.5"><csymbol cd="latexml" id="S3.SS3.p5.7.m7.4.5.1.cmml" xref="S3.SS3.p5.7.m7.4.5.1">assign</csymbol><apply id="S3.SS3.p5.7.m7.4.5.2.cmml" xref="S3.SS3.p5.7.m7.4.5.2"><times id="S3.SS3.p5.7.m7.4.5.2.1.cmml" xref="S3.SS3.p5.7.m7.4.5.2.1"></times><ci id="S3.SS3.p5.7.m7.4.5.2.2.cmml" xref="S3.SS3.p5.7.m7.4.5.2.2">𝐸</ci><interval closure="open" id="S3.SS3.p5.7.m7.4.5.2.3.1.cmml" xref="S3.SS3.p5.7.m7.4.5.2.3.2"><ci id="S3.SS3.p5.7.m7.1.1.cmml" xref="S3.SS3.p5.7.m7.1.1">𝒙</ci><ci id="S3.SS3.p5.7.m7.2.2.cmml" xref="S3.SS3.p5.7.m7.2.2">𝒚</ci></interval></apply><apply id="S3.SS3.p5.7.m7.4.5.3.cmml" xref="S3.SS3.p5.7.m7.4.5.3"><apply id="S3.SS3.p5.7.m7.4.5.3.1.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1"><csymbol cd="ambiguous" id="S3.SS3.p5.7.m7.4.5.3.1.1.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1">subscript</csymbol><sum id="S3.SS3.p5.7.m7.4.5.3.1.2.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.2"></sum><apply id="S3.SS3.p5.7.m7.4.5.3.1.3.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3"><and id="S3.SS3.p5.7.m7.4.5.3.1.3a.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3"></and><apply id="S3.SS3.p5.7.m7.4.5.3.1.3b.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3"><leq id="S3.SS3.p5.7.m7.4.5.3.1.3.3.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3.3"></leq><cn type="integer" id="S3.SS3.p5.7.m7.4.5.3.1.3.2.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3.2">1</cn><ci id="S3.SS3.p5.7.m7.4.5.3.1.3.4.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3.4">𝑘</ci></apply><apply id="S3.SS3.p5.7.m7.4.5.3.1.3c.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3"><leq id="S3.SS3.p5.7.m7.4.5.3.1.3.5.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3.5"></leq><share href="#S3.SS3.p5.7.m7.4.5.3.1.3.4.cmml" id="S3.SS3.p5.7.m7.4.5.3.1.3d.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3"></share><ci id="S3.SS3.p5.7.m7.4.5.3.1.3.6.cmml" xref="S3.SS3.p5.7.m7.4.5.3.1.3.6">𝐾</ci></apply></apply></apply><apply id="S3.SS3.p5.7.m7.4.5.3.2.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2"><times id="S3.SS3.p5.7.m7.4.5.3.2.1.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2.1"></times><apply id="S3.SS3.p5.7.m7.4.5.3.2.2.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2.2"><csymbol cd="ambiguous" id="S3.SS3.p5.7.m7.4.5.3.2.2.1.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2.2">subscript</csymbol><ci id="S3.SS3.p5.7.m7.4.5.3.2.2.2.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2.2.2">𝑒</ci><ci id="S3.SS3.p5.7.m7.4.5.3.2.2.3.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2.2.3">𝑘</ci></apply><interval closure="open" id="S3.SS3.p5.7.m7.4.5.3.2.3.1.cmml" xref="S3.SS3.p5.7.m7.4.5.3.2.3.2"><ci id="S3.SS3.p5.7.m7.3.3.cmml" xref="S3.SS3.p5.7.m7.3.3">𝒙</ci><ci id="S3.SS3.p5.7.m7.4.4.cmml" xref="S3.SS3.p5.7.m7.4.4">𝒚</ci></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.7.m7.4c">E({\bm{x}},{\bm{y}}):=\sum_{1\leq k\leq K}e_{k}({\bm{x}},{\bm{y}})</annotation></semantics></math>, where <math id="S3.SS3.p5.8.m8.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.p5.8.m8.1a"><mi id="S3.SS3.p5.8.m8.1.1" xref="S3.SS3.p5.8.m8.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.8.m8.1b"><ci id="S3.SS3.p5.8.m8.1.1.cmml" xref="S3.SS3.p5.8.m8.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.8.m8.1c">{\bm{x}}</annotation></semantics></math> encodes the clauses in a 3-SAT problem, <math id="S3.SS3.p5.9.m9.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S3.SS3.p5.9.m9.1a"><mi id="S3.SS3.p5.9.m9.1.1" xref="S3.SS3.p5.9.m9.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.9.m9.1b"><ci id="S3.SS3.p5.9.m9.1.1.cmml" xref="S3.SS3.p5.9.m9.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.9.m9.1c">{\bm{y}}</annotation></semantics></math> corresponds to boolean assignments to each variable, and <math id="S3.SS3.p5.10.m10.1" class="ltx_Math" alttext="e_{k}" display="inline"><semantics id="S3.SS3.p5.10.m10.1a"><msub id="S3.SS3.p5.10.m10.1.1" xref="S3.SS3.p5.10.m10.1.1.cmml"><mi id="S3.SS3.p5.10.m10.1.1.2" xref="S3.SS3.p5.10.m10.1.1.2.cmml">e</mi><mi id="S3.SS3.p5.10.m10.1.1.3" xref="S3.SS3.p5.10.m10.1.1.3.cmml">k</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.10.m10.1b"><apply id="S3.SS3.p5.10.m10.1.1.cmml" xref="S3.SS3.p5.10.m10.1.1"><csymbol cd="ambiguous" id="S3.SS3.p5.10.m10.1.1.1.cmml" xref="S3.SS3.p5.10.m10.1.1">subscript</csymbol><ci id="S3.SS3.p5.10.m10.1.1.2.cmml" xref="S3.SS3.p5.10.m10.1.1.2">𝑒</ci><ci id="S3.SS3.p5.10.m10.1.1.3.cmml" xref="S3.SS3.p5.10.m10.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.10.m10.1c">e_{k}</annotation></semantics></math> verifies whether the boolean assignments in <math id="S3.SS3.p5.11.m11.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S3.SS3.p5.11.m11.1a"><mi id="S3.SS3.p5.11.m11.1.1" xref="S3.SS3.p5.11.m11.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.11.m11.1b"><ci id="S3.SS3.p5.11.m11.1.1.cmml" xref="S3.SS3.p5.11.m11.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.11.m11.1c">{\bm{y}}</annotation></semantics></math> satisfies the <math id="S3.SS3.p5.12.m12.1" class="ltx_Math" alttext="k^{\text{th}}" display="inline"><semantics id="S3.SS3.p5.12.m12.1a"><msup id="S3.SS3.p5.12.m12.1.1" xref="S3.SS3.p5.12.m12.1.1.cmml"><mi id="S3.SS3.p5.12.m12.1.1.2" xref="S3.SS3.p5.12.m12.1.1.2.cmml">k</mi><mtext id="S3.SS3.p5.12.m12.1.1.3" xref="S3.SS3.p5.12.m12.1.1.3a.cmml">th</mtext></msup><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.12.m12.1b"><apply id="S3.SS3.p5.12.m12.1.1.cmml" xref="S3.SS3.p5.12.m12.1.1"><csymbol cd="ambiguous" id="S3.SS3.p5.12.m12.1.1.1.cmml" xref="S3.SS3.p5.12.m12.1.1">superscript</csymbol><ci id="S3.SS3.p5.12.m12.1.1.2.cmml" xref="S3.SS3.p5.12.m12.1.1.2">𝑘</ci><ci id="S3.SS3.p5.12.m12.1.1.3a.cmml" xref="S3.SS3.p5.12.m12.1.1.3"><mtext mathsize="70%" id="S3.SS3.p5.12.m12.1.1.3.cmml" xref="S3.SS3.p5.12.m12.1.1.3">th</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.12.m12.1c">k^{\text{th}}</annotation></semantics></math> clause. To encode a set of K clauses using <math id="S3.SS3.p5.13.m13.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.p5.13.m13.1a"><mi id="S3.SS3.p5.13.m13.1.1" xref="S3.SS3.p5.13.m13.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.13.m13.1b"><ci id="S3.SS3.p5.13.m13.1.1.cmml" xref="S3.SS3.p5.13.m13.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.13.m13.1c">{\bm{x}}</annotation></semantics></math>, we utilize an ordinal representation (e.g. <math id="S3.SS3.p5.14.m14.3" class="ltx_Math" alttext="{\bm{x}}_{1}=[1,2,3]" display="inline"><semantics id="S3.SS3.p5.14.m14.3a"><mrow id="S3.SS3.p5.14.m14.3.4" xref="S3.SS3.p5.14.m14.3.4.cmml"><msub id="S3.SS3.p5.14.m14.3.4.2" xref="S3.SS3.p5.14.m14.3.4.2.cmml"><mi id="S3.SS3.p5.14.m14.3.4.2.2" xref="S3.SS3.p5.14.m14.3.4.2.2.cmml">𝒙</mi><mn id="S3.SS3.p5.14.m14.3.4.2.3" xref="S3.SS3.p5.14.m14.3.4.2.3.cmml">1</mn></msub><mo id="S3.SS3.p5.14.m14.3.4.1" xref="S3.SS3.p5.14.m14.3.4.1.cmml">=</mo><mrow id="S3.SS3.p5.14.m14.3.4.3.2" xref="S3.SS3.p5.14.m14.3.4.3.1.cmml"><mo stretchy="false" id="S3.SS3.p5.14.m14.3.4.3.2.1" xref="S3.SS3.p5.14.m14.3.4.3.1.cmml">[</mo><mn id="S3.SS3.p5.14.m14.1.1" xref="S3.SS3.p5.14.m14.1.1.cmml">1</mn><mo id="S3.SS3.p5.14.m14.3.4.3.2.2" xref="S3.SS3.p5.14.m14.3.4.3.1.cmml">,</mo><mn id="S3.SS3.p5.14.m14.2.2" xref="S3.SS3.p5.14.m14.2.2.cmml">2</mn><mo id="S3.SS3.p5.14.m14.3.4.3.2.3" xref="S3.SS3.p5.14.m14.3.4.3.1.cmml">,</mo><mn id="S3.SS3.p5.14.m14.3.3" xref="S3.SS3.p5.14.m14.3.3.cmml">3</mn><mo stretchy="false" id="S3.SS3.p5.14.m14.3.4.3.2.4" xref="S3.SS3.p5.14.m14.3.4.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.14.m14.3b"><apply id="S3.SS3.p5.14.m14.3.4.cmml" xref="S3.SS3.p5.14.m14.3.4"><eq id="S3.SS3.p5.14.m14.3.4.1.cmml" xref="S3.SS3.p5.14.m14.3.4.1"></eq><apply id="S3.SS3.p5.14.m14.3.4.2.cmml" xref="S3.SS3.p5.14.m14.3.4.2"><csymbol cd="ambiguous" id="S3.SS3.p5.14.m14.3.4.2.1.cmml" xref="S3.SS3.p5.14.m14.3.4.2">subscript</csymbol><ci id="S3.SS3.p5.14.m14.3.4.2.2.cmml" xref="S3.SS3.p5.14.m14.3.4.2.2">𝒙</ci><cn type="integer" id="S3.SS3.p5.14.m14.3.4.2.3.cmml" xref="S3.SS3.p5.14.m14.3.4.2.3">1</cn></apply><list id="S3.SS3.p5.14.m14.3.4.3.1.cmml" xref="S3.SS3.p5.14.m14.3.4.3.2"><cn type="integer" id="S3.SS3.p5.14.m14.1.1.cmml" xref="S3.SS3.p5.14.m14.1.1">1</cn><cn type="integer" id="S3.SS3.p5.14.m14.2.2.cmml" xref="S3.SS3.p5.14.m14.2.2">2</cn><cn type="integer" id="S3.SS3.p5.14.m14.3.3.cmml" xref="S3.SS3.p5.14.m14.3.3">3</cn></list></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.14.m14.3c">{\bm{x}}_{1}=[1,2,3]</annotation></semantics></math> to represent a particular clause <math id="S3.SS3.p5.15.m15.1" class="ltx_Math" alttext="({\bm{y}}_{1}\land{\bm{y}}_{2}\land{\bm{y}}_{3})" display="inline"><semantics id="S3.SS3.p5.15.m15.1a"><mrow id="S3.SS3.p5.15.m15.1.1.1" xref="S3.SS3.p5.15.m15.1.1.1.1.cmml"><mo stretchy="false" id="S3.SS3.p5.15.m15.1.1.1.2" xref="S3.SS3.p5.15.m15.1.1.1.1.cmml">(</mo><mrow id="S3.SS3.p5.15.m15.1.1.1.1" xref="S3.SS3.p5.15.m15.1.1.1.1.cmml"><msub id="S3.SS3.p5.15.m15.1.1.1.1.2" xref="S3.SS3.p5.15.m15.1.1.1.1.2.cmml"><mi id="S3.SS3.p5.15.m15.1.1.1.1.2.2" xref="S3.SS3.p5.15.m15.1.1.1.1.2.2.cmml">𝒚</mi><mn id="S3.SS3.p5.15.m15.1.1.1.1.2.3" xref="S3.SS3.p5.15.m15.1.1.1.1.2.3.cmml">1</mn></msub><mo id="S3.SS3.p5.15.m15.1.1.1.1.1" xref="S3.SS3.p5.15.m15.1.1.1.1.1.cmml">∧</mo><msub id="S3.SS3.p5.15.m15.1.1.1.1.3" xref="S3.SS3.p5.15.m15.1.1.1.1.3.cmml"><mi id="S3.SS3.p5.15.m15.1.1.1.1.3.2" xref="S3.SS3.p5.15.m15.1.1.1.1.3.2.cmml">𝒚</mi><mn id="S3.SS3.p5.15.m15.1.1.1.1.3.3" xref="S3.SS3.p5.15.m15.1.1.1.1.3.3.cmml">2</mn></msub><mo id="S3.SS3.p5.15.m15.1.1.1.1.1a" xref="S3.SS3.p5.15.m15.1.1.1.1.1.cmml">∧</mo><msub id="S3.SS3.p5.15.m15.1.1.1.1.4" xref="S3.SS3.p5.15.m15.1.1.1.1.4.cmml"><mi id="S3.SS3.p5.15.m15.1.1.1.1.4.2" xref="S3.SS3.p5.15.m15.1.1.1.1.4.2.cmml">𝒚</mi><mn id="S3.SS3.p5.15.m15.1.1.1.1.4.3" xref="S3.SS3.p5.15.m15.1.1.1.1.4.3.cmml">3</mn></msub></mrow><mo stretchy="false" id="S3.SS3.p5.15.m15.1.1.1.3" xref="S3.SS3.p5.15.m15.1.1.1.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.15.m15.1b"><apply id="S3.SS3.p5.15.m15.1.1.1.1.cmml" xref="S3.SS3.p5.15.m15.1.1.1"><and id="S3.SS3.p5.15.m15.1.1.1.1.1.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.1"></and><apply id="S3.SS3.p5.15.m15.1.1.1.1.2.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.2"><csymbol cd="ambiguous" id="S3.SS3.p5.15.m15.1.1.1.1.2.1.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.2">subscript</csymbol><ci id="S3.SS3.p5.15.m15.1.1.1.1.2.2.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.2.2">𝒚</ci><cn type="integer" id="S3.SS3.p5.15.m15.1.1.1.1.2.3.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.2.3">1</cn></apply><apply id="S3.SS3.p5.15.m15.1.1.1.1.3.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.3"><csymbol cd="ambiguous" id="S3.SS3.p5.15.m15.1.1.1.1.3.1.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.3">subscript</csymbol><ci id="S3.SS3.p5.15.m15.1.1.1.1.3.2.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.3.2">𝒚</ci><cn type="integer" id="S3.SS3.p5.15.m15.1.1.1.1.3.3.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.3.3">2</cn></apply><apply id="S3.SS3.p5.15.m15.1.1.1.1.4.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.4"><csymbol cd="ambiguous" id="S3.SS3.p5.15.m15.1.1.1.1.4.1.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.4">subscript</csymbol><ci id="S3.SS3.p5.15.m15.1.1.1.1.4.2.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.4.2">𝒚</ci><cn type="integer" id="S3.SS3.p5.15.m15.1.1.1.1.4.3.cmml" xref="S3.SS3.p5.15.m15.1.1.1.1.4.3">3</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.15.m15.1c">({\bm{y}}_{1}\land{\bm{y}}_{2}\land{\bm{y}}_{3})</annotation></semantics></math>). The energy function <math id="S3.SS3.p5.16.m16.1" class="ltx_Math" alttext="e_{i}" display="inline"><semantics id="S3.SS3.p5.16.m16.1a"><msub id="S3.SS3.p5.16.m16.1.1" xref="S3.SS3.p5.16.m16.1.1.cmml"><mi id="S3.SS3.p5.16.m16.1.1.2" xref="S3.SS3.p5.16.m16.1.1.2.cmml">e</mi><mi id="S3.SS3.p5.16.m16.1.1.3" xref="S3.SS3.p5.16.m16.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.16.m16.1b"><apply id="S3.SS3.p5.16.m16.1.1.cmml" xref="S3.SS3.p5.16.m16.1.1"><csymbol cd="ambiguous" id="S3.SS3.p5.16.m16.1.1.1.cmml" xref="S3.SS3.p5.16.m16.1.1">subscript</csymbol><ci id="S3.SS3.p5.16.m16.1.1.2.cmml" xref="S3.SS3.p5.16.m16.1.1.2">𝑒</ci><ci id="S3.SS3.p5.16.m16.1.1.3.cmml" xref="S3.SS3.p5.16.m16.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.16.m16.1c">e_{i}</annotation></semantics></math> is then constructed by taking in clause <math id="S3.SS3.p5.17.m17.1" class="ltx_Math" alttext="{\bm{x}}_{i}" display="inline"><semantics id="S3.SS3.p5.17.m17.1a"><msub id="S3.SS3.p5.17.m17.1.1" xref="S3.SS3.p5.17.m17.1.1.cmml"><mi id="S3.SS3.p5.17.m17.1.1.2" xref="S3.SS3.p5.17.m17.1.1.2.cmml">𝒙</mi><mi id="S3.SS3.p5.17.m17.1.1.3" xref="S3.SS3.p5.17.m17.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.17.m17.1b"><apply id="S3.SS3.p5.17.m17.1.1.cmml" xref="S3.SS3.p5.17.m17.1.1"><csymbol cd="ambiguous" id="S3.SS3.p5.17.m17.1.1.1.cmml" xref="S3.SS3.p5.17.m17.1.1">subscript</csymbol><ci id="S3.SS3.p5.17.m17.1.1.2.cmml" xref="S3.SS3.p5.17.m17.1.1.2">𝒙</ci><ci id="S3.SS3.p5.17.m17.1.1.3.cmml" xref="S3.SS3.p5.17.m17.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.17.m17.1c">{\bm{x}}_{i}</annotation></semantics></math> and outputting 0 if the corresponding entries of <math id="S3.SS3.p5.18.m18.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S3.SS3.p5.18.m18.1a"><mi id="S3.SS3.p5.18.m18.1.1" xref="S3.SS3.p5.18.m18.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.18.m18.1b"><ci id="S3.SS3.p5.18.m18.1.1.cmml" xref="S3.SS3.p5.18.m18.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.18.m18.1c">{\bm{y}}</annotation></semantics></math> satisfy the encoded clause and 1 otherwise (polynomial time to compute). We assume the Exponential Time Hypothesis (ETH) <cite class="ltx_cite ltx_citemacro_citep">(Impagliazzo &amp; Paturi, <a href="#bib.bib24" title="" class="ltx_ref">1999</a>)</cite>, which states that checking the satisfiability of a <math id="S3.SS3.p5.19.m19.1" class="ltx_Math" alttext="3" display="inline"><semantics id="S3.SS3.p5.19.m19.1a"><mn id="S3.SS3.p5.19.m19.1.1" xref="S3.SS3.p5.19.m19.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="S3.SS3.p5.19.m19.1b"><cn type="integer" id="S3.SS3.p5.19.m19.1.1.cmml" xref="S3.SS3.p5.19.m19.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p5.19.m19.1c">3</annotation></semantics></math>-SAT formula takes time exponential in the sum of the number of variables and the number of clauses.</p>
</div>
<div id="Thmtheorem1" class="ltx_theorem ltx_theorem_theorem">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="Thmtheorem1.1.1.1" class="ltx_text ltx_font_bold">Remark 1</span></span><span id="Thmtheorem1.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="Thmtheorem1.p1" class="ltx_para">
<p id="Thmtheorem1.p1.4" class="ltx_p"><span id="Thmtheorem1.p1.4.4" class="ltx_text ltx_font_italic">There exists a 3-SAT problem which may be encoded in an energy function <math id="Thmtheorem1.p1.1.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="Thmtheorem1.p1.1.1.m1.2a"><mrow id="Thmtheorem1.p1.1.1.m1.2.3" xref="Thmtheorem1.p1.1.1.m1.2.3.cmml"><mi id="Thmtheorem1.p1.1.1.m1.2.3.2" xref="Thmtheorem1.p1.1.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="Thmtheorem1.p1.1.1.m1.2.3.1" xref="Thmtheorem1.p1.1.1.m1.2.3.1.cmml">​</mo><mrow id="Thmtheorem1.p1.1.1.m1.2.3.3.2" xref="Thmtheorem1.p1.1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="Thmtheorem1.p1.1.1.m1.2.3.3.2.1" xref="Thmtheorem1.p1.1.1.m1.2.3.3.1.cmml">(</mo><mi id="Thmtheorem1.p1.1.1.m1.1.1" xref="Thmtheorem1.p1.1.1.m1.1.1.cmml">𝐱</mi><mo id="Thmtheorem1.p1.1.1.m1.2.3.3.2.2" xref="Thmtheorem1.p1.1.1.m1.2.3.3.1.cmml">,</mo><mi id="Thmtheorem1.p1.1.1.m1.2.2" xref="Thmtheorem1.p1.1.1.m1.2.2.cmml">𝐲</mi><mo stretchy="false" id="Thmtheorem1.p1.1.1.m1.2.3.3.2.3" xref="Thmtheorem1.p1.1.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="Thmtheorem1.p1.1.1.m1.2b"><apply id="Thmtheorem1.p1.1.1.m1.2.3.cmml" xref="Thmtheorem1.p1.1.1.m1.2.3"><times id="Thmtheorem1.p1.1.1.m1.2.3.1.cmml" xref="Thmtheorem1.p1.1.1.m1.2.3.1"></times><ci id="Thmtheorem1.p1.1.1.m1.2.3.2.cmml" xref="Thmtheorem1.p1.1.1.m1.2.3.2">𝐸</ci><interval closure="open" id="Thmtheorem1.p1.1.1.m1.2.3.3.1.cmml" xref="Thmtheorem1.p1.1.1.m1.2.3.3.2"><ci id="Thmtheorem1.p1.1.1.m1.1.1.cmml" xref="Thmtheorem1.p1.1.1.m1.1.1">𝐱</ci><ci id="Thmtheorem1.p1.1.1.m1.2.2.cmml" xref="Thmtheorem1.p1.1.1.m1.2.2">𝐲</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="Thmtheorem1.p1.1.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> which can be evaluated at any input in time polynomial in the number of dimensions of <math id="Thmtheorem1.p1.2.2.m2.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="Thmtheorem1.p1.2.2.m2.1a"><mi id="Thmtheorem1.p1.2.2.m2.1.1" xref="Thmtheorem1.p1.2.2.m2.1.1.cmml">𝐱</mi><annotation-xml encoding="MathML-Content" id="Thmtheorem1.p1.2.2.m2.1b"><ci id="Thmtheorem1.p1.2.2.m2.1.1.cmml" xref="Thmtheorem1.p1.2.2.m2.1.1">𝐱</ci></annotation-xml><annotation encoding="application/x-tex" id="Thmtheorem1.p1.2.2.m2.1c">{\bm{x}}</annotation></semantics></math> but for which the computational complexity of encoding a feedforward solution <math id="Thmtheorem1.p1.3.3.m3.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="Thmtheorem1.p1.3.3.m3.1a"><mrow id="Thmtheorem1.p1.3.3.m3.1.2" xref="Thmtheorem1.p1.3.3.m3.1.2.cmml"><mi id="Thmtheorem1.p1.3.3.m3.1.2.2" xref="Thmtheorem1.p1.3.3.m3.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="Thmtheorem1.p1.3.3.m3.1.2.1" xref="Thmtheorem1.p1.3.3.m3.1.2.1.cmml">​</mo><mrow id="Thmtheorem1.p1.3.3.m3.1.2.3.2" xref="Thmtheorem1.p1.3.3.m3.1.2.cmml"><mo stretchy="false" id="Thmtheorem1.p1.3.3.m3.1.2.3.2.1" xref="Thmtheorem1.p1.3.3.m3.1.2.cmml">(</mo><mi id="Thmtheorem1.p1.3.3.m3.1.1" xref="Thmtheorem1.p1.3.3.m3.1.1.cmml">𝐱</mi><mo stretchy="false" id="Thmtheorem1.p1.3.3.m3.1.2.3.2.2" xref="Thmtheorem1.p1.3.3.m3.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="Thmtheorem1.p1.3.3.m3.1b"><apply id="Thmtheorem1.p1.3.3.m3.1.2.cmml" xref="Thmtheorem1.p1.3.3.m3.1.2"><times id="Thmtheorem1.p1.3.3.m3.1.2.1.cmml" xref="Thmtheorem1.p1.3.3.m3.1.2.1"></times><ci id="Thmtheorem1.p1.3.3.m3.1.2.2.cmml" xref="Thmtheorem1.p1.3.3.m3.1.2.2">𝑓</ci><ci id="Thmtheorem1.p1.3.3.m3.1.1.cmml" xref="Thmtheorem1.p1.3.3.m3.1.1">𝐱</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="Thmtheorem1.p1.3.3.m3.1c">f({\bm{x}})</annotation></semantics></math> which may be evaluated at any input is (worse-case) exponential in the number of dimensions of <math id="Thmtheorem1.p1.4.4.m4.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="Thmtheorem1.p1.4.4.m4.1a"><mi id="Thmtheorem1.p1.4.4.m4.1.1" xref="Thmtheorem1.p1.4.4.m4.1.1.cmml">𝐱</mi><annotation-xml encoding="MathML-Content" id="Thmtheorem1.p1.4.4.m4.1b"><ci id="Thmtheorem1.p1.4.4.m4.1.1.cmml" xref="Thmtheorem1.p1.4.4.m4.1.1">𝐱</ci></annotation-xml><annotation encoding="application/x-tex" id="Thmtheorem1.p1.4.4.m4.1c">{\bm{x}}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="S3.SS3.1" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S3.SS3.1.p1" class="ltx_para">
<p id="S3.SS3.1.p1.4" class="ltx_p">We encode the 3-SAT energy function <math id="S3.SS3.1.p1.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.1.p1.1.m1.2a"><mrow id="S3.SS3.1.p1.1.m1.2.3" xref="S3.SS3.1.p1.1.m1.2.3.cmml"><mi id="S3.SS3.1.p1.1.m1.2.3.2" xref="S3.SS3.1.p1.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.1.p1.1.m1.2.3.1" xref="S3.SS3.1.p1.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS3.1.p1.1.m1.2.3.3.2" xref="S3.SS3.1.p1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.1.p1.1.m1.2.3.3.2.1" xref="S3.SS3.1.p1.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS3.1.p1.1.m1.1.1" xref="S3.SS3.1.p1.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS3.1.p1.1.m1.2.3.3.2.2" xref="S3.SS3.1.p1.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS3.1.p1.1.m1.2.2" xref="S3.SS3.1.p1.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.1.p1.1.m1.2.3.3.2.3" xref="S3.SS3.1.p1.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.1.p1.1.m1.2b"><apply id="S3.SS3.1.p1.1.m1.2.3.cmml" xref="S3.SS3.1.p1.1.m1.2.3"><times id="S3.SS3.1.p1.1.m1.2.3.1.cmml" xref="S3.SS3.1.p1.1.m1.2.3.1"></times><ci id="S3.SS3.1.p1.1.m1.2.3.2.cmml" xref="S3.SS3.1.p1.1.m1.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.1.p1.1.m1.2.3.3.1.cmml" xref="S3.SS3.1.p1.1.m1.2.3.3.2"><ci id="S3.SS3.1.p1.1.m1.1.1.cmml" xref="S3.SS3.1.p1.1.m1.1.1">𝒙</ci><ci id="S3.SS3.1.p1.1.m1.2.2.cmml" xref="S3.SS3.1.p1.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.1.p1.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> as defined above, which is evaluated in time polynomial in the number of dimensions of <math id="S3.SS3.1.p1.2.m2.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.1.p1.2.m2.1a"><mi id="S3.SS3.1.p1.2.m2.1.1" xref="S3.SS3.1.p1.2.m2.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.1.p1.2.m2.1b"><ci id="S3.SS3.1.p1.2.m2.1.1.cmml" xref="S3.SS3.1.p1.2.m2.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.1.p1.2.m2.1c">{\bm{x}}</annotation></semantics></math>. In contrast, ETH directly implies that constructing <math id="S3.SS3.1.p1.3.m3.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.1.p1.3.m3.1a"><mrow id="S3.SS3.1.p1.3.m3.1.2" xref="S3.SS3.1.p1.3.m3.1.2.cmml"><mi id="S3.SS3.1.p1.3.m3.1.2.2" xref="S3.SS3.1.p1.3.m3.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.1.p1.3.m3.1.2.1" xref="S3.SS3.1.p1.3.m3.1.2.1.cmml">​</mo><mrow id="S3.SS3.1.p1.3.m3.1.2.3.2" xref="S3.SS3.1.p1.3.m3.1.2.cmml"><mo stretchy="false" id="S3.SS3.1.p1.3.m3.1.2.3.2.1" xref="S3.SS3.1.p1.3.m3.1.2.cmml">(</mo><mi id="S3.SS3.1.p1.3.m3.1.1" xref="S3.SS3.1.p1.3.m3.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.1.p1.3.m3.1.2.3.2.2" xref="S3.SS3.1.p1.3.m3.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.1.p1.3.m3.1b"><apply id="S3.SS3.1.p1.3.m3.1.2.cmml" xref="S3.SS3.1.p1.3.m3.1.2"><times id="S3.SS3.1.p1.3.m3.1.2.1.cmml" xref="S3.SS3.1.p1.3.m3.1.2.1"></times><ci id="S3.SS3.1.p1.3.m3.1.2.2.cmml" xref="S3.SS3.1.p1.3.m3.1.2.2">𝑓</ci><ci id="S3.SS3.1.p1.3.m3.1.1.cmml" xref="S3.SS3.1.p1.3.m3.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.1.p1.3.m3.1c">f({\bm{x}})</annotation></semantics></math>, which corresponds to solving the 3-SAT problem, is exponential in dimension of <math id="S3.SS3.1.p1.4.m4.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.1.p1.4.m4.1a"><mi id="S3.SS3.1.p1.4.m4.1.1" xref="S3.SS3.1.p1.4.m4.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.1.p1.4.m4.1b"><ci id="S3.SS3.1.p1.4.m4.1.1.cmml" xref="S3.SS3.1.p1.4.m4.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.1.p1.4.m4.1c">{\bm{x}}</annotation></semantics></math>.
∎</p>
</div>
</div>
<div id="S3.SS3.p6" class="ltx_para">
<p id="S3.SS3.p6.15" class="ltx_p">Our remark shows that it is computationally advantageous to represent an energy function <math id="S3.SS3.p6.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.1.m1.2a"><mrow id="S3.SS3.p6.1.m1.2.3" xref="S3.SS3.p6.1.m1.2.3.cmml"><mi id="S3.SS3.p6.1.m1.2.3.2" xref="S3.SS3.p6.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.1.m1.2.3.1" xref="S3.SS3.p6.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.1.m1.2.3.3.2" xref="S3.SS3.p6.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.1.m1.2.3.3.2.1" xref="S3.SS3.p6.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.1.m1.1.1" xref="S3.SS3.p6.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.1.m1.2.3.3.2.2" xref="S3.SS3.p6.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.1.m1.2.2" xref="S3.SS3.p6.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.1.m1.2.3.3.2.3" xref="S3.SS3.p6.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.1.m1.2b"><apply id="S3.SS3.p6.1.m1.2.3.cmml" xref="S3.SS3.p6.1.m1.2.3"><times id="S3.SS3.p6.1.m1.2.3.1.cmml" xref="S3.SS3.p6.1.m1.2.3.1"></times><ci id="S3.SS3.p6.1.m1.2.3.2.cmml" xref="S3.SS3.p6.1.m1.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p6.1.m1.2.3.3.1.cmml" xref="S3.SS3.p6.1.m1.2.3.3.2"><ci id="S3.SS3.p6.1.m1.1.1.cmml" xref="S3.SS3.p6.1.m1.1.1">𝒙</ci><ci id="S3.SS3.p6.1.m1.2.2.cmml" xref="S3.SS3.p6.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> as opposed to a feedforward decoder <math id="S3.SS3.p6.2.m2.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.2.m2.1a"><mrow id="S3.SS3.p6.2.m2.1.2" xref="S3.SS3.p6.2.m2.1.2.cmml"><mi id="S3.SS3.p6.2.m2.1.2.2" xref="S3.SS3.p6.2.m2.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.2.m2.1.2.1" xref="S3.SS3.p6.2.m2.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.2.m2.1.2.3.2" xref="S3.SS3.p6.2.m2.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.2.m2.1.2.3.2.1" xref="S3.SS3.p6.2.m2.1.2.cmml">(</mo><mi id="S3.SS3.p6.2.m2.1.1" xref="S3.SS3.p6.2.m2.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.2.m2.1.2.3.2.2" xref="S3.SS3.p6.2.m2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.2.m2.1b"><apply id="S3.SS3.p6.2.m2.1.2.cmml" xref="S3.SS3.p6.2.m2.1.2"><times id="S3.SS3.p6.2.m2.1.2.1.cmml" xref="S3.SS3.p6.2.m2.1.2.1"></times><ci id="S3.SS3.p6.2.m2.1.2.2.cmml" xref="S3.SS3.p6.2.m2.1.2.2">𝑓</ci><ci id="S3.SS3.p6.2.m2.1.1.cmml" xref="S3.SS3.p6.2.m2.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.2.m2.1c">f({\bm{x}})</annotation></semantics></math>. In particular, learning neural networks <math id="S3.SS3.p6.3.m3.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.3.m3.2a"><mrow id="S3.SS3.p6.3.m3.2.3" xref="S3.SS3.p6.3.m3.2.3.cmml"><msub id="S3.SS3.p6.3.m3.2.3.2" xref="S3.SS3.p6.3.m3.2.3.2.cmml"><mi id="S3.SS3.p6.3.m3.2.3.2.2" xref="S3.SS3.p6.3.m3.2.3.2.2.cmml">E</mi><mi id="S3.SS3.p6.3.m3.2.3.2.3" xref="S3.SS3.p6.3.m3.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p6.3.m3.2.3.1" xref="S3.SS3.p6.3.m3.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.3.m3.2.3.3.2" xref="S3.SS3.p6.3.m3.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.3.m3.2.3.3.2.1" xref="S3.SS3.p6.3.m3.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.3.m3.1.1" xref="S3.SS3.p6.3.m3.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.3.m3.2.3.3.2.2" xref="S3.SS3.p6.3.m3.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.3.m3.2.2" xref="S3.SS3.p6.3.m3.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.3.m3.2.3.3.2.3" xref="S3.SS3.p6.3.m3.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.3.m3.2b"><apply id="S3.SS3.p6.3.m3.2.3.cmml" xref="S3.SS3.p6.3.m3.2.3"><times id="S3.SS3.p6.3.m3.2.3.1.cmml" xref="S3.SS3.p6.3.m3.2.3.1"></times><apply id="S3.SS3.p6.3.m3.2.3.2.cmml" xref="S3.SS3.p6.3.m3.2.3.2"><csymbol cd="ambiguous" id="S3.SS3.p6.3.m3.2.3.2.1.cmml" xref="S3.SS3.p6.3.m3.2.3.2">subscript</csymbol><ci id="S3.SS3.p6.3.m3.2.3.2.2.cmml" xref="S3.SS3.p6.3.m3.2.3.2.2">𝐸</ci><ci id="S3.SS3.p6.3.m3.2.3.2.3.cmml" xref="S3.SS3.p6.3.m3.2.3.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS3.p6.3.m3.2.3.3.1.cmml" xref="S3.SS3.p6.3.m3.2.3.3.2"><ci id="S3.SS3.p6.3.m3.1.1.cmml" xref="S3.SS3.p6.3.m3.1.1">𝒙</ci><ci id="S3.SS3.p6.3.m3.2.2.cmml" xref="S3.SS3.p6.3.m3.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.3.m3.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math> and <math id="S3.SS3.p6.4.m4.1" class="ltx_Math" alttext="f_{\theta}({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.4.m4.1a"><mrow id="S3.SS3.p6.4.m4.1.2" xref="S3.SS3.p6.4.m4.1.2.cmml"><msub id="S3.SS3.p6.4.m4.1.2.2" xref="S3.SS3.p6.4.m4.1.2.2.cmml"><mi id="S3.SS3.p6.4.m4.1.2.2.2" xref="S3.SS3.p6.4.m4.1.2.2.2.cmml">f</mi><mi id="S3.SS3.p6.4.m4.1.2.2.3" xref="S3.SS3.p6.4.m4.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p6.4.m4.1.2.1" xref="S3.SS3.p6.4.m4.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.4.m4.1.2.3.2" xref="S3.SS3.p6.4.m4.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.4.m4.1.2.3.2.1" xref="S3.SS3.p6.4.m4.1.2.cmml">(</mo><mi id="S3.SS3.p6.4.m4.1.1" xref="S3.SS3.p6.4.m4.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.4.m4.1.2.3.2.2" xref="S3.SS3.p6.4.m4.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.4.m4.1b"><apply id="S3.SS3.p6.4.m4.1.2.cmml" xref="S3.SS3.p6.4.m4.1.2"><times id="S3.SS3.p6.4.m4.1.2.1.cmml" xref="S3.SS3.p6.4.m4.1.2.1"></times><apply id="S3.SS3.p6.4.m4.1.2.2.cmml" xref="S3.SS3.p6.4.m4.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p6.4.m4.1.2.2.1.cmml" xref="S3.SS3.p6.4.m4.1.2.2">subscript</csymbol><ci id="S3.SS3.p6.4.m4.1.2.2.2.cmml" xref="S3.SS3.p6.4.m4.1.2.2.2">𝑓</ci><ci id="S3.SS3.p6.4.m4.1.2.2.3.cmml" xref="S3.SS3.p6.4.m4.1.2.2.3">𝜃</ci></apply><ci id="S3.SS3.p6.4.m4.1.1.cmml" xref="S3.SS3.p6.4.m4.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.4.m4.1c">f_{\theta}({\bm{x}})</annotation></semantics></math> to approximate either <math id="S3.SS3.p6.5.m5.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.5.m5.2a"><mrow id="S3.SS3.p6.5.m5.2.3" xref="S3.SS3.p6.5.m5.2.3.cmml"><mi id="S3.SS3.p6.5.m5.2.3.2" xref="S3.SS3.p6.5.m5.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.5.m5.2.3.1" xref="S3.SS3.p6.5.m5.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.5.m5.2.3.3.2" xref="S3.SS3.p6.5.m5.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.5.m5.2.3.3.2.1" xref="S3.SS3.p6.5.m5.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.5.m5.1.1" xref="S3.SS3.p6.5.m5.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.5.m5.2.3.3.2.2" xref="S3.SS3.p6.5.m5.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.5.m5.2.2" xref="S3.SS3.p6.5.m5.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.5.m5.2.3.3.2.3" xref="S3.SS3.p6.5.m5.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.5.m5.2b"><apply id="S3.SS3.p6.5.m5.2.3.cmml" xref="S3.SS3.p6.5.m5.2.3"><times id="S3.SS3.p6.5.m5.2.3.1.cmml" xref="S3.SS3.p6.5.m5.2.3.1"></times><ci id="S3.SS3.p6.5.m5.2.3.2.cmml" xref="S3.SS3.p6.5.m5.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p6.5.m5.2.3.3.1.cmml" xref="S3.SS3.p6.5.m5.2.3.3.2"><ci id="S3.SS3.p6.5.m5.1.1.cmml" xref="S3.SS3.p6.5.m5.1.1">𝒙</ci><ci id="S3.SS3.p6.5.m5.2.2.cmml" xref="S3.SS3.p6.5.m5.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.5.m5.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> or <math id="S3.SS3.p6.6.m6.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.6.m6.1a"><mrow id="S3.SS3.p6.6.m6.1.2" xref="S3.SS3.p6.6.m6.1.2.cmml"><mi id="S3.SS3.p6.6.m6.1.2.2" xref="S3.SS3.p6.6.m6.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.6.m6.1.2.1" xref="S3.SS3.p6.6.m6.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.6.m6.1.2.3.2" xref="S3.SS3.p6.6.m6.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.6.m6.1.2.3.2.1" xref="S3.SS3.p6.6.m6.1.2.cmml">(</mo><mi id="S3.SS3.p6.6.m6.1.1" xref="S3.SS3.p6.6.m6.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.6.m6.1.2.3.2.2" xref="S3.SS3.p6.6.m6.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.6.m6.1b"><apply id="S3.SS3.p6.6.m6.1.2.cmml" xref="S3.SS3.p6.6.m6.1.2"><times id="S3.SS3.p6.6.m6.1.2.1.cmml" xref="S3.SS3.p6.6.m6.1.2.1"></times><ci id="S3.SS3.p6.6.m6.1.2.2.cmml" xref="S3.SS3.p6.6.m6.1.2.2">𝑓</ci><ci id="S3.SS3.p6.6.m6.1.1.cmml" xref="S3.SS3.p6.6.m6.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.6.m6.1c">f({\bm{x}})</annotation></semantics></math>, our remark implies that a larger network is necessary to represent the exponential computations of <math id="S3.SS3.p6.7.m7.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.7.m7.1a"><mrow id="S3.SS3.p6.7.m7.1.2" xref="S3.SS3.p6.7.m7.1.2.cmml"><mi id="S3.SS3.p6.7.m7.1.2.2" xref="S3.SS3.p6.7.m7.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.7.m7.1.2.1" xref="S3.SS3.p6.7.m7.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.7.m7.1.2.3.2" xref="S3.SS3.p6.7.m7.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.7.m7.1.2.3.2.1" xref="S3.SS3.p6.7.m7.1.2.cmml">(</mo><mi id="S3.SS3.p6.7.m7.1.1" xref="S3.SS3.p6.7.m7.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.7.m7.1.2.3.2.2" xref="S3.SS3.p6.7.m7.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.7.m7.1b"><apply id="S3.SS3.p6.7.m7.1.2.cmml" xref="S3.SS3.p6.7.m7.1.2"><times id="S3.SS3.p6.7.m7.1.2.1.cmml" xref="S3.SS3.p6.7.m7.1.2.1"></times><ci id="S3.SS3.p6.7.m7.1.2.2.cmml" xref="S3.SS3.p6.7.m7.1.2.2">𝑓</ci><ci id="S3.SS3.p6.7.m7.1.1.cmml" xref="S3.SS3.p6.7.m7.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.7.m7.1c">f({\bm{x}})</annotation></semantics></math>. Such a network <math id="S3.SS3.p6.8.m8.1" class="ltx_Math" alttext="f_{\theta}({\bm{s}})" display="inline"><semantics id="S3.SS3.p6.8.m8.1a"><mrow id="S3.SS3.p6.8.m8.1.2" xref="S3.SS3.p6.8.m8.1.2.cmml"><msub id="S3.SS3.p6.8.m8.1.2.2" xref="S3.SS3.p6.8.m8.1.2.2.cmml"><mi id="S3.SS3.p6.8.m8.1.2.2.2" xref="S3.SS3.p6.8.m8.1.2.2.2.cmml">f</mi><mi id="S3.SS3.p6.8.m8.1.2.2.3" xref="S3.SS3.p6.8.m8.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p6.8.m8.1.2.1" xref="S3.SS3.p6.8.m8.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.8.m8.1.2.3.2" xref="S3.SS3.p6.8.m8.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.8.m8.1.2.3.2.1" xref="S3.SS3.p6.8.m8.1.2.cmml">(</mo><mi id="S3.SS3.p6.8.m8.1.1" xref="S3.SS3.p6.8.m8.1.1.cmml">𝒔</mi><mo stretchy="false" id="S3.SS3.p6.8.m8.1.2.3.2.2" xref="S3.SS3.p6.8.m8.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.8.m8.1b"><apply id="S3.SS3.p6.8.m8.1.2.cmml" xref="S3.SS3.p6.8.m8.1.2"><times id="S3.SS3.p6.8.m8.1.2.1.cmml" xref="S3.SS3.p6.8.m8.1.2.1"></times><apply id="S3.SS3.p6.8.m8.1.2.2.cmml" xref="S3.SS3.p6.8.m8.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p6.8.m8.1.2.2.1.cmml" xref="S3.SS3.p6.8.m8.1.2.2">subscript</csymbol><ci id="S3.SS3.p6.8.m8.1.2.2.2.cmml" xref="S3.SS3.p6.8.m8.1.2.2.2">𝑓</ci><ci id="S3.SS3.p6.8.m8.1.2.2.3.cmml" xref="S3.SS3.p6.8.m8.1.2.2.3">𝜃</ci></apply><ci id="S3.SS3.p6.8.m8.1.1.cmml" xref="S3.SS3.p6.8.m8.1.1">𝒔</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.8.m8.1c">f_{\theta}({\bm{s}})</annotation></semantics></math> would require either exponentially deeper or wider layers, if there is no iterative computation. With iterative computation, we may parameterize <math id="S3.SS3.p6.9.m9.1" class="ltx_Math" alttext="f_{\theta}({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.9.m9.1a"><mrow id="S3.SS3.p6.9.m9.1.2" xref="S3.SS3.p6.9.m9.1.2.cmml"><msub id="S3.SS3.p6.9.m9.1.2.2" xref="S3.SS3.p6.9.m9.1.2.2.cmml"><mi id="S3.SS3.p6.9.m9.1.2.2.2" xref="S3.SS3.p6.9.m9.1.2.2.2.cmml">f</mi><mi id="S3.SS3.p6.9.m9.1.2.2.3" xref="S3.SS3.p6.9.m9.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p6.9.m9.1.2.1" xref="S3.SS3.p6.9.m9.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.9.m9.1.2.3.2" xref="S3.SS3.p6.9.m9.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.9.m9.1.2.3.2.1" xref="S3.SS3.p6.9.m9.1.2.cmml">(</mo><mi id="S3.SS3.p6.9.m9.1.1" xref="S3.SS3.p6.9.m9.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.9.m9.1.2.3.2.2" xref="S3.SS3.p6.9.m9.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.9.m9.1b"><apply id="S3.SS3.p6.9.m9.1.2.cmml" xref="S3.SS3.p6.9.m9.1.2"><times id="S3.SS3.p6.9.m9.1.2.1.cmml" xref="S3.SS3.p6.9.m9.1.2.1"></times><apply id="S3.SS3.p6.9.m9.1.2.2.cmml" xref="S3.SS3.p6.9.m9.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p6.9.m9.1.2.2.1.cmml" xref="S3.SS3.p6.9.m9.1.2.2">subscript</csymbol><ci id="S3.SS3.p6.9.m9.1.2.2.2.cmml" xref="S3.SS3.p6.9.m9.1.2.2.2">𝑓</ci><ci id="S3.SS3.p6.9.m9.1.2.2.3.cmml" xref="S3.SS3.p6.9.m9.1.2.2.3">𝜃</ci></apply><ci id="S3.SS3.p6.9.m9.1.1.cmml" xref="S3.SS3.p6.9.m9.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.9.m9.1c">f_{\theta}({\bm{x}})</annotation></semantics></math> and <math id="S3.SS3.p6.10.m10.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.10.m10.2a"><mrow id="S3.SS3.p6.10.m10.2.3" xref="S3.SS3.p6.10.m10.2.3.cmml"><msub id="S3.SS3.p6.10.m10.2.3.2" xref="S3.SS3.p6.10.m10.2.3.2.cmml"><mi id="S3.SS3.p6.10.m10.2.3.2.2" xref="S3.SS3.p6.10.m10.2.3.2.2.cmml">E</mi><mi id="S3.SS3.p6.10.m10.2.3.2.3" xref="S3.SS3.p6.10.m10.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p6.10.m10.2.3.1" xref="S3.SS3.p6.10.m10.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.10.m10.2.3.3.2" xref="S3.SS3.p6.10.m10.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.10.m10.2.3.3.2.1" xref="S3.SS3.p6.10.m10.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.10.m10.1.1" xref="S3.SS3.p6.10.m10.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.10.m10.2.3.3.2.2" xref="S3.SS3.p6.10.m10.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.10.m10.2.2" xref="S3.SS3.p6.10.m10.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.10.m10.2.3.3.2.3" xref="S3.SS3.p6.10.m10.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.10.m10.2b"><apply id="S3.SS3.p6.10.m10.2.3.cmml" xref="S3.SS3.p6.10.m10.2.3"><times id="S3.SS3.p6.10.m10.2.3.1.cmml" xref="S3.SS3.p6.10.m10.2.3.1"></times><apply id="S3.SS3.p6.10.m10.2.3.2.cmml" xref="S3.SS3.p6.10.m10.2.3.2"><csymbol cd="ambiguous" id="S3.SS3.p6.10.m10.2.3.2.1.cmml" xref="S3.SS3.p6.10.m10.2.3.2">subscript</csymbol><ci id="S3.SS3.p6.10.m10.2.3.2.2.cmml" xref="S3.SS3.p6.10.m10.2.3.2.2">𝐸</ci><ci id="S3.SS3.p6.10.m10.2.3.2.3.cmml" xref="S3.SS3.p6.10.m10.2.3.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS3.p6.10.m10.2.3.3.1.cmml" xref="S3.SS3.p6.10.m10.2.3.3.2"><ci id="S3.SS3.p6.10.m10.1.1.cmml" xref="S3.SS3.p6.10.m10.1.1">𝒙</ci><ci id="S3.SS3.p6.10.m10.2.2.cmml" xref="S3.SS3.p6.10.m10.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.10.m10.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math> with a similar number of parameters, but then it would be necessary for <math id="S3.SS3.p6.11.m11.1" class="ltx_Math" alttext="f_{\theta}({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.11.m11.1a"><mrow id="S3.SS3.p6.11.m11.1.2" xref="S3.SS3.p6.11.m11.1.2.cmml"><msub id="S3.SS3.p6.11.m11.1.2.2" xref="S3.SS3.p6.11.m11.1.2.2.cmml"><mi id="S3.SS3.p6.11.m11.1.2.2.2" xref="S3.SS3.p6.11.m11.1.2.2.2.cmml">f</mi><mi id="S3.SS3.p6.11.m11.1.2.2.3" xref="S3.SS3.p6.11.m11.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p6.11.m11.1.2.1" xref="S3.SS3.p6.11.m11.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.11.m11.1.2.3.2" xref="S3.SS3.p6.11.m11.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.11.m11.1.2.3.2.1" xref="S3.SS3.p6.11.m11.1.2.cmml">(</mo><mi id="S3.SS3.p6.11.m11.1.1" xref="S3.SS3.p6.11.m11.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.11.m11.1.2.3.2.2" xref="S3.SS3.p6.11.m11.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.11.m11.1b"><apply id="S3.SS3.p6.11.m11.1.2.cmml" xref="S3.SS3.p6.11.m11.1.2"><times id="S3.SS3.p6.11.m11.1.2.1.cmml" xref="S3.SS3.p6.11.m11.1.2.1"></times><apply id="S3.SS3.p6.11.m11.1.2.2.cmml" xref="S3.SS3.p6.11.m11.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p6.11.m11.1.2.2.1.cmml" xref="S3.SS3.p6.11.m11.1.2.2">subscript</csymbol><ci id="S3.SS3.p6.11.m11.1.2.2.2.cmml" xref="S3.SS3.p6.11.m11.1.2.2.2">𝑓</ci><ci id="S3.SS3.p6.11.m11.1.2.2.3.cmml" xref="S3.SS3.p6.11.m11.1.2.2.3">𝜃</ci></apply><ci id="S3.SS3.p6.11.m11.1.1.cmml" xref="S3.SS3.p6.11.m11.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.11.m11.1c">f_{\theta}({\bm{x}})</annotation></semantics></math> to be iteratively applied a exponential number of times. Direct recurrent backpropagation through such a number of iterative computations has been proven to be unstable to train. While in principle computing <math id="S3.SS3.p6.12.m12.2" class="ltx_Math" alttext="\operatorname*{arg\,min}_{\bm{y}}E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.12.m12.2a"><mrow id="S3.SS3.p6.12.m12.2.3" xref="S3.SS3.p6.12.m12.2.3.cmml"><mrow id="S3.SS3.p6.12.m12.2.3.2" xref="S3.SS3.p6.12.m12.2.3.2.cmml"><msub id="S3.SS3.p6.12.m12.2.3.2.1" xref="S3.SS3.p6.12.m12.2.3.2.1.cmml"><mrow id="S3.SS3.p6.12.m12.2.3.2.1.2" xref="S3.SS3.p6.12.m12.2.3.2.1.2.cmml"><mi id="S3.SS3.p6.12.m12.2.3.2.1.2.2" xref="S3.SS3.p6.12.m12.2.3.2.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S3.SS3.p6.12.m12.2.3.2.1.2.1" xref="S3.SS3.p6.12.m12.2.3.2.1.2.1.cmml">​</mo><mi id="S3.SS3.p6.12.m12.2.3.2.1.2.3" xref="S3.SS3.p6.12.m12.2.3.2.1.2.3.cmml">min</mi></mrow><mi id="S3.SS3.p6.12.m12.2.3.2.1.3" xref="S3.SS3.p6.12.m12.2.3.2.1.3.cmml">𝒚</mi></msub><mo lspace="0.167em" id="S3.SS3.p6.12.m12.2.3.2a" xref="S3.SS3.p6.12.m12.2.3.2.cmml">⁡</mo><mi id="S3.SS3.p6.12.m12.2.3.2.2" xref="S3.SS3.p6.12.m12.2.3.2.2.cmml">E</mi></mrow><mo lspace="0em" rspace="0em" id="S3.SS3.p6.12.m12.2.3.1" xref="S3.SS3.p6.12.m12.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.12.m12.2.3.3.2" xref="S3.SS3.p6.12.m12.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.12.m12.2.3.3.2.1" xref="S3.SS3.p6.12.m12.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.12.m12.1.1" xref="S3.SS3.p6.12.m12.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.12.m12.2.3.3.2.2" xref="S3.SS3.p6.12.m12.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.12.m12.2.2" xref="S3.SS3.p6.12.m12.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.12.m12.2.3.3.2.3" xref="S3.SS3.p6.12.m12.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.12.m12.2b"><apply id="S3.SS3.p6.12.m12.2.3.cmml" xref="S3.SS3.p6.12.m12.2.3"><times id="S3.SS3.p6.12.m12.2.3.1.cmml" xref="S3.SS3.p6.12.m12.2.3.1"></times><apply id="S3.SS3.p6.12.m12.2.3.2.cmml" xref="S3.SS3.p6.12.m12.2.3.2"><apply id="S3.SS3.p6.12.m12.2.3.2.1.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1"><csymbol cd="ambiguous" id="S3.SS3.p6.12.m12.2.3.2.1.1.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1">subscript</csymbol><apply id="S3.SS3.p6.12.m12.2.3.2.1.2.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1.2"><times id="S3.SS3.p6.12.m12.2.3.2.1.2.1.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1.2.1"></times><ci id="S3.SS3.p6.12.m12.2.3.2.1.2.2.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1.2.2">arg</ci><ci id="S3.SS3.p6.12.m12.2.3.2.1.2.3.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1.2.3">min</ci></apply><ci id="S3.SS3.p6.12.m12.2.3.2.1.3.cmml" xref="S3.SS3.p6.12.m12.2.3.2.1.3">𝒚</ci></apply><ci id="S3.SS3.p6.12.m12.2.3.2.2.cmml" xref="S3.SS3.p6.12.m12.2.3.2.2">𝐸</ci></apply><interval closure="open" id="S3.SS3.p6.12.m12.2.3.3.1.cmml" xref="S3.SS3.p6.12.m12.2.3.3.2"><ci id="S3.SS3.p6.12.m12.1.1.cmml" xref="S3.SS3.p6.12.m12.1.1">𝒙</ci><ci id="S3.SS3.p6.12.m12.2.2.cmml" xref="S3.SS3.p6.12.m12.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.12.m12.2c">\operatorname*{arg\,min}_{\bm{y}}E({\bm{x}},{\bm{y}})</annotation></semantics></math>, would require a similar number of computations, we may train <math id="S3.SS3.p6.13.m13.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.13.m13.2a"><mrow id="S3.SS3.p6.13.m13.2.3" xref="S3.SS3.p6.13.m13.2.3.cmml"><mi id="S3.SS3.p6.13.m13.2.3.2" xref="S3.SS3.p6.13.m13.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.13.m13.2.3.1" xref="S3.SS3.p6.13.m13.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.13.m13.2.3.3.2" xref="S3.SS3.p6.13.m13.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.13.m13.2.3.3.2.1" xref="S3.SS3.p6.13.m13.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.13.m13.1.1" xref="S3.SS3.p6.13.m13.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.13.m13.2.3.3.2.2" xref="S3.SS3.p6.13.m13.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.13.m13.2.2" xref="S3.SS3.p6.13.m13.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.13.m13.2.3.3.2.3" xref="S3.SS3.p6.13.m13.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.13.m13.2b"><apply id="S3.SS3.p6.13.m13.2.3.cmml" xref="S3.SS3.p6.13.m13.2.3"><times id="S3.SS3.p6.13.m13.2.3.1.cmml" xref="S3.SS3.p6.13.m13.2.3.1"></times><ci id="S3.SS3.p6.13.m13.2.3.2.cmml" xref="S3.SS3.p6.13.m13.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p6.13.m13.2.3.3.1.cmml" xref="S3.SS3.p6.13.m13.2.3.3.2"><ci id="S3.SS3.p6.13.m13.1.1.cmml" xref="S3.SS3.p6.13.m13.1.1">𝒙</ci><ci id="S3.SS3.p6.13.m13.2.2.cmml" xref="S3.SS3.p6.13.m13.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.13.m13.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> with a simple inexact energy minimization procedure and run a more extensive minimization procedure at test time. We next analyze the computational complexity of representing <math id="S3.SS3.p6.14.m14.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p6.14.m14.2a"><mrow id="S3.SS3.p6.14.m14.2.3" xref="S3.SS3.p6.14.m14.2.3.cmml"><mi id="S3.SS3.p6.14.m14.2.3.2" xref="S3.SS3.p6.14.m14.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.14.m14.2.3.1" xref="S3.SS3.p6.14.m14.2.3.1.cmml">​</mo><mrow id="S3.SS3.p6.14.m14.2.3.3.2" xref="S3.SS3.p6.14.m14.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p6.14.m14.2.3.3.2.1" xref="S3.SS3.p6.14.m14.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p6.14.m14.1.1" xref="S3.SS3.p6.14.m14.1.1.cmml">𝒙</mi><mo id="S3.SS3.p6.14.m14.2.3.3.2.2" xref="S3.SS3.p6.14.m14.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p6.14.m14.2.2" xref="S3.SS3.p6.14.m14.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p6.14.m14.2.3.3.2.3" xref="S3.SS3.p6.14.m14.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.14.m14.2b"><apply id="S3.SS3.p6.14.m14.2.3.cmml" xref="S3.SS3.p6.14.m14.2.3"><times id="S3.SS3.p6.14.m14.2.3.1.cmml" xref="S3.SS3.p6.14.m14.2.3.1"></times><ci id="S3.SS3.p6.14.m14.2.3.2.cmml" xref="S3.SS3.p6.14.m14.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.p6.14.m14.2.3.3.1.cmml" xref="S3.SS3.p6.14.m14.2.3.3.2"><ci id="S3.SS3.p6.14.m14.1.1.cmml" xref="S3.SS3.p6.14.m14.1.1">𝒙</ci><ci id="S3.SS3.p6.14.m14.2.2.cmml" xref="S3.SS3.p6.14.m14.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.14.m14.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> and <math id="S3.SS3.p6.15.m15.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p6.15.m15.1a"><mrow id="S3.SS3.p6.15.m15.1.2" xref="S3.SS3.p6.15.m15.1.2.cmml"><mi id="S3.SS3.p6.15.m15.1.2.2" xref="S3.SS3.p6.15.m15.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p6.15.m15.1.2.1" xref="S3.SS3.p6.15.m15.1.2.1.cmml">​</mo><mrow id="S3.SS3.p6.15.m15.1.2.3.2" xref="S3.SS3.p6.15.m15.1.2.cmml"><mo stretchy="false" id="S3.SS3.p6.15.m15.1.2.3.2.1" xref="S3.SS3.p6.15.m15.1.2.cmml">(</mo><mi id="S3.SS3.p6.15.m15.1.1" xref="S3.SS3.p6.15.m15.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p6.15.m15.1.2.3.2.2" xref="S3.SS3.p6.15.m15.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p6.15.m15.1b"><apply id="S3.SS3.p6.15.m15.1.2.cmml" xref="S3.SS3.p6.15.m15.1.2"><times id="S3.SS3.p6.15.m15.1.2.1.cmml" xref="S3.SS3.p6.15.m15.1.2.1"></times><ci id="S3.SS3.p6.15.m15.1.2.2.cmml" xref="S3.SS3.p6.15.m15.1.2.2">𝑓</ci><ci id="S3.SS3.p6.15.m15.1.1.cmml" xref="S3.SS3.p6.15.m15.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p6.15.m15.1c">f({\bm{x}})</annotation></semantics></math> as a function of problem size.</p>
</div>
<div id="Thmtheorem2" class="ltx_theorem ltx_theorem_theorem">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="Thmtheorem2.1.1.1" class="ltx_text ltx_font_bold">Remark 2</span></span><span id="Thmtheorem2.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="Thmtheorem2.p1" class="ltx_para">
<p id="Thmtheorem2.p1.2" class="ltx_p"><span id="Thmtheorem2.p1.2.2" class="ltx_text ltx_font_italic">As the underlying number of variables in 3-SAT problem increases, we may construct <math id="Thmtheorem2.p1.1.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="Thmtheorem2.p1.1.1.m1.2a"><mrow id="Thmtheorem2.p1.1.1.m1.2.3" xref="Thmtheorem2.p1.1.1.m1.2.3.cmml"><mi id="Thmtheorem2.p1.1.1.m1.2.3.2" xref="Thmtheorem2.p1.1.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="Thmtheorem2.p1.1.1.m1.2.3.1" xref="Thmtheorem2.p1.1.1.m1.2.3.1.cmml">​</mo><mrow id="Thmtheorem2.p1.1.1.m1.2.3.3.2" xref="Thmtheorem2.p1.1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="Thmtheorem2.p1.1.1.m1.2.3.3.2.1" xref="Thmtheorem2.p1.1.1.m1.2.3.3.1.cmml">(</mo><mi id="Thmtheorem2.p1.1.1.m1.1.1" xref="Thmtheorem2.p1.1.1.m1.1.1.cmml">𝐱</mi><mo id="Thmtheorem2.p1.1.1.m1.2.3.3.2.2" xref="Thmtheorem2.p1.1.1.m1.2.3.3.1.cmml">,</mo><mi id="Thmtheorem2.p1.1.1.m1.2.2" xref="Thmtheorem2.p1.1.1.m1.2.2.cmml">𝐲</mi><mo stretchy="false" id="Thmtheorem2.p1.1.1.m1.2.3.3.2.3" xref="Thmtheorem2.p1.1.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="Thmtheorem2.p1.1.1.m1.2b"><apply id="Thmtheorem2.p1.1.1.m1.2.3.cmml" xref="Thmtheorem2.p1.1.1.m1.2.3"><times id="Thmtheorem2.p1.1.1.m1.2.3.1.cmml" xref="Thmtheorem2.p1.1.1.m1.2.3.1"></times><ci id="Thmtheorem2.p1.1.1.m1.2.3.2.cmml" xref="Thmtheorem2.p1.1.1.m1.2.3.2">𝐸</ci><interval closure="open" id="Thmtheorem2.p1.1.1.m1.2.3.3.1.cmml" xref="Thmtheorem2.p1.1.1.m1.2.3.3.2"><ci id="Thmtheorem2.p1.1.1.m1.1.1.cmml" xref="Thmtheorem2.p1.1.1.m1.1.1">𝐱</ci><ci id="Thmtheorem2.p1.1.1.m1.2.2.cmml" xref="Thmtheorem2.p1.1.1.m1.2.2">𝐲</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="Thmtheorem2.p1.1.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> which can be evaluated at any input in time polynomial in the number of variables, but for which the computational complexity of encoding a feedforward solution <math id="Thmtheorem2.p1.2.2.m2.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="Thmtheorem2.p1.2.2.m2.1a"><mrow id="Thmtheorem2.p1.2.2.m2.1.2" xref="Thmtheorem2.p1.2.2.m2.1.2.cmml"><mi id="Thmtheorem2.p1.2.2.m2.1.2.2" xref="Thmtheorem2.p1.2.2.m2.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="Thmtheorem2.p1.2.2.m2.1.2.1" xref="Thmtheorem2.p1.2.2.m2.1.2.1.cmml">​</mo><mrow id="Thmtheorem2.p1.2.2.m2.1.2.3.2" xref="Thmtheorem2.p1.2.2.m2.1.2.cmml"><mo stretchy="false" id="Thmtheorem2.p1.2.2.m2.1.2.3.2.1" xref="Thmtheorem2.p1.2.2.m2.1.2.cmml">(</mo><mi id="Thmtheorem2.p1.2.2.m2.1.1" xref="Thmtheorem2.p1.2.2.m2.1.1.cmml">𝐱</mi><mo stretchy="false" id="Thmtheorem2.p1.2.2.m2.1.2.3.2.2" xref="Thmtheorem2.p1.2.2.m2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="Thmtheorem2.p1.2.2.m2.1b"><apply id="Thmtheorem2.p1.2.2.m2.1.2.cmml" xref="Thmtheorem2.p1.2.2.m2.1.2"><times id="Thmtheorem2.p1.2.2.m2.1.2.1.cmml" xref="Thmtheorem2.p1.2.2.m2.1.2.1"></times><ci id="Thmtheorem2.p1.2.2.m2.1.2.2.cmml" xref="Thmtheorem2.p1.2.2.m2.1.2.2">𝑓</ci><ci id="Thmtheorem2.p1.2.2.m2.1.1.cmml" xref="Thmtheorem2.p1.2.2.m2.1.1">𝐱</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="Thmtheorem2.p1.2.2.m2.1c">f({\bm{x}})</annotation></semantics></math> which may be evaluated at any input is (worse-case) exponential in the number of underlying variables.</span></p>
</div>
</div>
<div id="S3.SS3.2" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S3.SS3.2.p1" class="ltx_para">
<p id="S3.SS3.2.p1.2" class="ltx_p">Our constructed energy function <math id="S3.SS3.2.p1.1.m1.2" class="ltx_Math" alttext="E({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.2.p1.1.m1.2a"><mrow id="S3.SS3.2.p1.1.m1.2.3" xref="S3.SS3.2.p1.1.m1.2.3.cmml"><mi id="S3.SS3.2.p1.1.m1.2.3.2" xref="S3.SS3.2.p1.1.m1.2.3.2.cmml">E</mi><mo lspace="0em" rspace="0em" id="S3.SS3.2.p1.1.m1.2.3.1" xref="S3.SS3.2.p1.1.m1.2.3.1.cmml">​</mo><mrow id="S3.SS3.2.p1.1.m1.2.3.3.2" xref="S3.SS3.2.p1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.2.p1.1.m1.2.3.3.2.1" xref="S3.SS3.2.p1.1.m1.2.3.3.1.cmml">(</mo><mi id="S3.SS3.2.p1.1.m1.1.1" xref="S3.SS3.2.p1.1.m1.1.1.cmml">𝒙</mi><mo id="S3.SS3.2.p1.1.m1.2.3.3.2.2" xref="S3.SS3.2.p1.1.m1.2.3.3.1.cmml">,</mo><mi id="S3.SS3.2.p1.1.m1.2.2" xref="S3.SS3.2.p1.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.2.p1.1.m1.2.3.3.2.3" xref="S3.SS3.2.p1.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.2.p1.1.m1.2b"><apply id="S3.SS3.2.p1.1.m1.2.3.cmml" xref="S3.SS3.2.p1.1.m1.2.3"><times id="S3.SS3.2.p1.1.m1.2.3.1.cmml" xref="S3.SS3.2.p1.1.m1.2.3.1"></times><ci id="S3.SS3.2.p1.1.m1.2.3.2.cmml" xref="S3.SS3.2.p1.1.m1.2.3.2">𝐸</ci><interval closure="open" id="S3.SS3.2.p1.1.m1.2.3.3.1.cmml" xref="S3.SS3.2.p1.1.m1.2.3.3.2"><ci id="S3.SS3.2.p1.1.m1.1.1.cmml" xref="S3.SS3.2.p1.1.m1.1.1">𝒙</ci><ci id="S3.SS3.2.p1.1.m1.2.2.cmml" xref="S3.SS3.2.p1.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.2.p1.1.m1.2c">E({\bm{x}},{\bm{y}})</annotation></semantics></math> above may be evaluated in time polynomial in the number of input variables. In contrast, ETH implies that constructing <math id="S3.SS3.2.p1.2.m2.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.2.p1.2.m2.1a"><mrow id="S3.SS3.2.p1.2.m2.1.2" xref="S3.SS3.2.p1.2.m2.1.2.cmml"><mi id="S3.SS3.2.p1.2.m2.1.2.2" xref="S3.SS3.2.p1.2.m2.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.2.p1.2.m2.1.2.1" xref="S3.SS3.2.p1.2.m2.1.2.1.cmml">​</mo><mrow id="S3.SS3.2.p1.2.m2.1.2.3.2" xref="S3.SS3.2.p1.2.m2.1.2.cmml"><mo stretchy="false" id="S3.SS3.2.p1.2.m2.1.2.3.2.1" xref="S3.SS3.2.p1.2.m2.1.2.cmml">(</mo><mi id="S3.SS3.2.p1.2.m2.1.1" xref="S3.SS3.2.p1.2.m2.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.2.p1.2.m2.1.2.3.2.2" xref="S3.SS3.2.p1.2.m2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.2.p1.2.m2.1b"><apply id="S3.SS3.2.p1.2.m2.1.2.cmml" xref="S3.SS3.2.p1.2.m2.1.2"><times id="S3.SS3.2.p1.2.m2.1.2.1.cmml" xref="S3.SS3.2.p1.2.m2.1.2.1"></times><ci id="S3.SS3.2.p1.2.m2.1.2.2.cmml" xref="S3.SS3.2.p1.2.m2.1.2.2">𝑓</ci><ci id="S3.SS3.2.p1.2.m2.1.1.cmml" xref="S3.SS3.2.p1.2.m2.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.2.p1.2.m2.1c">f({\bm{x}})</annotation></semantics></math>, which corresponds to solving the 3-SAT problem, is exponential in the number of underlying variables.
∎</p>
</div>
</div>
<div id="S3.SS3.p7" class="ltx_para">
<p id="S3.SS3.p7.7" class="ltx_p">Similar to our previous remark, our result implies that representing <math id="S3.SS3.p7.1.m1.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p7.1.m1.1a"><mrow id="S3.SS3.p7.1.m1.1.2" xref="S3.SS3.p7.1.m1.1.2.cmml"><mi id="S3.SS3.p7.1.m1.1.2.2" xref="S3.SS3.p7.1.m1.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p7.1.m1.1.2.1" xref="S3.SS3.p7.1.m1.1.2.1.cmml">​</mo><mrow id="S3.SS3.p7.1.m1.1.2.3.2" xref="S3.SS3.p7.1.m1.1.2.cmml"><mo stretchy="false" id="S3.SS3.p7.1.m1.1.2.3.2.1" xref="S3.SS3.p7.1.m1.1.2.cmml">(</mo><mi id="S3.SS3.p7.1.m1.1.1" xref="S3.SS3.p7.1.m1.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p7.1.m1.1.2.3.2.2" xref="S3.SS3.p7.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.1.m1.1b"><apply id="S3.SS3.p7.1.m1.1.2.cmml" xref="S3.SS3.p7.1.m1.1.2"><times id="S3.SS3.p7.1.m1.1.2.1.cmml" xref="S3.SS3.p7.1.m1.1.2.1"></times><ci id="S3.SS3.p7.1.m1.1.2.2.cmml" xref="S3.SS3.p7.1.m1.1.2.2">𝑓</ci><ci id="S3.SS3.p7.1.m1.1.1.cmml" xref="S3.SS3.p7.1.m1.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.1.m1.1c">f({\bm{x}})</annotation></semantics></math> requires exponentially more computation as the underlying problem size of a 3-SAT problem increases. In particular, our remark has implications for <span id="S3.SS3.p7.7.1" class="ltx_text ltx_font_bold">generalization</span>. If we learn <math id="S3.SS3.p7.2.m2.1" class="ltx_Math" alttext="f_{\theta}({\bm{x}})" display="inline"><semantics id="S3.SS3.p7.2.m2.1a"><mrow id="S3.SS3.p7.2.m2.1.2" xref="S3.SS3.p7.2.m2.1.2.cmml"><msub id="S3.SS3.p7.2.m2.1.2.2" xref="S3.SS3.p7.2.m2.1.2.2.cmml"><mi id="S3.SS3.p7.2.m2.1.2.2.2" xref="S3.SS3.p7.2.m2.1.2.2.2.cmml">f</mi><mi id="S3.SS3.p7.2.m2.1.2.2.3" xref="S3.SS3.p7.2.m2.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p7.2.m2.1.2.1" xref="S3.SS3.p7.2.m2.1.2.1.cmml">​</mo><mrow id="S3.SS3.p7.2.m2.1.2.3.2" xref="S3.SS3.p7.2.m2.1.2.cmml"><mo stretchy="false" id="S3.SS3.p7.2.m2.1.2.3.2.1" xref="S3.SS3.p7.2.m2.1.2.cmml">(</mo><mi id="S3.SS3.p7.2.m2.1.1" xref="S3.SS3.p7.2.m2.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p7.2.m2.1.2.3.2.2" xref="S3.SS3.p7.2.m2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.2.m2.1b"><apply id="S3.SS3.p7.2.m2.1.2.cmml" xref="S3.SS3.p7.2.m2.1.2"><times id="S3.SS3.p7.2.m2.1.2.1.cmml" xref="S3.SS3.p7.2.m2.1.2.1"></times><apply id="S3.SS3.p7.2.m2.1.2.2.cmml" xref="S3.SS3.p7.2.m2.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p7.2.m2.1.2.2.1.cmml" xref="S3.SS3.p7.2.m2.1.2.2">subscript</csymbol><ci id="S3.SS3.p7.2.m2.1.2.2.2.cmml" xref="S3.SS3.p7.2.m2.1.2.2.2">𝑓</ci><ci id="S3.SS3.p7.2.m2.1.2.2.3.cmml" xref="S3.SS3.p7.2.m2.1.2.2.3">𝜃</ci></apply><ci id="S3.SS3.p7.2.m2.1.1.cmml" xref="S3.SS3.p7.2.m2.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.2.m2.1c">f_{\theta}({\bm{x}})</annotation></semantics></math> and <math id="S3.SS3.p7.3.m3.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p7.3.m3.2a"><mrow id="S3.SS3.p7.3.m3.2.3" xref="S3.SS3.p7.3.m3.2.3.cmml"><msub id="S3.SS3.p7.3.m3.2.3.2" xref="S3.SS3.p7.3.m3.2.3.2.cmml"><mi id="S3.SS3.p7.3.m3.2.3.2.2" xref="S3.SS3.p7.3.m3.2.3.2.2.cmml">E</mi><mi id="S3.SS3.p7.3.m3.2.3.2.3" xref="S3.SS3.p7.3.m3.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p7.3.m3.2.3.1" xref="S3.SS3.p7.3.m3.2.3.1.cmml">​</mo><mrow id="S3.SS3.p7.3.m3.2.3.3.2" xref="S3.SS3.p7.3.m3.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p7.3.m3.2.3.3.2.1" xref="S3.SS3.p7.3.m3.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p7.3.m3.1.1" xref="S3.SS3.p7.3.m3.1.1.cmml">𝒙</mi><mo id="S3.SS3.p7.3.m3.2.3.3.2.2" xref="S3.SS3.p7.3.m3.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p7.3.m3.2.2" xref="S3.SS3.p7.3.m3.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p7.3.m3.2.3.3.2.3" xref="S3.SS3.p7.3.m3.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.3.m3.2b"><apply id="S3.SS3.p7.3.m3.2.3.cmml" xref="S3.SS3.p7.3.m3.2.3"><times id="S3.SS3.p7.3.m3.2.3.1.cmml" xref="S3.SS3.p7.3.m3.2.3.1"></times><apply id="S3.SS3.p7.3.m3.2.3.2.cmml" xref="S3.SS3.p7.3.m3.2.3.2"><csymbol cd="ambiguous" id="S3.SS3.p7.3.m3.2.3.2.1.cmml" xref="S3.SS3.p7.3.m3.2.3.2">subscript</csymbol><ci id="S3.SS3.p7.3.m3.2.3.2.2.cmml" xref="S3.SS3.p7.3.m3.2.3.2.2">𝐸</ci><ci id="S3.SS3.p7.3.m3.2.3.2.3.cmml" xref="S3.SS3.p7.3.m3.2.3.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS3.p7.3.m3.2.3.3.1.cmml" xref="S3.SS3.p7.3.m3.2.3.3.2"><ci id="S3.SS3.p7.3.m3.1.1.cmml" xref="S3.SS3.p7.3.m3.1.1">𝒙</ci><ci id="S3.SS3.p7.3.m3.2.2.cmml" xref="S3.SS3.p7.3.m3.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.3.m3.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math> on smaller problem instances of <math id="S3.SS3.p7.4.m4.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="S3.SS3.p7.4.m4.1a"><mi id="S3.SS3.p7.4.m4.1.1" xref="S3.SS3.p7.4.m4.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.4.m4.1b"><ci id="S3.SS3.p7.4.m4.1.1.cmml" xref="S3.SS3.p7.4.m4.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.4.m4.1c">{\bm{x}}</annotation></semantics></math>, for our neural networks to generalize correctly to larger problems instances, the underlying computation executed by the neural network must increase polynomially for <math id="S3.SS3.p7.5.m5.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="S3.SS3.p7.5.m5.2a"><mrow id="S3.SS3.p7.5.m5.2.3" xref="S3.SS3.p7.5.m5.2.3.cmml"><msub id="S3.SS3.p7.5.m5.2.3.2" xref="S3.SS3.p7.5.m5.2.3.2.cmml"><mi id="S3.SS3.p7.5.m5.2.3.2.2" xref="S3.SS3.p7.5.m5.2.3.2.2.cmml">E</mi><mi id="S3.SS3.p7.5.m5.2.3.2.3" xref="S3.SS3.p7.5.m5.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p7.5.m5.2.3.1" xref="S3.SS3.p7.5.m5.2.3.1.cmml">​</mo><mrow id="S3.SS3.p7.5.m5.2.3.3.2" xref="S3.SS3.p7.5.m5.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS3.p7.5.m5.2.3.3.2.1" xref="S3.SS3.p7.5.m5.2.3.3.1.cmml">(</mo><mi id="S3.SS3.p7.5.m5.1.1" xref="S3.SS3.p7.5.m5.1.1.cmml">𝒙</mi><mo id="S3.SS3.p7.5.m5.2.3.3.2.2" xref="S3.SS3.p7.5.m5.2.3.3.1.cmml">,</mo><mi id="S3.SS3.p7.5.m5.2.2" xref="S3.SS3.p7.5.m5.2.2.cmml">𝒚</mi><mo stretchy="false" id="S3.SS3.p7.5.m5.2.3.3.2.3" xref="S3.SS3.p7.5.m5.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.5.m5.2b"><apply id="S3.SS3.p7.5.m5.2.3.cmml" xref="S3.SS3.p7.5.m5.2.3"><times id="S3.SS3.p7.5.m5.2.3.1.cmml" xref="S3.SS3.p7.5.m5.2.3.1"></times><apply id="S3.SS3.p7.5.m5.2.3.2.cmml" xref="S3.SS3.p7.5.m5.2.3.2"><csymbol cd="ambiguous" id="S3.SS3.p7.5.m5.2.3.2.1.cmml" xref="S3.SS3.p7.5.m5.2.3.2">subscript</csymbol><ci id="S3.SS3.p7.5.m5.2.3.2.2.cmml" xref="S3.SS3.p7.5.m5.2.3.2.2">𝐸</ci><ci id="S3.SS3.p7.5.m5.2.3.2.3.cmml" xref="S3.SS3.p7.5.m5.2.3.2.3">𝜃</ci></apply><interval closure="open" id="S3.SS3.p7.5.m5.2.3.3.1.cmml" xref="S3.SS3.p7.5.m5.2.3.3.2"><ci id="S3.SS3.p7.5.m5.1.1.cmml" xref="S3.SS3.p7.5.m5.1.1">𝒙</ci><ci id="S3.SS3.p7.5.m5.2.2.cmml" xref="S3.SS3.p7.5.m5.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.5.m5.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math> and exponentially for <math id="S3.SS3.p7.6.m6.1" class="ltx_Math" alttext="f_{\theta}(x)" display="inline"><semantics id="S3.SS3.p7.6.m6.1a"><mrow id="S3.SS3.p7.6.m6.1.2" xref="S3.SS3.p7.6.m6.1.2.cmml"><msub id="S3.SS3.p7.6.m6.1.2.2" xref="S3.SS3.p7.6.m6.1.2.2.cmml"><mi id="S3.SS3.p7.6.m6.1.2.2.2" xref="S3.SS3.p7.6.m6.1.2.2.2.cmml">f</mi><mi id="S3.SS3.p7.6.m6.1.2.2.3" xref="S3.SS3.p7.6.m6.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S3.SS3.p7.6.m6.1.2.1" xref="S3.SS3.p7.6.m6.1.2.1.cmml">​</mo><mrow id="S3.SS3.p7.6.m6.1.2.3.2" xref="S3.SS3.p7.6.m6.1.2.cmml"><mo stretchy="false" id="S3.SS3.p7.6.m6.1.2.3.2.1" xref="S3.SS3.p7.6.m6.1.2.cmml">(</mo><mi id="S3.SS3.p7.6.m6.1.1" xref="S3.SS3.p7.6.m6.1.1.cmml">x</mi><mo stretchy="false" id="S3.SS3.p7.6.m6.1.2.3.2.2" xref="S3.SS3.p7.6.m6.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.6.m6.1b"><apply id="S3.SS3.p7.6.m6.1.2.cmml" xref="S3.SS3.p7.6.m6.1.2"><times id="S3.SS3.p7.6.m6.1.2.1.cmml" xref="S3.SS3.p7.6.m6.1.2.1"></times><apply id="S3.SS3.p7.6.m6.1.2.2.cmml" xref="S3.SS3.p7.6.m6.1.2.2"><csymbol cd="ambiguous" id="S3.SS3.p7.6.m6.1.2.2.1.cmml" xref="S3.SS3.p7.6.m6.1.2.2">subscript</csymbol><ci id="S3.SS3.p7.6.m6.1.2.2.2.cmml" xref="S3.SS3.p7.6.m6.1.2.2.2">𝑓</ci><ci id="S3.SS3.p7.6.m6.1.2.2.3.cmml" xref="S3.SS3.p7.6.m6.1.2.2.3">𝜃</ci></apply><ci id="S3.SS3.p7.6.m6.1.1.cmml" xref="S3.SS3.p7.6.m6.1.1">𝑥</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.6.m6.1c">f_{\theta}(x)</annotation></semantics></math>. Existing architectures, such as transformers and graph networks, adaptively increase computation polynomially for larger inputs. In contrast, few architectures can adaptively increase computation exponentially for larger problem instances. To realize <math id="S3.SS3.p7.7.m7.1" class="ltx_Math" alttext="f({\bm{x}})" display="inline"><semantics id="S3.SS3.p7.7.m7.1a"><mrow id="S3.SS3.p7.7.m7.1.2" xref="S3.SS3.p7.7.m7.1.2.cmml"><mi id="S3.SS3.p7.7.m7.1.2.2" xref="S3.SS3.p7.7.m7.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="S3.SS3.p7.7.m7.1.2.1" xref="S3.SS3.p7.7.m7.1.2.1.cmml">​</mo><mrow id="S3.SS3.p7.7.m7.1.2.3.2" xref="S3.SS3.p7.7.m7.1.2.cmml"><mo stretchy="false" id="S3.SS3.p7.7.m7.1.2.3.2.1" xref="S3.SS3.p7.7.m7.1.2.cmml">(</mo><mi id="S3.SS3.p7.7.m7.1.1" xref="S3.SS3.p7.7.m7.1.1.cmml">𝒙</mi><mo stretchy="false" id="S3.SS3.p7.7.m7.1.2.3.2.2" xref="S3.SS3.p7.7.m7.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS3.p7.7.m7.1b"><apply id="S3.SS3.p7.7.m7.1.2.cmml" xref="S3.SS3.p7.7.m7.1.2"><times id="S3.SS3.p7.7.m7.1.2.1.cmml" xref="S3.SS3.p7.7.m7.1.2.1"></times><ci id="S3.SS3.p7.7.m7.1.2.2.cmml" xref="S3.SS3.p7.7.m7.1.2.2">𝑓</ci><ci id="S3.SS3.p7.7.m7.1.1.cmml" xref="S3.SS3.p7.7.m7.1.1">𝒙</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS3.p7.7.m7.1c">f({\bm{x}})</annotation></semantics></math>, we thus require iterative approaches which can execute exponentially longer on larger inputs. As seen in Figure <a href="#S4.F2" title="Figure 2 ‣ 4.2 Graphical Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, this is difficult for existing approaches.</p>
</div>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Experiments</h2>

<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Experimental Setup</h3>

<div id="S4.SS1.p1" class="ltx_para">
<p id="S4.SS1.p1.1" class="ltx_p">We compare IREM with feedforward and iterative baselines for reasoning. We discuss each approach in detail below.</p>
</div>
<div id="S4.SS1.p2" class="ltx_para">
<p id="S4.SS1.p2.1" class="ltx_p"><span id="S4.SS1.p2.1.1" class="ltx_text ltx_font_bold">Feedforward Computation.</span>  First, we compare with (one-step) feedforward computation, where we train a neural network that directly outputs the values of solutions.</p>
</div>
<div id="S4.SS1.p3" class="ltx_para">
<p id="S4.SS1.p3.1" class="ltx_p"><span id="S4.SS1.p3.1.1" class="ltx_text ltx_font_bold">Recurrent Network Computation.</span>  Next, we compare our approach with methods utilizing a recurrent neural network to execute iterative computation. Recurrent architectures have been shown to successfully execute reasoning recently <cite class="ltx_cite ltx_citemacro_citep">(Schwarzschild et al., <a href="#bib.bib38" title="" class="ltx_ref">2021</a>)</cite>. We use a LSTM network to represent iterative computation.</p>
</div>
<div id="S4.SS1.p4" class="ltx_para">
<p id="S4.SS1.p4.1" class="ltx_p"><span id="S4.SS1.p4.1.1" class="ltx_text ltx_font_bold">Learned Programmatic Computation.</span>  We compare our method with past works which construct iterative computation through building programmatic structures with neural networks. We compare our method with the recent architecture and training objective of PonderNet <cite class="ltx_cite ltx_citemacro_citep">(Banino et al., <a href="#bib.bib5" title="" class="ltx_ref">2021</a>)</cite>, which variationally learns both a halting probability and individual computation step networks.</p>
</div>
<div id="S4.SS1.p5" class="ltx_para">
<p id="S4.SS1.p5.1" class="ltx_p"><span id="S4.SS1.p5.1.1" class="ltx_text ltx_font_bold">Iterative Feedforward Computation.</span>  We further compare our approach with direct iterative application of a feedforward computation. We train the iterative feedforward computation using an iterative denoising objective <cite class="ltx_cite ltx_citemacro_citep">(Sohl-Dickstein et al., <a href="#bib.bib39" title="" class="ltx_ref">2015</a>)</cite>.</p>
</div>
<div id="S4.SS1.p6" class="ltx_para">
<p id="S4.SS1.p6.1" class="ltx_p">We scale network sizes to ensure that each individual baseline has roughly the same number of parameters. The architectural details of each model are provided in Appendix <a href="#A4" title="Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">D</span></a> of the paper. We utilize MLP neural networks for continuous algorithmic reasoning tasks and graph neural networks for graph algorithmic reasoning tasks. All iterative methods are trained with 5 steps of reasoning. We provide comparisons with each baseline on graphical algorithmic reasoning in Section <a href="#S4.SS2" title="4.2 Graphical Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.2</span></a> and on continuous algorithmic reasoning in Section <a href="#S4.SS3" title="4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a>. We provide additional benchmark comparisons of IREM on an existing iterative image denoising task in Appendix <a href="#A1" title="Appendix A Image Denoising ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A</span></a>.</p>
</div>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Graphical Algorithmic Reasoning</h3>

<figure id="S4.F2" class="ltx_figure"><img src="/html/2206.15448/assets/x2.png" id="S4.F2.g1" class="ltx_graphics ltx_img_landscape" width="461" height="191" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 2: </span><span id="S4.F2.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Computation Steps vs Problem Size<span id="S4.F2.2.1.1" class="ltx_text ltx_font_medium"> – Error on shortest path computation as a factor of graph size and number of computation executed. Models are trained 5 steps of iterative computation on graph sizes of 10. IREM is able to generalize computation to larger graphs and a greater number of iterative computation steps, while a recurrent network fails to do so.</span></span></figcaption>
</figure>
<div id="S4.SS2.p1" class="ltx_para">
<p id="S4.SS2.p1.1" class="ltx_p"><span id="S4.SS2.p1.1.1" class="ltx_text ltx_font_bold">Setup.</span>  We first evaluate our approach on graphical algorithmic reasoning. We train models on fully connected graphs with between 2 to 10 nodes, and evaluate performance on larger fully connected graphs with size 15 nodes. We report the underlying elementwise mean squared error between predictions from models and their associated ground truth outputs. We evaluate performance on the three different graphical algorithmic reasoning tasks, aiming to capture different aspects of reasoning, which we detail below, with additional details about each dataset in Appendix <a href="#A3" title="Appendix C Experimental Details ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a>.</p>
<ol id="S4.I1" class="ltx_enumerate">
<li id="S4.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S4.I1.i1.p1" class="ltx_para">
<p id="S4.I1.i1.p1.1" class="ltx_p"><span id="S4.I1.i1.p1.1.1" class="ltx_text ltx_font_italic">Edge Copy</span>: We first test the ability of models to copy and output the values of all edges in a dense graph that is given as input. This task serves as a simple test for iterative reasoning, and requires a method to sequentially copy over input edge values to the output.</p>
</div>
</li>
<li id="S4.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S4.I1.i2.p1" class="ltx_para">
<p id="S4.I1.i2.p1.1" class="ltx_p"><span id="S4.I1.i2.p1.1.1" class="ltx_text ltx_font_italic">Connected Components</span>: Next, we evaluate the ability of models to infer the underlying connected components of a graph. We construct a sparse graph, where 5% of all fully connected edges exist, and ask models to predict a binary indicator on whether a node of a graph is connected to another for all pairs of nodes in a graph. This task tests structural discovery, an aspect of cognitive reasoning <cite class="ltx_cite ltx_citemacro_citep">(Kemp &amp; Tenenbaum, <a href="#bib.bib27" title="" class="ltx_ref">2008</a>)</cite>.</p>
</div>
</li>
<li id="S4.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="S4.I1.i3.p1" class="ltx_para">
<p id="S4.I1.i3.p1.1" class="ltx_p"><span id="S4.I1.i3.p1.1.1" class="ltx_text ltx_font_italic">Shortest Path</span>: Finally, we evaluate the ability of models to compute the shortest path distances between all pairs of nodes in a graph. This task tests for planning, and the underlying calculation necessary to compute the shortest paths between nodes is analogous to that of planning.</p>
</div>
</li>
</ol>
</div>
<div id="S4.SS2.p2" class="ltx_para">
<p id="S4.SS2.p2.1" class="ltx_p"><span id="S4.SS2.p2.1.1" class="ltx_text ltx_font_bold">Quantitative Results.</span>  We present quantitative results on graphical algorithmic reasoning in the left column of Table <a href="#S3.T1" title="Table 1 ‣ 3.3 Analysis ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>. During test time, we evaluate on problems with either similar or larger sizes than those seen during training. Across all three tasks, IREM outperforms all the compared baselines. This difference in performance is magnified when evaluating on larger graphs during testing. Approaches other than IREM fail even on the relatively simple edge copy task. This is due to the fact that the small size of graph networks (hidden size 128), requires methods to iteratively copy different subsets of input edges to output predictions. While IREM successfully learns this iterative computation, the compared baselines were unable to do so.</p>
</div>
<figure id="S4.T2" class="ltx_table">
<div id="S4.T2.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:224.2pt;height:76.5pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-19.8pt,6.8pt) scale(0.85,0.85) ;">
<table id="S4.T2.1.1" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S4.T2.1.1.1.1" class="ltx_tr">
<th id="S4.T2.1.1.1.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S4.T2.1.1.1.1.1.1" class="ltx_text ltx_font_bold">Replay</span></th>
<th id="S4.T2.1.1.1.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S4.T2.1.1.1.1.2.1" class="ltx_text ltx_font_bold">Truncate</span></th>
<th id="S4.T2.1.1.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S4.T2.1.1.1.1.3.1" class="ltx_text ltx_font_bold">Test</span></th>
<th id="S4.T2.1.1.1.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S4.T2.1.1.1.1.4.1" class="ltx_text ltx_font_bold">Training</span></th>
<th id="S4.T2.1.1.1.1.5" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="S4.T2.1.1.1.1.5.1" class="ltx_text ltx_font_bold">Memory</span></th>
</tr>
<tr id="S4.T2.1.1.2.2" class="ltx_tr">
<th id="S4.T2.1.1.2.2.1" class="ltx_td ltx_align_center ltx_th ltx_th_column"><span id="S4.T2.1.1.2.2.1.1" class="ltx_text ltx_font_bold">Buffer</span></th>
<th id="S4.T2.1.1.2.2.2" class="ltx_td ltx_align_center ltx_th ltx_th_column"><span id="S4.T2.1.1.2.2.2.1" class="ltx_text ltx_font_bold">Gradient</span></th>
<th id="S4.T2.1.1.2.2.3" class="ltx_td ltx_align_center ltx_th ltx_th_column"><span id="S4.T2.1.1.2.2.3.1" class="ltx_text ltx_font_bold">Performance</span></th>
<th id="S4.T2.1.1.2.2.4" class="ltx_td ltx_align_center ltx_th ltx_th_column"><span id="S4.T2.1.1.2.2.4.1" class="ltx_text ltx_font_bold">Speed</span></th>
<th id="S4.T2.1.1.2.2.5" class="ltx_td ltx_align_center ltx_th ltx_th_column"><span id="S4.T2.1.1.2.2.5.1" class="ltx_text ltx_font_bold">Usage</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S4.T2.1.1.3.1" class="ltx_tr">
<td id="S4.T2.1.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t">No</td>
<td id="S4.T2.1.1.3.1.2" class="ltx_td ltx_align_center ltx_border_t">No</td>
<td id="S4.T2.1.1.3.1.3" class="ltx_td ltx_align_center ltx_border_t">0.0491</td>
<td id="S4.T2.1.1.3.1.4" class="ltx_td ltx_align_center ltx_border_t">87%</td>
<td id="S4.T2.1.1.3.1.5" class="ltx_td ltx_align_center ltx_border_t">295%</td>
</tr>
<tr id="S4.T2.1.1.4.2" class="ltx_tr">
<td id="S4.T2.1.1.4.2.1" class="ltx_td ltx_align_center">Yes</td>
<td id="S4.T2.1.1.4.2.2" class="ltx_td ltx_align_center">No</td>
<td id="S4.T2.1.1.4.2.3" class="ltx_td ltx_align_center">0.0287</td>
<td id="S4.T2.1.1.4.2.4" class="ltx_td ltx_align_center">83%</td>
<td id="S4.T2.1.1.4.2.5" class="ltx_td ltx_align_center">295%</td>
</tr>
<tr id="S4.T2.1.1.5.3" class="ltx_tr">
<td id="S4.T2.1.1.5.3.1" class="ltx_td ltx_align_center ltx_border_bb">Yes</td>
<td id="S4.T2.1.1.5.3.2" class="ltx_td ltx_align_center ltx_border_bb">Yes</td>
<td id="S4.T2.1.1.5.3.3" class="ltx_td ltx_align_center ltx_border_bb">0.0274</td>
<td id="S4.T2.1.1.5.3.4" class="ltx_td ltx_align_center ltx_border_bb">100%</td>
<td id="S4.T2.1.1.5.3.5" class="ltx_td ltx_align_center ltx_border_bb">100%</td>
</tr>
</tbody>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 2: </span><span id="S4.T2.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Ablations –<span id="S4.T2.3.1.1" class="ltx_text ltx_font_medium"> Ablations of proposed components of IREM on test performance on the shortest path algorithmic reasoning task. The use a replay buffer boosts the underlying performance of IREM and truncating gradient backpropogation to the last step of optimization reduces both training time and memory cost. </span></span></figcaption>
</figure>
<div id="S4.SS2.p3" class="ltx_para">
<p id="S4.SS2.p3.1" class="ltx_p"><span id="S4.SS2.p3.1.1" class="ltx_text ltx_font_bold">Adaptive Computation.</span>  Next, we analyze the ability of IREM and baselines to adapt its underlying computational budget to different larger problem instances. In Figure <a href="#S4.F2" title="Figure 2 ‣ 4.2 Graphical Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, we illustrate shortest path computation error on different input graphs sizes at test time. While all methods are only trained with 5 steps of iterative computation on size 10 graphs, we find that IREM can generalize iterative computation to size 30 graphs. In contrast, the learned iterative baselines, such as the recurrent network in Figure <a href="#S4.F2" title="Figure 2 ‣ 4.2 Graphical Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, fail to do so. We find similar behavior across other graphical algorithmic reasoning tasks.</p>
</div>
<div id="S4.SS2.p4" class="ltx_para">
<p id="S4.SS2.p4.1" class="ltx_p"><span id="S4.SS2.p4.1.1" class="ltx_text ltx_font_bold">Ablations.</span>  We run an ablation analysis on the impact of utilizing a replay buffer to train IREM, as well as the effect of truncating backpropagation to the last step of optimization. In Table <a href="#S4.T2" title="Table 2 ‣ 4.2 Graphical Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>, we find that utilizing a replay buffer significantly improves the performance of IREM. We further find that truncating backpropagation to the last step of optimization has a limited impact on IREM and greatly reduces the overall memory cost of training, as well as slightly improving the training speed of IREM.</p>
</div>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>Continuous Algorithmic Reasoning</h3>

<figure id="S4.F3" class="ltx_figure"><img src="/html/2206.15448/assets/x3.png" id="S4.F3.g1" class="ltx_graphics ltx_centering ltx_img_portrait" width="461" height="573" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 3: </span><span id="S4.F3.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Computation Steps vs Problem Difficulty<span id="S4.F3.2.1.1" class="ltx_text ltx_font_medium"> – Illustration of MSE error of prediction as a factor of the test time difficulty (harder difficulty right) of the task and computation steps applied. Each model is trained with 5 steps of iterative computation. Models missing in plots have errors greater than the range displayed in the plot. The error of IREM improves with the number of underlying algorithmic computation steps, with larger number of computation steps benefiting performance on harder algorithmic tasks. </span></span></figcaption>
</figure>
<div id="S4.SS3.p1" class="ltx_para">
<p id="S4.SS3.p1.1" class="ltx_p"><span id="S4.SS3.p1.1.1" class="ltx_text ltx_font_bold">Data Setup.</span>  We next evaluate IREM and baselines on continuous algorithmic reasoning tasks. We apply algorithmic operations on input vectors of size 400 (resized to <math id="S4.SS3.p1.1.m1.1" class="ltx_Math" alttext="20\times 20" display="inline"><semantics id="S4.SS3.p1.1.m1.1a"><mrow id="S4.SS3.p1.1.m1.1.1" xref="S4.SS3.p1.1.m1.1.1.cmml"><mn id="S4.SS3.p1.1.m1.1.1.2" xref="S4.SS3.p1.1.m1.1.1.2.cmml">20</mn><mo lspace="0.222em" rspace="0.222em" id="S4.SS3.p1.1.m1.1.1.1" xref="S4.SS3.p1.1.m1.1.1.1.cmml">×</mo><mn id="S4.SS3.p1.1.m1.1.1.3" xref="S4.SS3.p1.1.m1.1.1.3.cmml">20</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.1.m1.1b"><apply id="S4.SS3.p1.1.m1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1"><times id="S4.SS3.p1.1.m1.1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1.1"></times><cn type="integer" id="S4.SS3.p1.1.m1.1.1.2.cmml" xref="S4.SS3.p1.1.m1.1.1.2">20</cn><cn type="integer" id="S4.SS3.p1.1.m1.1.1.3.cmml" xref="S4.SS3.p1.1.m1.1.1.3">20</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.1.m1.1c">20\times 20</annotation></semantics></math> matrices for matrix operations). We report the underlying MSE error between the predictions and the associated ground truth outputs on test problem instances. We evaluate different methods on the following three tasks, aiming to capture different aspects of reasoning, with additional details in the Appendix <a href="#A3" title="Appendix C Experimental Details ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a>.</p>
</div>
<div id="S4.SS3.p2" class="ltx_para">
<ol id="S4.I2" class="ltx_enumerate">
<li id="S4.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S4.I2.i1.p1" class="ltx_para">
<p id="S4.I2.i1.p1.1" class="ltx_p"><span id="S4.I2.i1.p1.1.1" class="ltx_text ltx_font_italic">Addition</span>: We first evaluate the algorithmic computation of addition. We train networks to add entries in two separate input vectors (element-wise). We construct harder variants of the addition problems at test time by feeding input vectors with larger magnitudes. This task aims to test simple arithmetic reasoning.</p>
</div>
</li>
<li id="S4.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S4.I2.i2.p1" class="ltx_para">
<p id="S4.I2.i2.p1.5" class="ltx_p"><span id="S4.I2.i2.p1.5.1" class="ltx_text ltx_font_italic">Matrix Completion</span>: Next, we evaluate the algorithmic computation of matrix completion. We mask out <math id="S4.I2.i2.p1.1.m1.1" class="ltx_Math" alttext="50\%" display="inline"><semantics id="S4.I2.i2.p1.1.m1.1a"><mrow id="S4.I2.i2.p1.1.m1.1.1" xref="S4.I2.i2.p1.1.m1.1.1.cmml"><mn id="S4.I2.i2.p1.1.m1.1.1.2" xref="S4.I2.i2.p1.1.m1.1.1.2.cmml">50</mn><mo id="S4.I2.i2.p1.1.m1.1.1.1" xref="S4.I2.i2.p1.1.m1.1.1.1.cmml">%</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.I2.i2.p1.1.m1.1b"><apply id="S4.I2.i2.p1.1.m1.1.1.cmml" xref="S4.I2.i2.p1.1.m1.1.1"><csymbol cd="latexml" id="S4.I2.i2.p1.1.m1.1.1.1.cmml" xref="S4.I2.i2.p1.1.m1.1.1.1">percent</csymbol><cn type="integer" id="S4.I2.i2.p1.1.m1.1.1.2.cmml" xref="S4.I2.i2.p1.1.m1.1.1.2">50</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.I2.i2.p1.1.m1.1c">50\%</annotation></semantics></math> of the entries of a low-rank input matrix constructed from two separate low-rank matrices <math id="S4.I2.i2.p1.2.m2.1" class="ltx_Math" alttext="U" display="inline"><semantics id="S4.I2.i2.p1.2.m2.1a"><mi id="S4.I2.i2.p1.2.m2.1.1" xref="S4.I2.i2.p1.2.m2.1.1.cmml">U</mi><annotation-xml encoding="MathML-Content" id="S4.I2.i2.p1.2.m2.1b"><ci id="S4.I2.i2.p1.2.m2.1.1.cmml" xref="S4.I2.i2.p1.2.m2.1.1">𝑈</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.I2.i2.p1.2.m2.1c">U</annotation></semantics></math> and <math id="S4.I2.i2.p1.3.m3.1" class="ltx_Math" alttext="V" display="inline"><semantics id="S4.I2.i2.p1.3.m3.1a"><mi id="S4.I2.i2.p1.3.m3.1.1" xref="S4.I2.i2.p1.3.m3.1.1.cmml">V</mi><annotation-xml encoding="MathML-Content" id="S4.I2.i2.p1.3.m3.1b"><ci id="S4.I2.i2.p1.3.m3.1.1.cmml" xref="S4.I2.i2.p1.3.m3.1.1">𝑉</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.I2.i2.p1.3.m3.1c">V</annotation></semantics></math>, and train networks to reconstruct the original input matrix. We construct harder variants of the matrix completion problem at test time by increasing the complexity of <math id="S4.I2.i2.p1.4.m4.1" class="ltx_Math" alttext="U" display="inline"><semantics id="S4.I2.i2.p1.4.m4.1a"><mi id="S4.I2.i2.p1.4.m4.1.1" xref="S4.I2.i2.p1.4.m4.1.1.cmml">U</mi><annotation-xml encoding="MathML-Content" id="S4.I2.i2.p1.4.m4.1b"><ci id="S4.I2.i2.p1.4.m4.1.1.cmml" xref="S4.I2.i2.p1.4.m4.1.1">𝑈</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.I2.i2.p1.4.m4.1c">U</annotation></semantics></math> and <math id="S4.I2.i2.p1.5.m5.1" class="ltx_Math" alttext="V" display="inline"><semantics id="S4.I2.i2.p1.5.m5.1a"><mi id="S4.I2.i2.p1.5.m5.1.1" xref="S4.I2.i2.p1.5.m5.1.1.cmml">V</mi><annotation-xml encoding="MathML-Content" id="S4.I2.i2.p1.5.m5.1b"><ci id="S4.I2.i2.p1.5.m5.1.1.cmml" xref="S4.I2.i2.p1.5.m5.1.1">𝑉</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.I2.i2.p1.5.m5.1c">V</annotation></semantics></math>. This task aims to test both structural and analogical reasoning, with both shown to be equivalent to matrix completion <cite class="ltx_cite ltx_citemacro_citep">(Lampinen et al., <a href="#bib.bib28" title="" class="ltx_ref">2017</a>)</cite>.</p>
</div>
</li>
<li id="S4.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="S4.I2.i3.p1" class="ltx_para">
<p id="S4.I2.i3.p1.1" class="ltx_p"><span id="S4.I2.i3.p1.1.1" class="ltx_text ltx_font_italic">Matrix Inverse</span>: Finally, we evaluate the algorithmic computation of matrix inverse. We train networks to compute the matrix inverse of an input matrix. We construct harder matrix inverse problems by considering less well-conditioned input matrices. This task aims to test the numerical reasoning, with matrix inversion a crucial operation across various numerical algorithms.</p>
</div>
</li>
</ol>
</div>
<figure id="S4.F4" class="ltx_figure"><img src="/html/2206.15448/assets/x4.png" id="S4.F4.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="124" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 4: </span><span id="S4.F4.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Qualitative Illustration of Addition<span id="S4.F4.2.1.1" class="ltx_text ltx_font_medium"> – Illustration of per element error on the addition task as a function of underlying number iterative computation steps run with IREM. Individual inputs gradually approach ground truth values, with different elements approaching zero error at different rates.</span></span></figcaption>
</figure>
<figure id="S4.F5" class="ltx_figure"><img src="/html/2206.15448/assets/x5.png" id="S4.F5.g1" class="ltx_graphics ltx_img_landscape" width="461" height="173" alt="Refer to caption">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 5: </span><span id="S4.F5.8.4" class="ltx_text ltx_font_bold" style="font-size:90%;">Energy Landscape<span id="S4.F5.8.4.4" class="ltx_text ltx_font_medium"> – Plot of predicted energy values for <math id="S4.F5.5.1.1.m1.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S4.F5.5.1.1.m1.1b"><mi id="S4.F5.5.1.1.m1.1.1" xref="S4.F5.5.1.1.m1.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S4.F5.5.1.1.m1.1c"><ci id="S4.F5.5.1.1.m1.1.1.cmml" xref="S4.F5.5.1.1.m1.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.5.1.1.m1.1d">{\bm{y}}</annotation></semantics></math> and the corresponding MSE distance of <math id="S4.F5.6.2.2.m2.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S4.F5.6.2.2.m2.1b"><mi id="S4.F5.6.2.2.m2.1.1" xref="S4.F5.6.2.2.m2.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S4.F5.6.2.2.m2.1c"><ci id="S4.F5.6.2.2.m2.1.1.cmml" xref="S4.F5.6.2.2.m2.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.6.2.2.m2.1d">{\bm{y}}</annotation></semantics></math> from the problem solution. The predicted energy of <math id="S4.F5.7.3.3.m3.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S4.F5.7.3.3.m3.1b"><mi id="S4.F5.7.3.3.m3.1.1" xref="S4.F5.7.3.3.m3.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S4.F5.7.3.3.m3.1c"><ci id="S4.F5.7.3.3.m3.1.1.cmml" xref="S4.F5.7.3.3.m3.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.7.3.3.m3.1d">{\bm{y}}</annotation></semantics></math> correlates well with the distance of <math id="S4.F5.8.4.4.m4.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="S4.F5.8.4.4.m4.1b"><mi id="S4.F5.8.4.4.m4.1.1" xref="S4.F5.8.4.4.m4.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="S4.F5.8.4.4.m4.1c"><ci id="S4.F5.8.4.4.m4.1.1.cmml" xref="S4.F5.8.4.4.m4.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.8.4.4.m4.1d">{\bm{y}}</annotation></semantics></math> from the ground truth. Plot for matrix inverse is similar to matrix completion. </span></span></figcaption>
</figure>
<figure id="S4.F6" class="ltx_figure">
<div id="S4.F6.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:335.0pt;height:15.7pt;vertical-align:-0.6pt;"><span class="ltx_transformed_inner" style="transform:translate(-47.2pt,2.1pt) scale(0.78,0.78) ;">
<p id="S4.F6.1.1" class="ltx_p"><math id="S4.F6.1.1.m1.5" class="ltx_Math" alttext="\begin{bmatrix}-0.695&amp;-0.840&amp;0.691\\
0.488&amp;-0.961&amp;-0.048\\
-0.824&amp;-0.235&amp;0.162\end{bmatrix}+\begin{bmatrix}-0.590&amp;0.972&amp;-0.544\\
-0.318&amp;0.748&amp;0.638\\
-0.216&amp;-0.118&amp;0.864\end{bmatrix}+\begin{bmatrix}-0.676&amp;-0.688&amp;0.422\\
0.075&amp;0.172&amp;-0.963\\
0.698&amp;0.837&amp;-0.735\end{bmatrix}+\begin{bmatrix}0.945&amp;-0.432&amp;-0.940\\
-0.567&amp;0.989&amp;-0.302\\
-0.526&amp;0.492&amp;0.574\end{bmatrix}={\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\begin{bmatrix}-1.033&amp;-1.076&amp;-0.433\\
-0.319&amp;1.010&amp;-0.677\\
-0.867&amp;0.984&amp;0.843\end{bmatrix}}" display="inline"><semantics id="S4.F6.1.1.m1.5a"><mrow id="S4.F6.1.1.m1.5.6" xref="S4.F6.1.1.m1.5.6.cmml"><mrow id="S4.F6.1.1.m1.5.6.2" xref="S4.F6.1.1.m1.5.6.2.cmml"><mrow id="S4.F6.1.1.m1.1.1.3" xref="S4.F6.1.1.m1.1.1.2.cmml"><mo id="S4.F6.1.1.m1.1.1.3.1" xref="S4.F6.1.1.m1.1.1.2.1.cmml">[</mo><mtable columnspacing="5pt" rowspacing="0pt" id="S4.F6.1.1.m1.1.1.1.1" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mtr id="S4.F6.1.1.m1.1.1.1.1a" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mtd id="S4.F6.1.1.m1.1.1.1.1b" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mrow id="S4.F6.1.1.m1.1.1.1.1.1.1.1" xref="S4.F6.1.1.m1.1.1.1.1.1.1.1.cmml"><mo id="S4.F6.1.1.m1.1.1.1.1.1.1.1a" xref="S4.F6.1.1.m1.1.1.1.1.1.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.1.1.1.1.1.1.1.2" xref="S4.F6.1.1.m1.1.1.1.1.1.1.1.2.cmml">0.695</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.1.1.1.1c" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mrow id="S4.F6.1.1.m1.1.1.1.1.1.2.1" xref="S4.F6.1.1.m1.1.1.1.1.1.2.1.cmml"><mo id="S4.F6.1.1.m1.1.1.1.1.1.2.1a" xref="S4.F6.1.1.m1.1.1.1.1.1.2.1.cmml">−</mo><mn id="S4.F6.1.1.m1.1.1.1.1.1.2.1.2" xref="S4.F6.1.1.m1.1.1.1.1.1.2.1.2.cmml">0.840</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.1.1.1.1d" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mn id="S4.F6.1.1.m1.1.1.1.1.1.3.1" xref="S4.F6.1.1.m1.1.1.1.1.1.3.1.cmml">0.691</mn></mtd></mtr><mtr id="S4.F6.1.1.m1.1.1.1.1e" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mtd id="S4.F6.1.1.m1.1.1.1.1f" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mn id="S4.F6.1.1.m1.1.1.1.1.2.1.1" xref="S4.F6.1.1.m1.1.1.1.1.2.1.1.cmml">0.488</mn></mtd><mtd id="S4.F6.1.1.m1.1.1.1.1g" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mrow id="S4.F6.1.1.m1.1.1.1.1.2.2.1" xref="S4.F6.1.1.m1.1.1.1.1.2.2.1.cmml"><mo id="S4.F6.1.1.m1.1.1.1.1.2.2.1a" xref="S4.F6.1.1.m1.1.1.1.1.2.2.1.cmml">−</mo><mn id="S4.F6.1.1.m1.1.1.1.1.2.2.1.2" xref="S4.F6.1.1.m1.1.1.1.1.2.2.1.2.cmml">0.961</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.1.1.1.1h" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mrow id="S4.F6.1.1.m1.1.1.1.1.2.3.1" xref="S4.F6.1.1.m1.1.1.1.1.2.3.1.cmml"><mo id="S4.F6.1.1.m1.1.1.1.1.2.3.1a" xref="S4.F6.1.1.m1.1.1.1.1.2.3.1.cmml">−</mo><mn id="S4.F6.1.1.m1.1.1.1.1.2.3.1.2" xref="S4.F6.1.1.m1.1.1.1.1.2.3.1.2.cmml">0.048</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.1.1.1.1i" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mtd id="S4.F6.1.1.m1.1.1.1.1j" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mrow id="S4.F6.1.1.m1.1.1.1.1.3.1.1" xref="S4.F6.1.1.m1.1.1.1.1.3.1.1.cmml"><mo id="S4.F6.1.1.m1.1.1.1.1.3.1.1a" xref="S4.F6.1.1.m1.1.1.1.1.3.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.1.1.1.1.3.1.1.2" xref="S4.F6.1.1.m1.1.1.1.1.3.1.1.2.cmml">0.824</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.1.1.1.1k" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mrow id="S4.F6.1.1.m1.1.1.1.1.3.2.1" xref="S4.F6.1.1.m1.1.1.1.1.3.2.1.cmml"><mo id="S4.F6.1.1.m1.1.1.1.1.3.2.1a" xref="S4.F6.1.1.m1.1.1.1.1.3.2.1.cmml">−</mo><mn id="S4.F6.1.1.m1.1.1.1.1.3.2.1.2" xref="S4.F6.1.1.m1.1.1.1.1.3.2.1.2.cmml">0.235</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.1.1.1.1l" xref="S4.F6.1.1.m1.1.1.1.1.cmml"><mn id="S4.F6.1.1.m1.1.1.1.1.3.3.1" xref="S4.F6.1.1.m1.1.1.1.1.3.3.1.cmml">0.162</mn></mtd></mtr></mtable><mo id="S4.F6.1.1.m1.1.1.3.2" xref="S4.F6.1.1.m1.1.1.2.1.cmml">]</mo></mrow><mo id="S4.F6.1.1.m1.5.6.2.1" xref="S4.F6.1.1.m1.5.6.2.1.cmml">+</mo><mrow id="S4.F6.1.1.m1.2.2.3" xref="S4.F6.1.1.m1.2.2.2.cmml"><mo id="S4.F6.1.1.m1.2.2.3.1" xref="S4.F6.1.1.m1.2.2.2.1.cmml">[</mo><mtable columnspacing="5pt" rowspacing="0pt" id="S4.F6.1.1.m1.2.2.1.1" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mtr id="S4.F6.1.1.m1.2.2.1.1a" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mtd id="S4.F6.1.1.m1.2.2.1.1b" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mrow id="S4.F6.1.1.m1.2.2.1.1.1.1.1" xref="S4.F6.1.1.m1.2.2.1.1.1.1.1.cmml"><mo id="S4.F6.1.1.m1.2.2.1.1.1.1.1a" xref="S4.F6.1.1.m1.2.2.1.1.1.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.2.2.1.1.1.1.1.2" xref="S4.F6.1.1.m1.2.2.1.1.1.1.1.2.cmml">0.590</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.2.2.1.1c" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mn id="S4.F6.1.1.m1.2.2.1.1.1.2.1" xref="S4.F6.1.1.m1.2.2.1.1.1.2.1.cmml">0.972</mn></mtd><mtd id="S4.F6.1.1.m1.2.2.1.1d" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mrow id="S4.F6.1.1.m1.2.2.1.1.1.3.1" xref="S4.F6.1.1.m1.2.2.1.1.1.3.1.cmml"><mo id="S4.F6.1.1.m1.2.2.1.1.1.3.1a" xref="S4.F6.1.1.m1.2.2.1.1.1.3.1.cmml">−</mo><mn id="S4.F6.1.1.m1.2.2.1.1.1.3.1.2" xref="S4.F6.1.1.m1.2.2.1.1.1.3.1.2.cmml">0.544</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.2.2.1.1e" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mtd id="S4.F6.1.1.m1.2.2.1.1f" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mrow id="S4.F6.1.1.m1.2.2.1.1.2.1.1" xref="S4.F6.1.1.m1.2.2.1.1.2.1.1.cmml"><mo id="S4.F6.1.1.m1.2.2.1.1.2.1.1a" xref="S4.F6.1.1.m1.2.2.1.1.2.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.2.2.1.1.2.1.1.2" xref="S4.F6.1.1.m1.2.2.1.1.2.1.1.2.cmml">0.318</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.2.2.1.1g" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mn id="S4.F6.1.1.m1.2.2.1.1.2.2.1" xref="S4.F6.1.1.m1.2.2.1.1.2.2.1.cmml">0.748</mn></mtd><mtd id="S4.F6.1.1.m1.2.2.1.1h" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mn id="S4.F6.1.1.m1.2.2.1.1.2.3.1" xref="S4.F6.1.1.m1.2.2.1.1.2.3.1.cmml">0.638</mn></mtd></mtr><mtr id="S4.F6.1.1.m1.2.2.1.1i" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mtd id="S4.F6.1.1.m1.2.2.1.1j" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mrow id="S4.F6.1.1.m1.2.2.1.1.3.1.1" xref="S4.F6.1.1.m1.2.2.1.1.3.1.1.cmml"><mo id="S4.F6.1.1.m1.2.2.1.1.3.1.1a" xref="S4.F6.1.1.m1.2.2.1.1.3.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.2.2.1.1.3.1.1.2" xref="S4.F6.1.1.m1.2.2.1.1.3.1.1.2.cmml">0.216</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.2.2.1.1k" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mrow id="S4.F6.1.1.m1.2.2.1.1.3.2.1" xref="S4.F6.1.1.m1.2.2.1.1.3.2.1.cmml"><mo id="S4.F6.1.1.m1.2.2.1.1.3.2.1a" xref="S4.F6.1.1.m1.2.2.1.1.3.2.1.cmml">−</mo><mn id="S4.F6.1.1.m1.2.2.1.1.3.2.1.2" xref="S4.F6.1.1.m1.2.2.1.1.3.2.1.2.cmml">0.118</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.2.2.1.1l" xref="S4.F6.1.1.m1.2.2.1.1.cmml"><mn id="S4.F6.1.1.m1.2.2.1.1.3.3.1" xref="S4.F6.1.1.m1.2.2.1.1.3.3.1.cmml">0.864</mn></mtd></mtr></mtable><mo id="S4.F6.1.1.m1.2.2.3.2" xref="S4.F6.1.1.m1.2.2.2.1.cmml">]</mo></mrow><mo id="S4.F6.1.1.m1.5.6.2.1a" xref="S4.F6.1.1.m1.5.6.2.1.cmml">+</mo><mrow id="S4.F6.1.1.m1.3.3.3" xref="S4.F6.1.1.m1.3.3.2.cmml"><mo id="S4.F6.1.1.m1.3.3.3.1" xref="S4.F6.1.1.m1.3.3.2.1.cmml">[</mo><mtable columnspacing="5pt" rowspacing="0pt" id="S4.F6.1.1.m1.3.3.1.1" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mtr id="S4.F6.1.1.m1.3.3.1.1a" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mtd id="S4.F6.1.1.m1.3.3.1.1b" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mrow id="S4.F6.1.1.m1.3.3.1.1.1.1.1" xref="S4.F6.1.1.m1.3.3.1.1.1.1.1.cmml"><mo id="S4.F6.1.1.m1.3.3.1.1.1.1.1a" xref="S4.F6.1.1.m1.3.3.1.1.1.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.3.3.1.1.1.1.1.2" xref="S4.F6.1.1.m1.3.3.1.1.1.1.1.2.cmml">0.676</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.3.3.1.1c" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mrow id="S4.F6.1.1.m1.3.3.1.1.1.2.1" xref="S4.F6.1.1.m1.3.3.1.1.1.2.1.cmml"><mo id="S4.F6.1.1.m1.3.3.1.1.1.2.1a" xref="S4.F6.1.1.m1.3.3.1.1.1.2.1.cmml">−</mo><mn id="S4.F6.1.1.m1.3.3.1.1.1.2.1.2" xref="S4.F6.1.1.m1.3.3.1.1.1.2.1.2.cmml">0.688</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.3.3.1.1d" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mn id="S4.F6.1.1.m1.3.3.1.1.1.3.1" xref="S4.F6.1.1.m1.3.3.1.1.1.3.1.cmml">0.422</mn></mtd></mtr><mtr id="S4.F6.1.1.m1.3.3.1.1e" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mtd id="S4.F6.1.1.m1.3.3.1.1f" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mn id="S4.F6.1.1.m1.3.3.1.1.2.1.1" xref="S4.F6.1.1.m1.3.3.1.1.2.1.1.cmml">0.075</mn></mtd><mtd id="S4.F6.1.1.m1.3.3.1.1g" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mn id="S4.F6.1.1.m1.3.3.1.1.2.2.1" xref="S4.F6.1.1.m1.3.3.1.1.2.2.1.cmml">0.172</mn></mtd><mtd id="S4.F6.1.1.m1.3.3.1.1h" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mrow id="S4.F6.1.1.m1.3.3.1.1.2.3.1" xref="S4.F6.1.1.m1.3.3.1.1.2.3.1.cmml"><mo id="S4.F6.1.1.m1.3.3.1.1.2.3.1a" xref="S4.F6.1.1.m1.3.3.1.1.2.3.1.cmml">−</mo><mn id="S4.F6.1.1.m1.3.3.1.1.2.3.1.2" xref="S4.F6.1.1.m1.3.3.1.1.2.3.1.2.cmml">0.963</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.3.3.1.1i" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mtd id="S4.F6.1.1.m1.3.3.1.1j" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mn id="S4.F6.1.1.m1.3.3.1.1.3.1.1" xref="S4.F6.1.1.m1.3.3.1.1.3.1.1.cmml">0.698</mn></mtd><mtd id="S4.F6.1.1.m1.3.3.1.1k" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mn id="S4.F6.1.1.m1.3.3.1.1.3.2.1" xref="S4.F6.1.1.m1.3.3.1.1.3.2.1.cmml">0.837</mn></mtd><mtd id="S4.F6.1.1.m1.3.3.1.1l" xref="S4.F6.1.1.m1.3.3.1.1.cmml"><mrow id="S4.F6.1.1.m1.3.3.1.1.3.3.1" xref="S4.F6.1.1.m1.3.3.1.1.3.3.1.cmml"><mo id="S4.F6.1.1.m1.3.3.1.1.3.3.1a" xref="S4.F6.1.1.m1.3.3.1.1.3.3.1.cmml">−</mo><mn id="S4.F6.1.1.m1.3.3.1.1.3.3.1.2" xref="S4.F6.1.1.m1.3.3.1.1.3.3.1.2.cmml">0.735</mn></mrow></mtd></mtr></mtable><mo id="S4.F6.1.1.m1.3.3.3.2" xref="S4.F6.1.1.m1.3.3.2.1.cmml">]</mo></mrow><mo id="S4.F6.1.1.m1.5.6.2.1b" xref="S4.F6.1.1.m1.5.6.2.1.cmml">+</mo><mrow id="S4.F6.1.1.m1.4.4.3" xref="S4.F6.1.1.m1.4.4.2.cmml"><mo id="S4.F6.1.1.m1.4.4.3.1" xref="S4.F6.1.1.m1.4.4.2.1.cmml">[</mo><mtable columnspacing="5pt" rowspacing="0pt" id="S4.F6.1.1.m1.4.4.1.1" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mtr id="S4.F6.1.1.m1.4.4.1.1a" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mtd id="S4.F6.1.1.m1.4.4.1.1b" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mn id="S4.F6.1.1.m1.4.4.1.1.1.1.1" xref="S4.F6.1.1.m1.4.4.1.1.1.1.1.cmml">0.945</mn></mtd><mtd id="S4.F6.1.1.m1.4.4.1.1c" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mrow id="S4.F6.1.1.m1.4.4.1.1.1.2.1" xref="S4.F6.1.1.m1.4.4.1.1.1.2.1.cmml"><mo id="S4.F6.1.1.m1.4.4.1.1.1.2.1a" xref="S4.F6.1.1.m1.4.4.1.1.1.2.1.cmml">−</mo><mn id="S4.F6.1.1.m1.4.4.1.1.1.2.1.2" xref="S4.F6.1.1.m1.4.4.1.1.1.2.1.2.cmml">0.432</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.4.4.1.1d" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mrow id="S4.F6.1.1.m1.4.4.1.1.1.3.1" xref="S4.F6.1.1.m1.4.4.1.1.1.3.1.cmml"><mo id="S4.F6.1.1.m1.4.4.1.1.1.3.1a" xref="S4.F6.1.1.m1.4.4.1.1.1.3.1.cmml">−</mo><mn id="S4.F6.1.1.m1.4.4.1.1.1.3.1.2" xref="S4.F6.1.1.m1.4.4.1.1.1.3.1.2.cmml">0.940</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.4.4.1.1e" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mtd id="S4.F6.1.1.m1.4.4.1.1f" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mrow id="S4.F6.1.1.m1.4.4.1.1.2.1.1" xref="S4.F6.1.1.m1.4.4.1.1.2.1.1.cmml"><mo id="S4.F6.1.1.m1.4.4.1.1.2.1.1a" xref="S4.F6.1.1.m1.4.4.1.1.2.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.4.4.1.1.2.1.1.2" xref="S4.F6.1.1.m1.4.4.1.1.2.1.1.2.cmml">0.567</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.4.4.1.1g" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mn id="S4.F6.1.1.m1.4.4.1.1.2.2.1" xref="S4.F6.1.1.m1.4.4.1.1.2.2.1.cmml">0.989</mn></mtd><mtd id="S4.F6.1.1.m1.4.4.1.1h" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mrow id="S4.F6.1.1.m1.4.4.1.1.2.3.1" xref="S4.F6.1.1.m1.4.4.1.1.2.3.1.cmml"><mo id="S4.F6.1.1.m1.4.4.1.1.2.3.1a" xref="S4.F6.1.1.m1.4.4.1.1.2.3.1.cmml">−</mo><mn id="S4.F6.1.1.m1.4.4.1.1.2.3.1.2" xref="S4.F6.1.1.m1.4.4.1.1.2.3.1.2.cmml">0.302</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.4.4.1.1i" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mtd id="S4.F6.1.1.m1.4.4.1.1j" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mrow id="S4.F6.1.1.m1.4.4.1.1.3.1.1" xref="S4.F6.1.1.m1.4.4.1.1.3.1.1.cmml"><mo id="S4.F6.1.1.m1.4.4.1.1.3.1.1a" xref="S4.F6.1.1.m1.4.4.1.1.3.1.1.cmml">−</mo><mn id="S4.F6.1.1.m1.4.4.1.1.3.1.1.2" xref="S4.F6.1.1.m1.4.4.1.1.3.1.1.2.cmml">0.526</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.4.4.1.1k" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mn id="S4.F6.1.1.m1.4.4.1.1.3.2.1" xref="S4.F6.1.1.m1.4.4.1.1.3.2.1.cmml">0.492</mn></mtd><mtd id="S4.F6.1.1.m1.4.4.1.1l" xref="S4.F6.1.1.m1.4.4.1.1.cmml"><mn id="S4.F6.1.1.m1.4.4.1.1.3.3.1" xref="S4.F6.1.1.m1.4.4.1.1.3.3.1.cmml">0.574</mn></mtd></mtr></mtable><mo id="S4.F6.1.1.m1.4.4.3.2" xref="S4.F6.1.1.m1.4.4.2.1.cmml">]</mo></mrow></mrow><mo id="S4.F6.1.1.m1.5.6.1" xref="S4.F6.1.1.m1.5.6.1.cmml">=</mo><mrow id="S4.F6.1.1.m1.5.5.3" xref="S4.F6.1.1.m1.5.5.2.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.3.1" xref="S4.F6.1.1.m1.5.5.2.1.cmml">[</mo><mtable columnspacing="5pt" rowspacing="0pt" id="S4.F6.1.1.m1.5.5.1.1" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mtr id="S4.F6.1.1.m1.5.5.1.1a" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mtd id="S4.F6.1.1.m1.5.5.1.1b" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mrow id="S4.F6.1.1.m1.5.5.1.1.1.1.1" xref="S4.F6.1.1.m1.5.5.1.1.1.1.1.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.1.1.1a" xref="S4.F6.1.1.m1.5.5.1.1.1.1.1.cmml">−</mo><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.1.1.1.2" xref="S4.F6.1.1.m1.5.5.1.1.1.1.1.2.cmml">1.033</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.5.5.1.1c" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mrow id="S4.F6.1.1.m1.5.5.1.1.1.2.1" xref="S4.F6.1.1.m1.5.5.1.1.1.2.1.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.1.2.1a" xref="S4.F6.1.1.m1.5.5.1.1.1.2.1.cmml">−</mo><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.1.2.1.2" xref="S4.F6.1.1.m1.5.5.1.1.1.2.1.2.cmml">1.076</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.5.5.1.1d" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mrow id="S4.F6.1.1.m1.5.5.1.1.1.3.1" xref="S4.F6.1.1.m1.5.5.1.1.1.3.1.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.1.3.1a" xref="S4.F6.1.1.m1.5.5.1.1.1.3.1.cmml">−</mo><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.1.3.1.2" xref="S4.F6.1.1.m1.5.5.1.1.1.3.1.2.cmml">0.433</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.5.5.1.1e" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mtd id="S4.F6.1.1.m1.5.5.1.1f" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mrow id="S4.F6.1.1.m1.5.5.1.1.2.1.1" xref="S4.F6.1.1.m1.5.5.1.1.2.1.1.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.2.1.1a" xref="S4.F6.1.1.m1.5.5.1.1.2.1.1.cmml">−</mo><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.2.1.1.2" xref="S4.F6.1.1.m1.5.5.1.1.2.1.1.2.cmml">0.319</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.5.5.1.1g" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.2.2.1" xref="S4.F6.1.1.m1.5.5.1.1.2.2.1.cmml">1.010</mn></mtd><mtd id="S4.F6.1.1.m1.5.5.1.1h" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mrow id="S4.F6.1.1.m1.5.5.1.1.2.3.1" xref="S4.F6.1.1.m1.5.5.1.1.2.3.1.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.2.3.1a" xref="S4.F6.1.1.m1.5.5.1.1.2.3.1.cmml">−</mo><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.2.3.1.2" xref="S4.F6.1.1.m1.5.5.1.1.2.3.1.2.cmml">0.677</mn></mrow></mtd></mtr><mtr id="S4.F6.1.1.m1.5.5.1.1i" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mtd id="S4.F6.1.1.m1.5.5.1.1j" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mrow id="S4.F6.1.1.m1.5.5.1.1.3.1.1" xref="S4.F6.1.1.m1.5.5.1.1.3.1.1.cmml"><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.3.1.1a" xref="S4.F6.1.1.m1.5.5.1.1.3.1.1.cmml">−</mo><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.3.1.1.2" xref="S4.F6.1.1.m1.5.5.1.1.3.1.1.2.cmml">0.867</mn></mrow></mtd><mtd id="S4.F6.1.1.m1.5.5.1.1k" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.3.2.1" xref="S4.F6.1.1.m1.5.5.1.1.3.2.1.cmml">0.984</mn></mtd><mtd id="S4.F6.1.1.m1.5.5.1.1l" xref="S4.F6.1.1.m1.5.5.1.1.cmml"><mn mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.1.1.3.3.1" xref="S4.F6.1.1.m1.5.5.1.1.3.3.1.cmml">0.843</mn></mtd></mtr></mtable><mo mathcolor="#0000FF" id="S4.F6.1.1.m1.5.5.3.2" xref="S4.F6.1.1.m1.5.5.2.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.F6.1.1.m1.5b"><apply id="S4.F6.1.1.m1.5.6.cmml" xref="S4.F6.1.1.m1.5.6"><eq id="S4.F6.1.1.m1.5.6.1.cmml" xref="S4.F6.1.1.m1.5.6.1"></eq><apply id="S4.F6.1.1.m1.5.6.2.cmml" xref="S4.F6.1.1.m1.5.6.2"><plus id="S4.F6.1.1.m1.5.6.2.1.cmml" xref="S4.F6.1.1.m1.5.6.2.1"></plus><apply id="S4.F6.1.1.m1.1.1.2.cmml" xref="S4.F6.1.1.m1.1.1.3"><csymbol cd="latexml" id="S4.F6.1.1.m1.1.1.2.1.cmml" xref="S4.F6.1.1.m1.1.1.3.1">matrix</csymbol><matrix id="S4.F6.1.1.m1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1"><matrixrow id="S4.F6.1.1.m1.1.1.1.1a.cmml" xref="S4.F6.1.1.m1.1.1.1.1"><apply id="S4.F6.1.1.m1.1.1.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.1.1"><minus id="S4.F6.1.1.m1.1.1.1.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.1.1.1.2.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.1.1.2">0.695</cn></apply><apply id="S4.F6.1.1.m1.1.1.1.1.1.2.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.2.1"><minus id="S4.F6.1.1.m1.1.1.1.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.1.2.1.2.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.2.1.2">0.840</cn></apply><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.1.3.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.1.3.1">0.691</cn></matrixrow><matrixrow id="S4.F6.1.1.m1.1.1.1.1b.cmml" xref="S4.F6.1.1.m1.1.1.1.1"><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.1.1">0.488</cn><apply id="S4.F6.1.1.m1.1.1.1.1.2.2.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.2.1"><minus id="S4.F6.1.1.m1.1.1.1.1.2.2.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.2.2.1.2.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.2.1.2">0.961</cn></apply><apply id="S4.F6.1.1.m1.1.1.1.1.2.3.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.3.1"><minus id="S4.F6.1.1.m1.1.1.1.1.2.3.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.2.3.1.2.cmml" xref="S4.F6.1.1.m1.1.1.1.1.2.3.1.2">0.048</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.1.1.1.1c.cmml" xref="S4.F6.1.1.m1.1.1.1.1"><apply id="S4.F6.1.1.m1.1.1.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.1.1"><minus id="S4.F6.1.1.m1.1.1.1.1.3.1.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.3.1.1.2.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.1.1.2">0.824</cn></apply><apply id="S4.F6.1.1.m1.1.1.1.1.3.2.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.2.1"><minus id="S4.F6.1.1.m1.1.1.1.1.3.2.1.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.3.2.1.2.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.2.1.2">0.235</cn></apply><cn type="float" id="S4.F6.1.1.m1.1.1.1.1.3.3.1.cmml" xref="S4.F6.1.1.m1.1.1.1.1.3.3.1">0.162</cn></matrixrow></matrix></apply><apply id="S4.F6.1.1.m1.2.2.2.cmml" xref="S4.F6.1.1.m1.2.2.3"><csymbol cd="latexml" id="S4.F6.1.1.m1.2.2.2.1.cmml" xref="S4.F6.1.1.m1.2.2.3.1">matrix</csymbol><matrix id="S4.F6.1.1.m1.2.2.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1"><matrixrow id="S4.F6.1.1.m1.2.2.1.1a.cmml" xref="S4.F6.1.1.m1.2.2.1.1"><apply id="S4.F6.1.1.m1.2.2.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.1.1"><minus id="S4.F6.1.1.m1.2.2.1.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.1.1.1.2.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.1.1.2">0.590</cn></apply><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.1.2.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.2.1">0.972</cn><apply id="S4.F6.1.1.m1.2.2.1.1.1.3.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.3.1"><minus id="S4.F6.1.1.m1.2.2.1.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.1.3.1.2.cmml" xref="S4.F6.1.1.m1.2.2.1.1.1.3.1.2">0.544</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.2.2.1.1b.cmml" xref="S4.F6.1.1.m1.2.2.1.1"><apply id="S4.F6.1.1.m1.2.2.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.2.1.1"><minus id="S4.F6.1.1.m1.2.2.1.1.2.1.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.2.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.2.1.1.2.cmml" xref="S4.F6.1.1.m1.2.2.1.1.2.1.1.2">0.318</cn></apply><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.2.2.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.2.2.1">0.748</cn><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.2.3.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.2.3.1">0.638</cn></matrixrow><matrixrow id="S4.F6.1.1.m1.2.2.1.1c.cmml" xref="S4.F6.1.1.m1.2.2.1.1"><apply id="S4.F6.1.1.m1.2.2.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.1.1"><minus id="S4.F6.1.1.m1.2.2.1.1.3.1.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.3.1.1.2.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.1.1.2">0.216</cn></apply><apply id="S4.F6.1.1.m1.2.2.1.1.3.2.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.2.1"><minus id="S4.F6.1.1.m1.2.2.1.1.3.2.1.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.3.2.1.2.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.2.1.2">0.118</cn></apply><cn type="float" id="S4.F6.1.1.m1.2.2.1.1.3.3.1.cmml" xref="S4.F6.1.1.m1.2.2.1.1.3.3.1">0.864</cn></matrixrow></matrix></apply><apply id="S4.F6.1.1.m1.3.3.2.cmml" xref="S4.F6.1.1.m1.3.3.3"><csymbol cd="latexml" id="S4.F6.1.1.m1.3.3.2.1.cmml" xref="S4.F6.1.1.m1.3.3.3.1">matrix</csymbol><matrix id="S4.F6.1.1.m1.3.3.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1"><matrixrow id="S4.F6.1.1.m1.3.3.1.1a.cmml" xref="S4.F6.1.1.m1.3.3.1.1"><apply id="S4.F6.1.1.m1.3.3.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.1.1"><minus id="S4.F6.1.1.m1.3.3.1.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.1.1.1.2.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.1.1.2">0.676</cn></apply><apply id="S4.F6.1.1.m1.3.3.1.1.1.2.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.2.1"><minus id="S4.F6.1.1.m1.3.3.1.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.1.2.1.2.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.2.1.2">0.688</cn></apply><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.1.3.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.1.3.1">0.422</cn></matrixrow><matrixrow id="S4.F6.1.1.m1.3.3.1.1b.cmml" xref="S4.F6.1.1.m1.3.3.1.1"><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.2.1.1">0.075</cn><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.2.2.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.2.2.1">0.172</cn><apply id="S4.F6.1.1.m1.3.3.1.1.2.3.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.2.3.1"><minus id="S4.F6.1.1.m1.3.3.1.1.2.3.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.2.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.2.3.1.2.cmml" xref="S4.F6.1.1.m1.3.3.1.1.2.3.1.2">0.963</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.3.3.1.1c.cmml" xref="S4.F6.1.1.m1.3.3.1.1"><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.3.1.1">0.698</cn><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.3.2.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.3.2.1">0.837</cn><apply id="S4.F6.1.1.m1.3.3.1.1.3.3.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.3.3.1"><minus id="S4.F6.1.1.m1.3.3.1.1.3.3.1.1.cmml" xref="S4.F6.1.1.m1.3.3.1.1.3.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.3.3.1.1.3.3.1.2.cmml" xref="S4.F6.1.1.m1.3.3.1.1.3.3.1.2">0.735</cn></apply></matrixrow></matrix></apply><apply id="S4.F6.1.1.m1.4.4.2.cmml" xref="S4.F6.1.1.m1.4.4.3"><csymbol cd="latexml" id="S4.F6.1.1.m1.4.4.2.1.cmml" xref="S4.F6.1.1.m1.4.4.3.1">matrix</csymbol><matrix id="S4.F6.1.1.m1.4.4.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1"><matrixrow id="S4.F6.1.1.m1.4.4.1.1a.cmml" xref="S4.F6.1.1.m1.4.4.1.1"><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.1.1">0.945</cn><apply id="S4.F6.1.1.m1.4.4.1.1.1.2.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.2.1"><minus id="S4.F6.1.1.m1.4.4.1.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.1.2.1.2.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.2.1.2">0.432</cn></apply><apply id="S4.F6.1.1.m1.4.4.1.1.1.3.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.3.1"><minus id="S4.F6.1.1.m1.4.4.1.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.1.3.1.2.cmml" xref="S4.F6.1.1.m1.4.4.1.1.1.3.1.2">0.940</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.4.4.1.1b.cmml" xref="S4.F6.1.1.m1.4.4.1.1"><apply id="S4.F6.1.1.m1.4.4.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.1.1"><minus id="S4.F6.1.1.m1.4.4.1.1.2.1.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.2.1.1.2.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.1.1.2">0.567</cn></apply><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.2.2.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.2.1">0.989</cn><apply id="S4.F6.1.1.m1.4.4.1.1.2.3.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.3.1"><minus id="S4.F6.1.1.m1.4.4.1.1.2.3.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.2.3.1.2.cmml" xref="S4.F6.1.1.m1.4.4.1.1.2.3.1.2">0.302</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.4.4.1.1c.cmml" xref="S4.F6.1.1.m1.4.4.1.1"><apply id="S4.F6.1.1.m1.4.4.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.3.1.1"><minus id="S4.F6.1.1.m1.4.4.1.1.3.1.1.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.3.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.3.1.1.2.cmml" xref="S4.F6.1.1.m1.4.4.1.1.3.1.1.2">0.526</cn></apply><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.3.2.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.3.2.1">0.492</cn><cn type="float" id="S4.F6.1.1.m1.4.4.1.1.3.3.1.cmml" xref="S4.F6.1.1.m1.4.4.1.1.3.3.1">0.574</cn></matrixrow></matrix></apply></apply><apply id="S4.F6.1.1.m1.5.5.2.cmml" xref="S4.F6.1.1.m1.5.5.3"><csymbol cd="latexml" id="S4.F6.1.1.m1.5.5.2.1.cmml" xref="S4.F6.1.1.m1.5.5.3.1">matrix</csymbol><matrix id="S4.F6.1.1.m1.5.5.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1"><matrixrow id="S4.F6.1.1.m1.5.5.1.1a.cmml" xref="S4.F6.1.1.m1.5.5.1.1"><apply id="S4.F6.1.1.m1.5.5.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.1.1"><minus id="S4.F6.1.1.m1.5.5.1.1.1.1.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.1.1.1.2.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.1.1.2">1.033</cn></apply><apply id="S4.F6.1.1.m1.5.5.1.1.1.2.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.2.1"><minus id="S4.F6.1.1.m1.5.5.1.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.2.1"></minus><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.1.2.1.2.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.2.1.2">1.076</cn></apply><apply id="S4.F6.1.1.m1.5.5.1.1.1.3.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.3.1"><minus id="S4.F6.1.1.m1.5.5.1.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.1.3.1.2.cmml" xref="S4.F6.1.1.m1.5.5.1.1.1.3.1.2">0.433</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.5.5.1.1b.cmml" xref="S4.F6.1.1.m1.5.5.1.1"><apply id="S4.F6.1.1.m1.5.5.1.1.2.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.1.1"><minus id="S4.F6.1.1.m1.5.5.1.1.2.1.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.2.1.1.2.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.1.1.2">0.319</cn></apply><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.2.2.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.2.1">1.010</cn><apply id="S4.F6.1.1.m1.5.5.1.1.2.3.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.3.1"><minus id="S4.F6.1.1.m1.5.5.1.1.2.3.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.3.1"></minus><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.2.3.1.2.cmml" xref="S4.F6.1.1.m1.5.5.1.1.2.3.1.2">0.677</cn></apply></matrixrow><matrixrow id="S4.F6.1.1.m1.5.5.1.1c.cmml" xref="S4.F6.1.1.m1.5.5.1.1"><apply id="S4.F6.1.1.m1.5.5.1.1.3.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.3.1.1"><minus id="S4.F6.1.1.m1.5.5.1.1.3.1.1.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.3.1.1"></minus><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.3.1.1.2.cmml" xref="S4.F6.1.1.m1.5.5.1.1.3.1.1.2">0.867</cn></apply><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.3.2.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.3.2.1">0.984</cn><cn type="float" id="S4.F6.1.1.m1.5.5.1.1.3.3.1.cmml" xref="S4.F6.1.1.m1.5.5.1.1.3.3.1">0.843</cn></matrixrow></matrix></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.F6.1.1.m1.5c">\begin{bmatrix}-0.695&amp;-0.840&amp;0.691\\
0.488&amp;-0.961&amp;-0.048\\
-0.824&amp;-0.235&amp;0.162\end{bmatrix}+\begin{bmatrix}-0.590&amp;0.972&amp;-0.544\\
-0.318&amp;0.748&amp;0.638\\
-0.216&amp;-0.118&amp;0.864\end{bmatrix}+\begin{bmatrix}-0.676&amp;-0.688&amp;0.422\\
0.075&amp;0.172&amp;-0.963\\
0.698&amp;0.837&amp;-0.735\end{bmatrix}+\begin{bmatrix}0.945&amp;-0.432&amp;-0.940\\
-0.567&amp;0.989&amp;-0.302\\
-0.526&amp;0.492&amp;0.574\end{bmatrix}={\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\begin{bmatrix}-1.033&amp;-1.076&amp;-0.433\\
-0.319&amp;1.010&amp;-0.677\\
-0.867&amp;0.984&amp;0.843\end{bmatrix}}</annotation></semantics></math></p>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 6: </span><span id="S4.F6.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Addition Composition<span id="S4.F6.3.1.1" class="ltx_text ltx_font_medium"> – Illustration of predictions from IREM (in blue) when three separate addition executions are nested together. IREM is applied on vectors with 400 entries – we visualize the first 9 elements of inputs and predictions.</span></span></figcaption>
</figure>
<div id="S4.SS3.p3" class="ltx_para">
<p id="S4.SS3.p3.1" class="ltx_p"><span id="S4.SS3.p3.1.1" class="ltx_text ltx_font_bold">Quantitative Results.</span>  We present quantitative results on each of our three continuous algorithmic reasoning problems on the right side of Table <a href="#S3.T1" title="Table 1 ‣ 3.3 Analysis ‣ 3 Learning Iterative Reasoning through Energy Optimization ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> on test problems with either the same or harder difficulty than training problems. Across all three tasks, we find that IREM outperforms baselines, with the underlying difference magnified on harder, out-of-distribution test problems. In particular, on the task of addition, we find that IREM is able to nearly perfectly solve the underlying task even on harder, out-of-distribution test problems. In contrast, all other evaluated iterative and feedforward baselines perform poorly effectively solve the underlying problem. A primary difficulty is that underlying input vectors, both of size 400, are large compared to the underlying hidden unit size of 512, thus requiring the networks to iteratively reason and execute the algorithmic computation on subsets of the input.</p>
</div>
<figure id="S4.T3" class="ltx_table">
<table id="S4.T3.1" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S4.T3.1.1.1" class="ltx_tr">
<th id="S4.T3.1.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.1.1.1.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Step Size</span></th>
<th id="S4.T3.1.1.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.1.1.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Same Difficulty</span></th>
<th id="S4.T3.1.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.1.1.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Harder Difficulty</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S4.T3.1.2.1" class="ltx_tr">
<th id="S4.T3.1.2.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.2.1.1.1" class="ltx_text" style="font-size:90%;">10</span></th>
<td id="S4.T3.1.2.1.2" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.2.1.2.1" class="ltx_text" style="font-size:90%;">0.0003</span></td>
<td id="S4.T3.1.2.1.3" class="ltx_td ltx_align_center ltx_border_t" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.2.1.3.1" class="ltx_text" style="font-size:90%;">0.0021</span></td>
</tr>
<tr id="S4.T3.1.3.2" class="ltx_tr">
<th id="S4.T3.1.3.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.3.2.1.1" class="ltx_text" style="font-size:90%;">30</span></th>
<td id="S4.T3.1.3.2.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.3.2.2.1" class="ltx_text" style="font-size:90%;">0.0003</span></td>
<td id="S4.T3.1.3.2.3" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.3.2.3.1" class="ltx_text" style="font-size:90%;">0.0020</span></td>
</tr>
<tr id="S4.T3.1.4.3" class="ltx_tr">
<th id="S4.T3.1.4.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.4.3.1.1" class="ltx_text" style="font-size:90%;">100</span></th>
<td id="S4.T3.1.4.3.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.4.3.2.1" class="ltx_text" style="font-size:90%;">0.0003</span></td>
<td id="S4.T3.1.4.3.3" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.4.3.3.1" class="ltx_text" style="font-size:90%;">0.0021</span></td>
</tr>
<tr id="S4.T3.1.5.4" class="ltx_tr">
<th id="S4.T3.1.5.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.5.4.1.1" class="ltx_text" style="font-size:90%;">300</span></th>
<td id="S4.T3.1.5.4.2" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.5.4.2.1" class="ltx_text" style="font-size:90%;">0.0004</span></td>
<td id="S4.T3.1.5.4.3" class="ltx_td ltx_align_center" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.5.4.3.1" class="ltx_text" style="font-size:90%;">0.0023</span></td>
</tr>
<tr id="S4.T3.1.6.5" class="ltx_tr">
<th id="S4.T3.1.6.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.6.5.1.1" class="ltx_text" style="font-size:90%;">1000</span></th>
<td id="S4.T3.1.6.5.2" class="ltx_td ltx_align_center ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.6.5.2.1" class="ltx_text" style="font-size:90%;">0.0004</span></td>
<td id="S4.T3.1.6.5.3" class="ltx_td ltx_align_center ltx_border_bb" style="padding-left:5.5pt;padding-right:5.5pt;"><span id="S4.T3.1.6.5.3.1" class="ltx_text" style="font-size:90%;">0.0025</span></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 3: </span><span id="S4.T3.5.1" class="ltx_text ltx_font_bold">Ablation Analysis of Step Size in IREM–</span> Analysis of training step size of IREM performance on the continuous addition reasoning task.</figcaption>
</figure>
<div id="S4.SS3.p4" class="ltx_para">
<p id="S4.SS3.p4.1" class="ltx_p"><span id="S4.SS3.p4.1.1" class="ltx_text ltx_font_bold">Qualitative Visualization.</span>  Next, we visualize the underlying iterative computation learned by IREM and baselines. In Figure <a href="#S4.F3" title="Figure 3 ‣ 4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>, we illustrate the prediction error of different methods as a function of the number of computation steps applied. As we increase the number of iterative computation steps, the underlying performance of IREM continues to improve. In particular, iterative computation helps more substantially on harder variants of the algorithmic problem (right column), such as addition. In contrast, several iterative methods show significant degradation of performance with increased iterative computation.</p>
</div>
<div id="S4.SS3.p5" class="ltx_para">
<p id="S4.SS3.p5.1" class="ltx_p">We further qualitatively visualize the underlying iterative computation learned by IREM. In Figure <a href="#S4.F4" title="Figure 4 ‣ 4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, we visualize the element-wise mean square error of the predicted solution as a function of the number of iterative reasoning steps applied on the addition task. We find that energy minimization gradually refines a predicted solution to the ground truth additive answer, with different elements of the solution exhibiting different convergence rates.</p>
</div>
<div id="S4.SS3.p6" class="ltx_para">
<p id="S4.SS3.p6.1" class="ltx_p"><span id="S4.SS3.p6.1.1" class="ltx_text ltx_font_bold">Energy Landscape.</span>  IREM parameterizes an energy landscape across all possible solutions for a given problem. Such an energy landscape enables us to assess the relative quality of solutions dependent on their associated energies, and further gives a natural objective to terminate iterative computation when an underlying local energy minimum is reached. In Figure <a href="#S4.F5" title="Figure 5 ‣ 4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, we visualize the predicted energy of different candidate solutions and their corresponding MSE distances from the ground truth answer. We find that across different continuous algorithmic tasks, the underlying energy value assigned to a candidate solution is well correlated with its distance from the ground truth answer, with low energy solutions close to the ground truth.</p>
</div>
<div id="S4.SS3.p7" class="ltx_para">
<p id="S4.SS3.p7.1" class="ltx_p"><span id="S4.SS3.p7.1.1" class="ltx_text ltx_font_bold">Sensitivity to Step Size.</span>  We assess the performance of IREM under different values of step size <math id="S4.SS3.p7.1.m1.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="S4.SS3.p7.1.m1.1a"><mi id="S4.SS3.p7.1.m1.1.1" xref="S4.SS3.p7.1.m1.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p7.1.m1.1b"><ci id="S4.SS3.p7.1.m1.1.1.cmml" xref="S4.SS3.p7.1.m1.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p7.1.m1.1c">\lambda</annotation></semantics></math> using during energy optimization at training. We consider the continuous addition task in Table <a href="#S4.T3" title="Table 3 ‣ 4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>. We find that the underlying performance is not sensitive to hyperparameter choice for step size, and utilize a fixed step size of 100 across our experiments.</p>
</div>
</section>
<section id="S4.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.4 </span>Recursive Algorithmic Computation</h3>

<div id="S4.SS4.p1" class="ltx_para">
<p id="S4.SS4.p1.1" class="ltx_p"><span id="S4.SS4.p1.1.1" class="ltx_text ltx_font_bold">Setup.</span>  We further evaluate the ability of algorithms represented by IREM to be recursively applied on inputs. Recursively nesting algorithms enable complex computations, but require learned networks to be robust to out-of-distribution outputs from prior algorithmic execution.</p>
</div>
<figure id="S4.T4" class="ltx_table">
<div id="S4.T4.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:217.4pt;height:113.4pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-12.1pt,6.3pt) scale(0.9,0.9) ;">
<table id="S4.T4.1.1" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S4.T4.1.1.1.1" class="ltx_tr">
<th id="S4.T4.1.1.1.1.1" class="ltx_td ltx_th ltx_th_row ltx_border_tt" style="padding-left:3.5pt;padding-right:3.5pt;"></th>
<th id="S4.T4.1.1.1.1.2" class="ltx_td ltx_nopad_r ltx_th ltx_th_column ltx_border_tt" style="padding-left:3.5pt;padding-right:3.5pt;"></th>
<th id="S4.T4.1.1.1.1.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center ltx_th ltx_th_column ltx_border_tt" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.1.1.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Composed Operations</span></th>
<td id="S4.T4.1.1.1.1.4" class="ltx_td ltx_border_tt" style="padding-left:3.5pt;padding-right:3.5pt;"></td>
</tr>
<tr id="S4.T4.1.1.2.2" class="ltx_tr">
<th id="S4.T4.1.1.2.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.2.2.1.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Method</span></th>
<th id="S4.T4.1.1.2.2.2" class="ltx_td ltx_nopad_r ltx_align_center ltx_th ltx_th_column ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.2.2.2.1" class="ltx_text" style="font-size:90%;">2</span></th>
<th id="S4.T4.1.1.2.2.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center ltx_th ltx_th_column ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.2.2.3.1" class="ltx_text" style="font-size:90%;">5</span></th>
<th id="S4.T4.1.1.2.2.4" class="ltx_td ltx_nopad_l ltx_align_center ltx_th ltx_th_column ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.2.2.4.1" class="ltx_text" style="font-size:90%;">10</span></th>
</tr>
<tr id="S4.T4.1.1.3.3" class="ltx_tr">
<th id="S4.T4.1.1.3.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.3.3.1.1" class="ltx_text" style="font-size:90%;">Feedforward</span></th>
<td id="S4.T4.1.1.3.3.2" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.3.3.2.1" class="ltx_text" style="font-size:90%;">0.0445</span></td>
<td id="S4.T4.1.1.3.3.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.3.3.3.1" class="ltx_text" style="font-size:90%;">0.2717</span></td>
<td id="S4.T4.1.1.3.3.4" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_t" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.3.3.4.1" class="ltx_text" style="font-size:90%;">0.8898</span></td>
</tr>
<tr id="S4.T4.1.1.4.4" class="ltx_tr">
<th id="S4.T4.1.1.4.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.4.4.1.1" class="ltx_text" style="font-size:90%;">Recurrent</span></th>
<td id="S4.T4.1.1.4.4.2" class="ltx_td ltx_nopad_r ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.4.4.2.1" class="ltx_text" style="font-size:90%;">2.1377</span></td>
<td id="S4.T4.1.1.4.4.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.4.4.3.1" class="ltx_text" style="font-size:90%;">3.1861</span></td>
<td id="S4.T4.1.1.4.4.4" class="ltx_td ltx_nopad_l ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.4.4.4.1" class="ltx_text" style="font-size:90%;">4.8706</span></td>
</tr>
<tr id="S4.T4.1.1.5.5" class="ltx_tr">
<th id="S4.T4.1.1.5.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.5.5.1.1" class="ltx_text" style="font-size:90%;">Programmatic</span></th>
<td id="S4.T4.1.1.5.5.2" class="ltx_td ltx_nopad_r ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.5.5.2.1" class="ltx_text" style="font-size:90%;">0.0203</span></td>
<td id="S4.T4.1.1.5.5.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.5.5.3.1" class="ltx_text" style="font-size:90%;">0.1068</span></td>
<td id="S4.T4.1.1.5.5.4" class="ltx_td ltx_nopad_l ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.5.5.4.1" class="ltx_text" style="font-size:90%;">0.4587</span></td>
</tr>
<tr id="S4.T4.1.1.6.6" class="ltx_tr">
<th id="S4.T4.1.1.6.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.6.6.1.1" class="ltx_text" style="font-size:90%;">Iterative Feedforward</span></th>
<td id="S4.T4.1.1.6.6.2" class="ltx_td ltx_nopad_r ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.6.6.2.1" class="ltx_text" style="font-size:90%;">0.0826</span></td>
<td id="S4.T4.1.1.6.6.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.6.6.3.1" class="ltx_text" style="font-size:90%;">0.5930</span></td>
<td id="S4.T4.1.1.6.6.4" class="ltx_td ltx_nopad_l ltx_align_center" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.6.6.4.1" class="ltx_text" style="font-size:90%;">3.6004</span></td>
</tr>
<tr id="S4.T4.1.1.7.7" class="ltx_tr">
<th id="S4.T4.1.1.7.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.7.7.1.1" class="ltx_text" style="font-size:90%;">IREM (Ours)</span></th>
<td id="S4.T4.1.1.7.7.2" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_bb" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.7.7.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0014</span></td>
<td id="S4.T4.1.1.7.7.3" class="ltx_td ltx_nopad_l ltx_nopad_r ltx_align_center ltx_border_bb" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.7.7.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0078</span></td>
<td id="S4.T4.1.1.7.7.4" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_bb" style="padding-left:3.5pt;padding-right:3.5pt;"><span id="S4.T4.1.1.7.7.4.1" class="ltx_text ltx_font_bold" style="font-size:90%;">0.0422</span></td>
</tr>
</tbody>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 4: </span><span id="S4.T4.5.1" class="ltx_text ltx_font_bold">Algorithmic Composition –</span> Test performance when composing multiple instances of the addition operation. Error is reported using element-wise mean square error. IREM is able to generalize well when composing algorithmic computation.</figcaption>
</figure>
<figure id="S4.F7" class="ltx_figure"><img src="/html/2206.15448/assets/x6.png" id="S4.F7.1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="368" height="281" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 7: </span><span id="S4.F7.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Error Composing Algorithmic Computation<span id="S4.F7.3.1.1" class="ltx_text ltx_font_medium"> – Plot of error of predictions as factor of the number of composed algorithmic operations. IREM exhibits lower error when nesting a series of algorithmic operations together.</span></span></figcaption>
</figure>
<div id="S4.SS4.p2" class="ltx_para">
<p id="S4.SS4.p2.3" class="ltx_p">We consider recursive applications of a learned algorithmic operator <math id="S4.SS4.p2.1.m1.1" class="ltx_Math" alttext="\text{Alg}_{\theta}(\cdot)" display="inline"><semantics id="S4.SS4.p2.1.m1.1a"><mrow id="S4.SS4.p2.1.m1.1.2" xref="S4.SS4.p2.1.m1.1.2.cmml"><msub id="S4.SS4.p2.1.m1.1.2.2" xref="S4.SS4.p2.1.m1.1.2.2.cmml"><mtext id="S4.SS4.p2.1.m1.1.2.2.2" xref="S4.SS4.p2.1.m1.1.2.2.2a.cmml">Alg</mtext><mi id="S4.SS4.p2.1.m1.1.2.2.3" xref="S4.SS4.p2.1.m1.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S4.SS4.p2.1.m1.1.2.1" xref="S4.SS4.p2.1.m1.1.2.1.cmml">​</mo><mrow id="S4.SS4.p2.1.m1.1.2.3.2" xref="S4.SS4.p2.1.m1.1.2.cmml"><mo stretchy="false" id="S4.SS4.p2.1.m1.1.2.3.2.1" xref="S4.SS4.p2.1.m1.1.2.cmml">(</mo><mo lspace="0em" rspace="0em" id="S4.SS4.p2.1.m1.1.1" xref="S4.SS4.p2.1.m1.1.1.cmml">⋅</mo><mo stretchy="false" id="S4.SS4.p2.1.m1.1.2.3.2.2" xref="S4.SS4.p2.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS4.p2.1.m1.1b"><apply id="S4.SS4.p2.1.m1.1.2.cmml" xref="S4.SS4.p2.1.m1.1.2"><times id="S4.SS4.p2.1.m1.1.2.1.cmml" xref="S4.SS4.p2.1.m1.1.2.1"></times><apply id="S4.SS4.p2.1.m1.1.2.2.cmml" xref="S4.SS4.p2.1.m1.1.2.2"><csymbol cd="ambiguous" id="S4.SS4.p2.1.m1.1.2.2.1.cmml" xref="S4.SS4.p2.1.m1.1.2.2">subscript</csymbol><ci id="S4.SS4.p2.1.m1.1.2.2.2a.cmml" xref="S4.SS4.p2.1.m1.1.2.2.2"><mtext id="S4.SS4.p2.1.m1.1.2.2.2.cmml" xref="S4.SS4.p2.1.m1.1.2.2.2">Alg</mtext></ci><ci id="S4.SS4.p2.1.m1.1.2.2.3.cmml" xref="S4.SS4.p2.1.m1.1.2.2.3">𝜃</ci></apply><ci id="S4.SS4.p2.1.m1.1.1.cmml" xref="S4.SS4.p2.1.m1.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS4.p2.1.m1.1c">\text{Alg}_{\theta}(\cdot)</annotation></semantics></math> representing addition as introduced in Section <a href="#S4.SS3" title="4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a>. We evaluate the element-wise mean square error of the predicted output <math id="S4.SS4.p2.2.m2.1" class="ltx_Math" alttext="\hat{y}" display="inline"><semantics id="S4.SS4.p2.2.m2.1a"><mover accent="true" id="S4.SS4.p2.2.m2.1.1" xref="S4.SS4.p2.2.m2.1.1.cmml"><mi id="S4.SS4.p2.2.m2.1.1.2" xref="S4.SS4.p2.2.m2.1.1.2.cmml">y</mi><mo id="S4.SS4.p2.2.m2.1.1.1" xref="S4.SS4.p2.2.m2.1.1.1.cmml">^</mo></mover><annotation-xml encoding="MathML-Content" id="S4.SS4.p2.2.m2.1b"><apply id="S4.SS4.p2.2.m2.1.1.cmml" xref="S4.SS4.p2.2.m2.1.1"><ci id="S4.SS4.p2.2.m2.1.1.1.cmml" xref="S4.SS4.p2.2.m2.1.1.1">^</ci><ci id="S4.SS4.p2.2.m2.1.1.2.cmml" xref="S4.SS4.p2.2.m2.1.1.2">𝑦</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS4.p2.2.m2.1c">\hat{y}</annotation></semantics></math> of recursively applying the learned algorithmic operator <math id="S4.SS4.p2.3.m3.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S4.SS4.p2.3.m3.1a"><mi id="S4.SS4.p2.3.m3.1.1" xref="S4.SS4.p2.3.m3.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S4.SS4.p2.3.m3.1b"><ci id="S4.SS4.p2.3.m3.1.1.cmml" xref="S4.SS4.p2.3.m3.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS4.p2.3.m3.1c">k</annotation></semantics></math> times</p>
<table id="S4.E7" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S4.E7.m1.1" class="ltx_Math" alttext="\hat{y}^{k}=\text{Alg}_{\theta}(\hat{y}^{k-1},y_{k})," display="block"><semantics id="S4.E7.m1.1a"><mrow id="S4.E7.m1.1.1.1" xref="S4.E7.m1.1.1.1.1.cmml"><mrow id="S4.E7.m1.1.1.1.1" xref="S4.E7.m1.1.1.1.1.cmml"><msup id="S4.E7.m1.1.1.1.1.4" xref="S4.E7.m1.1.1.1.1.4.cmml"><mover accent="true" id="S4.E7.m1.1.1.1.1.4.2" xref="S4.E7.m1.1.1.1.1.4.2.cmml"><mi id="S4.E7.m1.1.1.1.1.4.2.2" xref="S4.E7.m1.1.1.1.1.4.2.2.cmml">y</mi><mo id="S4.E7.m1.1.1.1.1.4.2.1" xref="S4.E7.m1.1.1.1.1.4.2.1.cmml">^</mo></mover><mi id="S4.E7.m1.1.1.1.1.4.3" xref="S4.E7.m1.1.1.1.1.4.3.cmml">k</mi></msup><mo id="S4.E7.m1.1.1.1.1.3" xref="S4.E7.m1.1.1.1.1.3.cmml">=</mo><mrow id="S4.E7.m1.1.1.1.1.2" xref="S4.E7.m1.1.1.1.1.2.cmml"><msub id="S4.E7.m1.1.1.1.1.2.4" xref="S4.E7.m1.1.1.1.1.2.4.cmml"><mtext id="S4.E7.m1.1.1.1.1.2.4.2" xref="S4.E7.m1.1.1.1.1.2.4.2a.cmml">Alg</mtext><mi id="S4.E7.m1.1.1.1.1.2.4.3" xref="S4.E7.m1.1.1.1.1.2.4.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="S4.E7.m1.1.1.1.1.2.3" xref="S4.E7.m1.1.1.1.1.2.3.cmml">​</mo><mrow id="S4.E7.m1.1.1.1.1.2.2.2" xref="S4.E7.m1.1.1.1.1.2.2.3.cmml"><mo stretchy="false" id="S4.E7.m1.1.1.1.1.2.2.2.3" xref="S4.E7.m1.1.1.1.1.2.2.3.cmml">(</mo><msup id="S4.E7.m1.1.1.1.1.1.1.1.1" xref="S4.E7.m1.1.1.1.1.1.1.1.1.cmml"><mover accent="true" id="S4.E7.m1.1.1.1.1.1.1.1.1.2" xref="S4.E7.m1.1.1.1.1.1.1.1.1.2.cmml"><mi id="S4.E7.m1.1.1.1.1.1.1.1.1.2.2" xref="S4.E7.m1.1.1.1.1.1.1.1.1.2.2.cmml">y</mi><mo id="S4.E7.m1.1.1.1.1.1.1.1.1.2.1" xref="S4.E7.m1.1.1.1.1.1.1.1.1.2.1.cmml">^</mo></mover><mrow id="S4.E7.m1.1.1.1.1.1.1.1.1.3" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.cmml"><mi id="S4.E7.m1.1.1.1.1.1.1.1.1.3.2" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.2.cmml">k</mi><mo id="S4.E7.m1.1.1.1.1.1.1.1.1.3.1" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.1.cmml">−</mo><mn id="S4.E7.m1.1.1.1.1.1.1.1.1.3.3" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.3.cmml">1</mn></mrow></msup><mo id="S4.E7.m1.1.1.1.1.2.2.2.4" xref="S4.E7.m1.1.1.1.1.2.2.3.cmml">,</mo><msub id="S4.E7.m1.1.1.1.1.2.2.2.2" xref="S4.E7.m1.1.1.1.1.2.2.2.2.cmml"><mi id="S4.E7.m1.1.1.1.1.2.2.2.2.2" xref="S4.E7.m1.1.1.1.1.2.2.2.2.2.cmml">y</mi><mi id="S4.E7.m1.1.1.1.1.2.2.2.2.3" xref="S4.E7.m1.1.1.1.1.2.2.2.2.3.cmml">k</mi></msub><mo stretchy="false" id="S4.E7.m1.1.1.1.1.2.2.2.5" xref="S4.E7.m1.1.1.1.1.2.2.3.cmml">)</mo></mrow></mrow></mrow><mo id="S4.E7.m1.1.1.1.2" xref="S4.E7.m1.1.1.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.E7.m1.1b"><apply id="S4.E7.m1.1.1.1.1.cmml" xref="S4.E7.m1.1.1.1"><eq id="S4.E7.m1.1.1.1.1.3.cmml" xref="S4.E7.m1.1.1.1.1.3"></eq><apply id="S4.E7.m1.1.1.1.1.4.cmml" xref="S4.E7.m1.1.1.1.1.4"><csymbol cd="ambiguous" id="S4.E7.m1.1.1.1.1.4.1.cmml" xref="S4.E7.m1.1.1.1.1.4">superscript</csymbol><apply id="S4.E7.m1.1.1.1.1.4.2.cmml" xref="S4.E7.m1.1.1.1.1.4.2"><ci id="S4.E7.m1.1.1.1.1.4.2.1.cmml" xref="S4.E7.m1.1.1.1.1.4.2.1">^</ci><ci id="S4.E7.m1.1.1.1.1.4.2.2.cmml" xref="S4.E7.m1.1.1.1.1.4.2.2">𝑦</ci></apply><ci id="S4.E7.m1.1.1.1.1.4.3.cmml" xref="S4.E7.m1.1.1.1.1.4.3">𝑘</ci></apply><apply id="S4.E7.m1.1.1.1.1.2.cmml" xref="S4.E7.m1.1.1.1.1.2"><times id="S4.E7.m1.1.1.1.1.2.3.cmml" xref="S4.E7.m1.1.1.1.1.2.3"></times><apply id="S4.E7.m1.1.1.1.1.2.4.cmml" xref="S4.E7.m1.1.1.1.1.2.4"><csymbol cd="ambiguous" id="S4.E7.m1.1.1.1.1.2.4.1.cmml" xref="S4.E7.m1.1.1.1.1.2.4">subscript</csymbol><ci id="S4.E7.m1.1.1.1.1.2.4.2a.cmml" xref="S4.E7.m1.1.1.1.1.2.4.2"><mtext id="S4.E7.m1.1.1.1.1.2.4.2.cmml" xref="S4.E7.m1.1.1.1.1.2.4.2">Alg</mtext></ci><ci id="S4.E7.m1.1.1.1.1.2.4.3.cmml" xref="S4.E7.m1.1.1.1.1.2.4.3">𝜃</ci></apply><interval closure="open" id="S4.E7.m1.1.1.1.1.2.2.3.cmml" xref="S4.E7.m1.1.1.1.1.2.2.2"><apply id="S4.E7.m1.1.1.1.1.1.1.1.1.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.E7.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1">superscript</csymbol><apply id="S4.E7.m1.1.1.1.1.1.1.1.1.2.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.2"><ci id="S4.E7.m1.1.1.1.1.1.1.1.1.2.1.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.2.1">^</ci><ci id="S4.E7.m1.1.1.1.1.1.1.1.1.2.2.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.2.2">𝑦</ci></apply><apply id="S4.E7.m1.1.1.1.1.1.1.1.1.3.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3"><minus id="S4.E7.m1.1.1.1.1.1.1.1.1.3.1.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.1"></minus><ci id="S4.E7.m1.1.1.1.1.1.1.1.1.3.2.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.2">𝑘</ci><cn type="integer" id="S4.E7.m1.1.1.1.1.1.1.1.1.3.3.cmml" xref="S4.E7.m1.1.1.1.1.1.1.1.1.3.3">1</cn></apply></apply><apply id="S4.E7.m1.1.1.1.1.2.2.2.2.cmml" xref="S4.E7.m1.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S4.E7.m1.1.1.1.1.2.2.2.2.1.cmml" xref="S4.E7.m1.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S4.E7.m1.1.1.1.1.2.2.2.2.2.cmml" xref="S4.E7.m1.1.1.1.1.2.2.2.2.2">𝑦</ci><ci id="S4.E7.m1.1.1.1.1.2.2.2.2.3.cmml" xref="S4.E7.m1.1.1.1.1.2.2.2.2.3">𝑘</ci></apply></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.E7.m1.1c">\hat{y}^{k}=\text{Alg}_{\theta}(\hat{y}^{k-1},y_{k}),</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(7)</span></td>
</tr></tbody>
</table>
<p id="S4.SS4.p2.5" class="ltx_p">with the corresponding ground truth solution being <math id="S4.SS4.p2.4.m1.1" class="ltx_Math" alttext="\sum_{k}y_{k}" display="inline"><semantics id="S4.SS4.p2.4.m1.1a"><mrow id="S4.SS4.p2.4.m1.1.1" xref="S4.SS4.p2.4.m1.1.1.cmml"><msub id="S4.SS4.p2.4.m1.1.1.1" xref="S4.SS4.p2.4.m1.1.1.1.cmml"><mo id="S4.SS4.p2.4.m1.1.1.1.2" xref="S4.SS4.p2.4.m1.1.1.1.2.cmml">∑</mo><mi id="S4.SS4.p2.4.m1.1.1.1.3" xref="S4.SS4.p2.4.m1.1.1.1.3.cmml">k</mi></msub><msub id="S4.SS4.p2.4.m1.1.1.2" xref="S4.SS4.p2.4.m1.1.1.2.cmml"><mi id="S4.SS4.p2.4.m1.1.1.2.2" xref="S4.SS4.p2.4.m1.1.1.2.2.cmml">y</mi><mi id="S4.SS4.p2.4.m1.1.1.2.3" xref="S4.SS4.p2.4.m1.1.1.2.3.cmml">k</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S4.SS4.p2.4.m1.1b"><apply id="S4.SS4.p2.4.m1.1.1.cmml" xref="S4.SS4.p2.4.m1.1.1"><apply id="S4.SS4.p2.4.m1.1.1.1.cmml" xref="S4.SS4.p2.4.m1.1.1.1"><csymbol cd="ambiguous" id="S4.SS4.p2.4.m1.1.1.1.1.cmml" xref="S4.SS4.p2.4.m1.1.1.1">subscript</csymbol><sum id="S4.SS4.p2.4.m1.1.1.1.2.cmml" xref="S4.SS4.p2.4.m1.1.1.1.2"></sum><ci id="S4.SS4.p2.4.m1.1.1.1.3.cmml" xref="S4.SS4.p2.4.m1.1.1.1.3">𝑘</ci></apply><apply id="S4.SS4.p2.4.m1.1.1.2.cmml" xref="S4.SS4.p2.4.m1.1.1.2"><csymbol cd="ambiguous" id="S4.SS4.p2.4.m1.1.1.2.1.cmml" xref="S4.SS4.p2.4.m1.1.1.2">subscript</csymbol><ci id="S4.SS4.p2.4.m1.1.1.2.2.cmml" xref="S4.SS4.p2.4.m1.1.1.2.2">𝑦</ci><ci id="S4.SS4.p2.4.m1.1.1.2.3.cmml" xref="S4.SS4.p2.4.m1.1.1.2.3">𝑘</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS4.p2.4.m1.1c">\sum_{k}y_{k}</annotation></semantics></math>, for different values <math id="S4.SS4.p2.5.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S4.SS4.p2.5.m2.1a"><mi id="S4.SS4.p2.5.m2.1.1" xref="S4.SS4.p2.5.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S4.SS4.p2.5.m2.1b"><ci id="S4.SS4.p2.5.m2.1.1.cmml" xref="S4.SS4.p2.5.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS4.p2.5.m2.1c">k</annotation></semantics></math> of recursive application.</p>
</div>
<div id="S4.SS4.p3" class="ltx_para">
<p id="S4.SS4.p3.1" class="ltx_p"><span id="S4.SS4.p3.1.1" class="ltx_text ltx_font_bold">Quantitative Results.</span>  We report the results of recursively applying each learned algorithmic operator between two to ten times in Table <a href="#S4.T4" title="Table 4 ‣ 4.4 Recursive Algorithmic Computation ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>. We find that IREM supports the most stable recursive application of algorithmic operators. IREM exhibits significantly lower error than all compared baseline, due to its ability to utilize iterative computation to deal with out-of-distribution inputs and to accurately compute intermediate algorithmic outputs.</p>
</div>
<div id="S4.SS4.p4" class="ltx_para">
<p id="S4.SS4.p4.1" class="ltx_p"><span id="S4.SS4.p4.1.1" class="ltx_text ltx_font_bold">Qualitative Results.</span>  We illustrate error as a factor of the number of applied algorithmic operations in Figure <a href="#S4.F7" title="Figure 7 ‣ 4.4 Recursive Algorithmic Computation ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>, and find that the error of predictions from IREM rises slowly in comparison to other baselines. We further visualize the nested algorithmic predictions from IREM. We illustrate the inferred array sum predicted by IREM when four separate inputs are summed in Figure <a href="#S4.F6" title="Figure 6 ‣ 4.3 Continuous Algorithmic Reasoning ‣ 4 Experiments ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>. As seen above, our approach enables us to closely approximate the addition of four input matrices (with the first nine entries shown).</p>
</div>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Conclusion and Limitations</h2>

<div id="S5.p1" class="ltx_para">
<p id="S5.p1.1" class="ltx_p">In this paper, we present IREM, a new approach towards iterative computation, by formulating it as an energy minimization process. We illustrate, on both continuous and graphical domains, how iterative computation utilizing IREM enables better algorithmic performance, as well as generalization to more complex instances of problems. We further illustrate how the underlying algorithmic computation learned by IREM may be nested to implement more complex algorithmic computations.</p>
</div>
<div id="S5.p2" class="ltx_para">
<p id="S5.p2.1" class="ltx_p">Iterative reasoning with IREM has several limitations. First, while IREM substantially outperforms existing iterative methods on tasks where output solutions have high dimensionality, limited gains are obtained when IREM is executed on problems with lower dimensionality solutions (such as parity prediction). Second, as training and inferring solutions with IREM relies on continuous gradient optimization, IREM struggles when output solutions have discrete values. An interesting line of future work would be to explore how discrete optimization could be integrated with training IREM to solve such discrete valued problems. Finally, since the training procedure of IREM requires backpropagation across gradient optimization steps, it is computationally expensive. An interesting line of future work could be exploring alternative ways to train an energy function for reasoning, such as utilizing gradient-free optimization.</p>
</div>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Acknowledgements</h2>

<div id="S6.p1" class="ltx_para">
<p id="S6.p1.1" class="ltx_p">We thank Ben Poole for feedback and helpful comments on a early version of the manuscript. Yilun Du is supported by a fellowship from the National Science Foundation.</p>
</div>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Amos &amp; Kolter (2017)</span>
<span class="ltx_bibblock">
Amos, B. and Kolter, J. Z.

</span>
<span class="ltx_bibblock">Optnet: Differentiable optimization as a layer in neural networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pp. 136–145. PMLR, 2017.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Andrychowicz et al. (2016)</span>
<span class="ltx_bibblock">
Andrychowicz, M., Denil, M., Gomez, S., Hoffman, M. W., Pfau, D., Schaul, T.,
Shillingford, B., and De Freitas, N.

</span>
<span class="ltx_bibblock">Learning to learn by gradient descent by gradient descent.

</span>
<span class="ltx_bibblock"><em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">Advances in neural information processing systems</em>, 29, 2016.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Arbel et al. (2020)</span>
<span class="ltx_bibblock">
Arbel, M., Zhou, L., and Gretton, A.

</span>
<span class="ltx_bibblock">Generalized energy based models.

</span>
<span class="ltx_bibblock"><em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv.org/abs/2003.05033</em>, 2020.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bai et al. (2019)</span>
<span class="ltx_bibblock">
Bai, S., Kolter, J. Z., and Koltun, V.

</span>
<span class="ltx_bibblock">Deep equilibrium models.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1909.01377</em>, 2019.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Banino et al. (2021)</span>
<span class="ltx_bibblock">
Banino, A., Balaguer, J., and Blundell, C.

</span>
<span class="ltx_bibblock">Pondernet: Learning to ponder.

</span>
<span class="ltx_bibblock"><em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2107.05407</em>, 2021.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bengio et al. (1995)</span>
<span class="ltx_bibblock">
Bengio, S., Bengio, Y., Cloutier, J., and Gecsei, J.

</span>
<span class="ltx_bibblock">On the optimization of a synaptic learning rule.

</span>
<span class="ltx_bibblock">In <em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">Preprints Conf. Optimality in Artificial and Biological
Neural Networks</em>, volume 2, 1995.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bolukbasi et al. (2017)</span>
<span class="ltx_bibblock">
Bolukbasi, T., Wang, J., Dekel, O., and Saligrama, V.

</span>
<span class="ltx_bibblock">Adaptive neural networks for efficient inference.

</span>
<span class="ltx_bibblock">In <em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pp. 527–536. PMLR, 2017.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Brockett (1991)</span>
<span class="ltx_bibblock">
Brockett, R. W.

</span>
<span class="ltx_bibblock">Dynamical systems that sort lists, diagonalize matrices, and solve
linear programming problems.

</span>
<span class="ltx_bibblock"><em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">Linear Algebra and its applications</em>, 146:79–91,
1991.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cai et al. (2017)</span>
<span class="ltx_bibblock">
Cai, J., Shin, R., and Song, D.

</span>
<span class="ltx_bibblock">Making neural programming architectures generalize via recursion.

</span>
<span class="ltx_bibblock"><em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1704.06611</em>, 2017.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2020)</span>
<span class="ltx_bibblock">
Chen, X., Dai, H., Li, Y., Gao, X., and Song, L.

</span>
<span class="ltx_bibblock">Learning to stop while learning to predict.

</span>
<span class="ltx_bibblock">In <em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pp. 1520–1530. PMLR, 2020.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chung et al. (2016)</span>
<span class="ltx_bibblock">
Chung, J., Ahn, S., and Bengio, Y.

</span>
<span class="ltx_bibblock">Hierarchical multiscale recurrent neural networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1609.01704</em>, 2016.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Djolonga &amp; Krause (2017)</span>
<span class="ltx_bibblock">
Djolonga, J. and Krause, A.

</span>
<span class="ltx_bibblock">Differentiable learning of submodular models.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>,
30:1013–1023, 2017.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Donti et al. (2017)</span>
<span class="ltx_bibblock">
Donti, P. L., Amos, B., and Kolter, J. Z.

</span>
<span class="ltx_bibblock">Task-based end-to-end model learning in stochastic optimization.

</span>
<span class="ltx_bibblock"><em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1703.04529</em>, 2017.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Du &amp; Mordatch (2019)</span>
<span class="ltx_bibblock">
Du, Y. and Mordatch, I.

</span>
<span class="ltx_bibblock">Implicit generation and generalization in energy-based models.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1903.08689</em>, 2019.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Du et al. (2020)</span>
<span class="ltx_bibblock">
Du, Y., Li, S., and Mordatch, I.

</span>
<span class="ltx_bibblock">Compositional visual generation with energy based models.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>,
33:6637–6647, 2020.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Du et al. (2021a)</span>
<span class="ltx_bibblock">
Du, Y., Li, S., Sharma, Y., Tenenbaum, B. J., and Mordatch, I.

</span>
<span class="ltx_bibblock">Unsupervised learning of compositional energy concepts.

</span>
<span class="ltx_bibblock">In <em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>,
2021a.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Du et al. (2021b)</span>
<span class="ltx_bibblock">
Du, Y., Li, S., Tenenbaum, B. J., and Mordatch, I.

</span>
<span class="ltx_bibblock">Improved contrastive divergence training of energy based models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 38th International Conference on Machine
Learning (ICML-21)</em>, 2021b.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Finn et al. (2017)</span>
<span class="ltx_bibblock">
Finn, C., Abbeel, P., and Levine, S.

</span>
<span class="ltx_bibblock">Model-agnostic meta-learning for fast adaptation of deep networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pp. 1126–1135. PMLR, 2017.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Grathwohl et al. (2020)</span>
<span class="ltx_bibblock">
Grathwohl, W., Wang, K.-C., Jacobsen, J.-H., Duvenaud, D., and Zemel, R.

</span>
<span class="ltx_bibblock">Cutting out the middle-man: Training and evaluating energy-based
models without sampling.

</span>
<span class="ltx_bibblock"><em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2002.05616</em>, 2020.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Graves (2016)</span>
<span class="ltx_bibblock">
Graves, A.

</span>
<span class="ltx_bibblock">Adaptive computation time for recurrent neural networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1603.08983</em>, 2016.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Graves et al. (2014)</span>
<span class="ltx_bibblock">
Graves, A., Wayne, G., and Danihelka, I.

</span>
<span class="ltx_bibblock">Neural turing machines.

</span>
<span class="ltx_bibblock"><em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1410.5401</em>, 2014.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hochreiter et al. (2001)</span>
<span class="ltx_bibblock">
Hochreiter, S., Younger, A. S., and Conwell, P. R.

</span>
<span class="ltx_bibblock">Learning to learn using gradient descent.

</span>
<span class="ltx_bibblock">In <em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">International Conference on Artificial Neural Networks</em>,
pp.  87–94. Springer, 2001.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hu et al. (2019)</span>
<span class="ltx_bibblock">
Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., and Leskovec, J.

</span>
<span class="ltx_bibblock">Strategies for pre-training graph neural networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1905.12265</em>, 2019.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Impagliazzo &amp; Paturi (1999)</span>
<span class="ltx_bibblock">
Impagliazzo, R. and Paturi, R.

</span>
<span class="ltx_bibblock">Complexity of k-sat.

</span>
<span class="ltx_bibblock"><em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">Proceedings. Fourteenth Annual IEEE Conference on Computational
Complexity (Formerly: Structure in Complexity Theory Conference)
(Cat.No.99CB36317)</em>, Jun 1999.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1109/ccc.1999.766282</span>.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kahneman (2011)</span>
<span class="ltx_bibblock">
Kahneman, D.

</span>
<span class="ltx_bibblock"><em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">Thinking, fast and slow</em>.

</span>
<span class="ltx_bibblock">Macmillan, 2011.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kaiser &amp; Sutskever (2015)</span>
<span class="ltx_bibblock">
Kaiser, Ł. and Sutskever, I.

</span>
<span class="ltx_bibblock">Neural gpus learn algorithms.

</span>
<span class="ltx_bibblock"><em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1511.08228</em>, 2015.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kemp &amp; Tenenbaum (2008)</span>
<span class="ltx_bibblock">
Kemp, C. and Tenenbaum, J. B.

</span>
<span class="ltx_bibblock">The discovery of structural form.

</span>
<span class="ltx_bibblock"><em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">Proceedings of the National Academy of Sciences</em>, 105(31):10687–10692, 2008.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lampinen et al. (2017)</span>
<span class="ltx_bibblock">
Lampinen, A. K., Hsu, S., and McClelland, J. L.

</span>
<span class="ltx_bibblock">Analogies emerge from learning dyamics in neural networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">CogSci</em>, 2017.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">LeCun et al. (2006)</span>
<span class="ltx_bibblock">
LeCun, Y., Chopra, S., and Hadsell, R.

</span>
<span class="ltx_bibblock">A tutorial on energy-based learning.

</span>
<span class="ltx_bibblock">2006.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li &amp; Malik (2016)</span>
<span class="ltx_bibblock">
Li, K. and Malik, J.

</span>
<span class="ltx_bibblock">Learning to optimize.

</span>
<span class="ltx_bibblock"><em id="bib.bib30.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1606.01885</em>, 2016.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2020)</span>
<span class="ltx_bibblock">
Li, S., Du, Y., van de Ven, G. M., and Mordatch, I.

</span>
<span class="ltx_bibblock">Energy-based models for continual learning.

</span>
<span class="ltx_bibblock"><em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2011.12216</em>, 2020.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Nijkamp et al. (2019)</span>
<span class="ltx_bibblock">
Nijkamp, E., Hill, M., Han, T., Zhu, S.-C., and Wu, Y. N.

</span>
<span class="ltx_bibblock">On the anatomy of mcmc-based maximum likelihood learning of
energy-based models.

</span>
<span class="ltx_bibblock"><em id="bib.bib32.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1903.12370</em>, 2019.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Radford et al. (2019)</span>
<span class="ltx_bibblock">
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.

</span>
<span class="ltx_bibblock">Language models are unsupervised multitask learners.

</span>
<span class="ltx_bibblock"><em id="bib.bib33.1.1" class="ltx_emph ltx_font_italic">OpenAI blog</em>, 1(8):9, 2019.

</span>
</li>
<li id="bib.bib34" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ravi &amp; Larochelle (2016)</span>
<span class="ltx_bibblock">
Ravi, S. and Larochelle, H.

</span>
<span class="ltx_bibblock">Optimization as a model for few-shot learning.

</span>
<span class="ltx_bibblock">2016.

</span>
</li>
<li id="bib.bib35" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Reed &amp; De Freitas (2015)</span>
<span class="ltx_bibblock">
Reed, S. and De Freitas, N.

</span>
<span class="ltx_bibblock">Neural programmer-interpreters.

</span>
<span class="ltx_bibblock"><em id="bib.bib35.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1511.06279</em>, 2015.

</span>
</li>
<li id="bib.bib36" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Rubanova et al. (2021)</span>
<span class="ltx_bibblock">
Rubanova, Y., Sanchez-Gonzalez, A., Pfaff, T., and Battaglia, P.

</span>
<span class="ltx_bibblock">Constraint-based graph network simulator.

</span>
<span class="ltx_bibblock"><em id="bib.bib36.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2112.09161</em>, 2021.

</span>
</li>
<li id="bib.bib37" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Schmidhuber (1992)</span>
<span class="ltx_bibblock">
Schmidhuber, J.

</span>
<span class="ltx_bibblock">Learning to control fast-weight memories: An alternative to dynamic
recurrent networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib37.1.1" class="ltx_emph ltx_font_italic">Neural Computation</em>, 4(1):131–139, 1992.

</span>
</li>
<li id="bib.bib38" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Schwarzschild et al. (2021)</span>
<span class="ltx_bibblock">
Schwarzschild, A., Borgnia, E., Gupta, A., Huang, F., Vishkin, U., Goldblum,
M., and Goldstein, T.

</span>
<span class="ltx_bibblock">Can you learn an algorithm? generalizing from easy to hard problems
with recurrent networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib38.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2106.04537</em>, 2021.

</span>
</li>
<li id="bib.bib39" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Sohl-Dickstein et al. (2015)</span>
<span class="ltx_bibblock">
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., and Ganguli, S.

</span>
<span class="ltx_bibblock">Deep unsupervised learning using nonequilibrium thermodynamics.

</span>
<span class="ltx_bibblock"><em id="bib.bib39.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:1503.03585</em>, 2015.

</span>
</li>
<li id="bib.bib40" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wilder et al. (2019)</span>
<span class="ltx_bibblock">
Wilder, B., Dilkina, B., and Tambe, M.

</span>
<span class="ltx_bibblock">Melding the data-decisions pipeline: Decision-focused learning for
combinatorial optimization.

</span>
<span class="ltx_bibblock">In <em id="bib.bib40.1.1" class="ltx_emph ltx_font_italic">Proceedings of the AAAI Conference on Artificial
Intelligence</em>, volume 33, pp.  1658–1665, 2019.

</span>
</li>
<li id="bib.bib41" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xiao et al. (2020)</span>
<span class="ltx_bibblock">
Xiao, Z., Kreis, K., Kautz, J., and Vahdat, A.

</span>
<span class="ltx_bibblock">Vaebm: A symbiosis between variational autoencoders and energy-based
models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib41.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2020.

</span>
</li>
<li id="bib.bib42" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang &amp; Chen (2018)</span>
<span class="ltx_bibblock">
Zhang, M. and Chen, Y.

</span>
<span class="ltx_bibblock">Link prediction based on graph neural networks.

</span>
<span class="ltx_bibblock"><em id="bib.bib42.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>,
31:5165–5175, 2018.

</span>
</li>
</ul>
</section>
<div class="ltx_pagination ltx_role_newpage"></div>
<div id="p3" class="ltx_para">
<p id="p3.1" class="ltx_p"><span id="p3.1.1" class="ltx_text ltx_font_bold" style="font-size:207%;">Appendix</span></p>
</div>
<div id="p4" class="ltx_para">
<p id="p4.1" class="ltx_p">In this appendix we provide additional evaluation and details on IREM. First, we illustrate how IREM may be applied to an existing image iterative reasoning task in Appendix <a href="#A1" title="Appendix A Image Denoising ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">A</span></a>. Next we discuss how we may utilize optimization to execute IREM with an external scratchpad in Appendix <a href="#A2" title="Appendix B Iterative Reasoning with a Scratchpad ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">B</span></a>. We further discuss additional experimental details on our evaluated algorithmic tasks in Appendix <a href="#A3" title="Appendix C Experimental Details ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">C</span></a>. Finally, we discuss individual model architectures used in Appendix <a href="#A4" title="Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">D</span></a></p>
</div>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Image Denoising</h2>

<div id="A1.p1" class="ltx_para">
<p id="A1.p1.1" class="ltx_p"><span id="A1.p1.1.1" class="ltx_text ltx_font_bold">Setup</span>  We compare IREM with existing approaches on an existing image based iterative reasoning benchmark from <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib10" title="" class="ltx_ref">2020</a>)</cite>. The benchmark task is to denoise images with various levels of Gaussian noise corruption added. Harder denoising tasks are constructed at test time by adding larger amounts of noise to input images. We directly compare with baselines and numbers for UNLNet, DnCNN and DNCNN-stop from <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib10" title="" class="ltx_ref">2020</a>)</cite> and utilize the authors provided training code to train IREM with five steps of iterative reasoning.</p>
</div>
<div id="A1.p2" class="ltx_para">
<p id="A1.p2.2" class="ltx_p"><span id="A1.p2.2.1" class="ltx_text ltx_font_bold">Quantitative Results</span>  We quantitatively evaluate the performance of IREM and baselines in terms of PSNR (numbers for baselines directly from  <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib10" title="" class="ltx_ref">2020</a>)</cite>) in Table <a href="#A2.T5" title="Table 5 ‣ Appendix B Iterative Reasoning with a Scratchpad ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>. While we found that IREM performed similarly to existing approaches when the test noise corruption is similar to that of training, IREM significantly outperformed the compared baselines when evaluated on harder images at test time which exhibited substantially larger amounts of noise corruption then seen during training. We found that such performance gain was due to iterative computation executed by IREM. While we trained IREM with 5 steps of reasoning, at test time we found that running up to 30 steps of reasoning at noise level <math id="A1.p2.1.m1.1" class="ltx_Math" alttext="\sigma=65" display="inline"><semantics id="A1.p2.1.m1.1a"><mrow id="A1.p2.1.m1.1.1" xref="A1.p2.1.m1.1.1.cmml"><mi id="A1.p2.1.m1.1.1.2" xref="A1.p2.1.m1.1.1.2.cmml">σ</mi><mo id="A1.p2.1.m1.1.1.1" xref="A1.p2.1.m1.1.1.1.cmml">=</mo><mn id="A1.p2.1.m1.1.1.3" xref="A1.p2.1.m1.1.1.3.cmml">65</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.p2.1.m1.1b"><apply id="A1.p2.1.m1.1.1.cmml" xref="A1.p2.1.m1.1.1"><eq id="A1.p2.1.m1.1.1.1.cmml" xref="A1.p2.1.m1.1.1.1"></eq><ci id="A1.p2.1.m1.1.1.2.cmml" xref="A1.p2.1.m1.1.1.2">𝜎</ci><cn type="integer" id="A1.p2.1.m1.1.1.3.cmml" xref="A1.p2.1.m1.1.1.3">65</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p2.1.m1.1c">\sigma=65</annotation></semantics></math> improved performance, while running up to 100 steps of reasoning at noise level <math id="A1.p2.2.m2.1" class="ltx_Math" alttext="\sigma=75" display="inline"><semantics id="A1.p2.2.m2.1a"><mrow id="A1.p2.2.m2.1.1" xref="A1.p2.2.m2.1.1.cmml"><mi id="A1.p2.2.m2.1.1.2" xref="A1.p2.2.m2.1.1.2.cmml">σ</mi><mo id="A1.p2.2.m2.1.1.1" xref="A1.p2.2.m2.1.1.1.cmml">=</mo><mn id="A1.p2.2.m2.1.1.3" xref="A1.p2.2.m2.1.1.3.cmml">75</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.p2.2.m2.1b"><apply id="A1.p2.2.m2.1.1.cmml" xref="A1.p2.2.m2.1.1"><eq id="A1.p2.2.m2.1.1.1.cmml" xref="A1.p2.2.m2.1.1.1"></eq><ci id="A1.p2.2.m2.1.1.2.cmml" xref="A1.p2.2.m2.1.1.2">𝜎</ci><cn type="integer" id="A1.p2.2.m2.1.1.3.cmml" xref="A1.p2.2.m2.1.1.3">75</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p2.2.m2.1c">\sigma=75</annotation></semantics></math> further improved performance.</p>
</div>
<figure id="A1.F8" class="ltx_figure"><img src="/html/2206.15448/assets/x7.png" id="A1.F8.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="438" height="273" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 8: </span><span id="A1.F8.2.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Qualitative Illustration of Iterative Image Denoising<span id="A1.F8.2.1.1" class="ltx_text ltx_font_medium"> – Illustration of iterative denoising using IREM. Images continue to be cleaner after up to 100 iterative steps of computation.</span></span></figcaption>
</figure>
<div id="A1.p3" class="ltx_para">
<p id="A1.p3.1" class="ltx_p"><span id="A1.p3.1.1" class="ltx_text ltx_font_bold">Qualitative Results</span>  We qualitatively illustrate IREM being applied to an image corrupted with an unseen noise level <math id="A1.p3.1.m1.1" class="ltx_Math" alttext="\sigma=75" display="inline"><semantics id="A1.p3.1.m1.1a"><mrow id="A1.p3.1.m1.1.1" xref="A1.p3.1.m1.1.1.cmml"><mi id="A1.p3.1.m1.1.1.2" xref="A1.p3.1.m1.1.1.2.cmml">σ</mi><mo id="A1.p3.1.m1.1.1.1" xref="A1.p3.1.m1.1.1.1.cmml">=</mo><mn id="A1.p3.1.m1.1.1.3" xref="A1.p3.1.m1.1.1.3.cmml">75</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.p3.1.m1.1b"><apply id="A1.p3.1.m1.1.1.cmml" xref="A1.p3.1.m1.1.1"><eq id="A1.p3.1.m1.1.1.1.cmml" xref="A1.p3.1.m1.1.1.1"></eq><ci id="A1.p3.1.m1.1.1.2.cmml" xref="A1.p3.1.m1.1.1.2">𝜎</ci><cn type="integer" id="A1.p3.1.m1.1.1.3.cmml" xref="A1.p3.1.m1.1.1.3">75</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p3.1.m1.1c">\sigma=75</annotation></semantics></math> (significantly larger than what is seen during training) in Figure <a href="#A1.F8" title="Figure 8 ‣ Appendix A Image Denoising ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>. As seen in Figure <a href="#A1.F8" title="Figure 8 ‣ Appendix A Image Denoising ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>, images continue to become clearer, even after 50 steps of iterative computation using IREM (with dress in the top row and floor in the bottom row becoming clearer).</p>
</div>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Iterative Reasoning with a Scratchpad</h2>

<div id="A2.p1" class="ltx_para">
<p id="A2.p1.5" class="ltx_p">We next discuss how we may incorporate an external computational scratchpad when executing an iterative computation with IREM. To enable the processing of an external scratchpad, <math id="A2.p1.1.m1.1" class="ltx_Math" alttext="{\bm{z}}\in\mathbb{R}^{D}" display="inline"><semantics id="A2.p1.1.m1.1a"><mrow id="A2.p1.1.m1.1.1" xref="A2.p1.1.m1.1.1.cmml"><mi id="A2.p1.1.m1.1.1.2" xref="A2.p1.1.m1.1.1.2.cmml">𝒛</mi><mo id="A2.p1.1.m1.1.1.1" xref="A2.p1.1.m1.1.1.1.cmml">∈</mo><msup id="A2.p1.1.m1.1.1.3" xref="A2.p1.1.m1.1.1.3.cmml"><mi id="A2.p1.1.m1.1.1.3.2" xref="A2.p1.1.m1.1.1.3.2.cmml">ℝ</mi><mi id="A2.p1.1.m1.1.1.3.3" xref="A2.p1.1.m1.1.1.3.3.cmml">D</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="A2.p1.1.m1.1b"><apply id="A2.p1.1.m1.1.1.cmml" xref="A2.p1.1.m1.1.1"><in id="A2.p1.1.m1.1.1.1.cmml" xref="A2.p1.1.m1.1.1.1"></in><ci id="A2.p1.1.m1.1.1.2.cmml" xref="A2.p1.1.m1.1.1.2">𝒛</ci><apply id="A2.p1.1.m1.1.1.3.cmml" xref="A2.p1.1.m1.1.1.3"><csymbol cd="ambiguous" id="A2.p1.1.m1.1.1.3.1.cmml" xref="A2.p1.1.m1.1.1.3">superscript</csymbol><ci id="A2.p1.1.m1.1.1.3.2.cmml" xref="A2.p1.1.m1.1.1.3.2">ℝ</ci><ci id="A2.p1.1.m1.1.1.3.3.cmml" xref="A2.p1.1.m1.1.1.3.3">𝐷</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.p1.1.m1.1c">{\bm{z}}\in\mathbb{R}^{D}</annotation></semantics></math>, we construct a EBM, <math id="A2.p1.2.m2.3" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}},{\bm{z}}):\mathbb{R}^{N}\times\mathbb{R}^{M}\times\mathbb{R}^{D}\rightarrow\mathbb{R}" display="inline"><semantics id="A2.p1.2.m2.3a"><mrow id="A2.p1.2.m2.3.4" xref="A2.p1.2.m2.3.4.cmml"><mrow id="A2.p1.2.m2.3.4.2" xref="A2.p1.2.m2.3.4.2.cmml"><msub id="A2.p1.2.m2.3.4.2.2" xref="A2.p1.2.m2.3.4.2.2.cmml"><mi id="A2.p1.2.m2.3.4.2.2.2" xref="A2.p1.2.m2.3.4.2.2.2.cmml">E</mi><mi id="A2.p1.2.m2.3.4.2.2.3" xref="A2.p1.2.m2.3.4.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="A2.p1.2.m2.3.4.2.1" xref="A2.p1.2.m2.3.4.2.1.cmml">​</mo><mrow id="A2.p1.2.m2.3.4.2.3.2" xref="A2.p1.2.m2.3.4.2.3.1.cmml"><mo stretchy="false" id="A2.p1.2.m2.3.4.2.3.2.1" xref="A2.p1.2.m2.3.4.2.3.1.cmml">(</mo><mi id="A2.p1.2.m2.1.1" xref="A2.p1.2.m2.1.1.cmml">𝒙</mi><mo id="A2.p1.2.m2.3.4.2.3.2.2" xref="A2.p1.2.m2.3.4.2.3.1.cmml">,</mo><mi id="A2.p1.2.m2.2.2" xref="A2.p1.2.m2.2.2.cmml">𝒚</mi><mo id="A2.p1.2.m2.3.4.2.3.2.3" xref="A2.p1.2.m2.3.4.2.3.1.cmml">,</mo><mi id="A2.p1.2.m2.3.3" xref="A2.p1.2.m2.3.3.cmml">𝒛</mi><mo rspace="0.278em" stretchy="false" id="A2.p1.2.m2.3.4.2.3.2.4" xref="A2.p1.2.m2.3.4.2.3.1.cmml">)</mo></mrow></mrow><mo rspace="0.278em" id="A2.p1.2.m2.3.4.1" xref="A2.p1.2.m2.3.4.1.cmml">:</mo><mrow id="A2.p1.2.m2.3.4.3" xref="A2.p1.2.m2.3.4.3.cmml"><mrow id="A2.p1.2.m2.3.4.3.2" xref="A2.p1.2.m2.3.4.3.2.cmml"><msup id="A2.p1.2.m2.3.4.3.2.2" xref="A2.p1.2.m2.3.4.3.2.2.cmml"><mi id="A2.p1.2.m2.3.4.3.2.2.2" xref="A2.p1.2.m2.3.4.3.2.2.2.cmml">ℝ</mi><mi id="A2.p1.2.m2.3.4.3.2.2.3" xref="A2.p1.2.m2.3.4.3.2.2.3.cmml">N</mi></msup><mo lspace="0.222em" rspace="0.222em" id="A2.p1.2.m2.3.4.3.2.1" xref="A2.p1.2.m2.3.4.3.2.1.cmml">×</mo><msup id="A2.p1.2.m2.3.4.3.2.3" xref="A2.p1.2.m2.3.4.3.2.3.cmml"><mi id="A2.p1.2.m2.3.4.3.2.3.2" xref="A2.p1.2.m2.3.4.3.2.3.2.cmml">ℝ</mi><mi id="A2.p1.2.m2.3.4.3.2.3.3" xref="A2.p1.2.m2.3.4.3.2.3.3.cmml">M</mi></msup><mo lspace="0.222em" rspace="0.222em" id="A2.p1.2.m2.3.4.3.2.1a" xref="A2.p1.2.m2.3.4.3.2.1.cmml">×</mo><msup id="A2.p1.2.m2.3.4.3.2.4" xref="A2.p1.2.m2.3.4.3.2.4.cmml"><mi id="A2.p1.2.m2.3.4.3.2.4.2" xref="A2.p1.2.m2.3.4.3.2.4.2.cmml">ℝ</mi><mi id="A2.p1.2.m2.3.4.3.2.4.3" xref="A2.p1.2.m2.3.4.3.2.4.3.cmml">D</mi></msup></mrow><mo stretchy="false" id="A2.p1.2.m2.3.4.3.1" xref="A2.p1.2.m2.3.4.3.1.cmml">→</mo><mi id="A2.p1.2.m2.3.4.3.3" xref="A2.p1.2.m2.3.4.3.3.cmml">ℝ</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A2.p1.2.m2.3b"><apply id="A2.p1.2.m2.3.4.cmml" xref="A2.p1.2.m2.3.4"><ci id="A2.p1.2.m2.3.4.1.cmml" xref="A2.p1.2.m2.3.4.1">:</ci><apply id="A2.p1.2.m2.3.4.2.cmml" xref="A2.p1.2.m2.3.4.2"><times id="A2.p1.2.m2.3.4.2.1.cmml" xref="A2.p1.2.m2.3.4.2.1"></times><apply id="A2.p1.2.m2.3.4.2.2.cmml" xref="A2.p1.2.m2.3.4.2.2"><csymbol cd="ambiguous" id="A2.p1.2.m2.3.4.2.2.1.cmml" xref="A2.p1.2.m2.3.4.2.2">subscript</csymbol><ci id="A2.p1.2.m2.3.4.2.2.2.cmml" xref="A2.p1.2.m2.3.4.2.2.2">𝐸</ci><ci id="A2.p1.2.m2.3.4.2.2.3.cmml" xref="A2.p1.2.m2.3.4.2.2.3">𝜃</ci></apply><vector id="A2.p1.2.m2.3.4.2.3.1.cmml" xref="A2.p1.2.m2.3.4.2.3.2"><ci id="A2.p1.2.m2.1.1.cmml" xref="A2.p1.2.m2.1.1">𝒙</ci><ci id="A2.p1.2.m2.2.2.cmml" xref="A2.p1.2.m2.2.2">𝒚</ci><ci id="A2.p1.2.m2.3.3.cmml" xref="A2.p1.2.m2.3.3">𝒛</ci></vector></apply><apply id="A2.p1.2.m2.3.4.3.cmml" xref="A2.p1.2.m2.3.4.3"><ci id="A2.p1.2.m2.3.4.3.1.cmml" xref="A2.p1.2.m2.3.4.3.1">→</ci><apply id="A2.p1.2.m2.3.4.3.2.cmml" xref="A2.p1.2.m2.3.4.3.2"><times id="A2.p1.2.m2.3.4.3.2.1.cmml" xref="A2.p1.2.m2.3.4.3.2.1"></times><apply id="A2.p1.2.m2.3.4.3.2.2.cmml" xref="A2.p1.2.m2.3.4.3.2.2"><csymbol cd="ambiguous" id="A2.p1.2.m2.3.4.3.2.2.1.cmml" xref="A2.p1.2.m2.3.4.3.2.2">superscript</csymbol><ci id="A2.p1.2.m2.3.4.3.2.2.2.cmml" xref="A2.p1.2.m2.3.4.3.2.2.2">ℝ</ci><ci id="A2.p1.2.m2.3.4.3.2.2.3.cmml" xref="A2.p1.2.m2.3.4.3.2.2.3">𝑁</ci></apply><apply id="A2.p1.2.m2.3.4.3.2.3.cmml" xref="A2.p1.2.m2.3.4.3.2.3"><csymbol cd="ambiguous" id="A2.p1.2.m2.3.4.3.2.3.1.cmml" xref="A2.p1.2.m2.3.4.3.2.3">superscript</csymbol><ci id="A2.p1.2.m2.3.4.3.2.3.2.cmml" xref="A2.p1.2.m2.3.4.3.2.3.2">ℝ</ci><ci id="A2.p1.2.m2.3.4.3.2.3.3.cmml" xref="A2.p1.2.m2.3.4.3.2.3.3">𝑀</ci></apply><apply id="A2.p1.2.m2.3.4.3.2.4.cmml" xref="A2.p1.2.m2.3.4.3.2.4"><csymbol cd="ambiguous" id="A2.p1.2.m2.3.4.3.2.4.1.cmml" xref="A2.p1.2.m2.3.4.3.2.4">superscript</csymbol><ci id="A2.p1.2.m2.3.4.3.2.4.2.cmml" xref="A2.p1.2.m2.3.4.3.2.4.2">ℝ</ci><ci id="A2.p1.2.m2.3.4.3.2.4.3.cmml" xref="A2.p1.2.m2.3.4.3.2.4.3">𝐷</ci></apply></apply><ci id="A2.p1.2.m2.3.4.3.3.cmml" xref="A2.p1.2.m2.3.4.3.3">ℝ</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.p1.2.m2.3c">E_{\theta}({\bm{x}},{\bm{y}},{\bm{z}}):\mathbb{R}^{N}\times\mathbb{R}^{M}\times\mathbb{R}^{D}\rightarrow\mathbb{R}</annotation></semantics></math>, which takes as input an input problem <math id="A2.p1.3.m3.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="A2.p1.3.m3.1a"><mi id="A2.p1.3.m3.1.1" xref="A2.p1.3.m3.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="A2.p1.3.m3.1b"><ci id="A2.p1.3.m3.1.1.cmml" xref="A2.p1.3.m3.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p1.3.m3.1c">{\bm{x}}</annotation></semantics></math>, a candidate solution <math id="A2.p1.4.m4.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="A2.p1.4.m4.1a"><mi id="A2.p1.4.m4.1.1" xref="A2.p1.4.m4.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="A2.p1.4.m4.1b"><ci id="A2.p1.4.m4.1.1.cmml" xref="A2.p1.4.m4.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p1.4.m4.1c">{\bm{y}}</annotation></semantics></math>, and a scratchpad state <math id="A2.p1.5.m5.1" class="ltx_Math" alttext="{\bm{z}}" display="inline"><semantics id="A2.p1.5.m5.1a"><mi id="A2.p1.5.m5.1.1" xref="A2.p1.5.m5.1.1.cmml">𝒛</mi><annotation-xml encoding="MathML-Content" id="A2.p1.5.m5.1b"><ci id="A2.p1.5.m5.1.1.cmml" xref="A2.p1.5.m5.1.1">𝒛</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p1.5.m5.1c">{\bm{z}}</annotation></semantics></math>.</p>
</div>
<figure id="A2.T5" class="ltx_table">
<div id="A2.T5.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:216.4pt;height:72pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-27.1pt,9.0pt) scale(0.8,0.8) ;">
<table id="A2.T5.1.1" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A2.T5.1.1.1" class="ltx_tr">
<th id="A2.T5.1.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_column ltx_th_row ltx_border_tt"><math id="A2.T5.1.1.1.1.m1.1" class="ltx_Math" alttext="\bm{\sigma}" display="inline"><semantics id="A2.T5.1.1.1.1.m1.1a"><mi id="A2.T5.1.1.1.1.m1.1.1" xref="A2.T5.1.1.1.1.m1.1.1.cmml">𝝈</mi><annotation-xml encoding="MathML-Content" id="A2.T5.1.1.1.1.m1.1b"><ci id="A2.T5.1.1.1.1.m1.1.1.cmml" xref="A2.T5.1.1.1.1.m1.1.1">𝝈</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.T5.1.1.1.1.m1.1c">\bm{\sigma}</annotation></semantics></math></th>
<th id="A2.T5.1.1.1.2" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A2.T5.1.1.1.2.1" class="ltx_text ltx_font_bold">UNLNet</span></th>
<th id="A2.T5.1.1.1.3" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A2.T5.1.1.1.3.1" class="ltx_text ltx_font_bold">DnCNN</span></th>
<th id="A2.T5.1.1.1.4" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A2.T5.1.1.1.4.1" class="ltx_text ltx_font_bold">DnCNN-stop</span></th>
<th id="A2.T5.1.1.1.5" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A2.T5.1.1.1.5.1" class="ltx_text ltx_font_bold">IREM (Ours)</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A2.T5.1.1.2.1" class="ltx_tr">
<th id="A2.T5.1.1.2.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><span id="A2.T5.1.1.2.1.1.1" class="ltx_text ltx_font_bold">45</span></th>
<td id="A2.T5.1.1.2.1.2" class="ltx_td ltx_align_center ltx_border_t">26.48</td>
<td id="A2.T5.1.1.2.1.3" class="ltx_td ltx_align_center ltx_border_t"><span id="A2.T5.1.1.2.1.3.1" class="ltx_text ltx_font_bold">26.56</span></td>
<td id="A2.T5.1.1.2.1.4" class="ltx_td ltx_align_center ltx_border_t">26.48</td>
<td id="A2.T5.1.1.2.1.5" class="ltx_td ltx_align_center ltx_border_t">26.52</td>
</tr>
<tr id="A2.T5.1.1.3.2" class="ltx_tr">
<th id="A2.T5.1.1.3.2.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><span id="A2.T5.1.1.3.2.1.1" class="ltx_text ltx_font_bold">55</span></th>
<td id="A2.T5.1.1.3.2.2" class="ltx_td ltx_align_center">25.64</td>
<td id="A2.T5.1.1.3.2.3" class="ltx_td ltx_align_center">25.71</td>
<td id="A2.T5.1.1.3.2.4" class="ltx_td ltx_align_center">25.79</td>
<td id="A2.T5.1.1.3.2.5" class="ltx_td ltx_align_center"><span id="A2.T5.1.1.3.2.5.1" class="ltx_text ltx_font_bold">25.85</span></td>
</tr>
<tr id="A2.T5.1.1.4.3" class="ltx_tr">
<th id="A2.T5.1.1.4.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><span id="A2.T5.1.1.4.3.1.1" class="ltx_text ltx_font_bold">*65</span></th>
<td id="A2.T5.1.1.4.3.2" class="ltx_td ltx_align_center ltx_border_t">-</td>
<td id="A2.T5.1.1.4.3.3" class="ltx_td ltx_align_center ltx_border_t">22.19</td>
<td id="A2.T5.1.1.4.3.4" class="ltx_td ltx_align_center ltx_border_t">23.56</td>
<td id="A2.T5.1.1.4.3.5" class="ltx_td ltx_align_center ltx_border_t"><span id="A2.T5.1.1.4.3.5.1" class="ltx_text ltx_font_bold">23.87</span></td>
</tr>
<tr id="A2.T5.1.1.5.4" class="ltx_tr">
<th id="A2.T5.1.1.5.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb"><span id="A2.T5.1.1.5.4.1.1" class="ltx_text ltx_font_bold">*75</span></th>
<td id="A2.T5.1.1.5.4.2" class="ltx_td ltx_align_center ltx_border_bb">-</td>
<td id="A2.T5.1.1.5.4.3" class="ltx_td ltx_align_center ltx_border_bb">17.90</td>
<td id="A2.T5.1.1.5.4.4" class="ltx_td ltx_align_center ltx_border_bb">18.62</td>
<td id="A2.T5.1.1.5.4.5" class="ltx_td ltx_align_center ltx_border_bb"><span id="A2.T5.1.1.5.4.5.1" class="ltx_text ltx_font_bold">23.43</span></td>
</tr>
</tbody>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 5: </span><span id="A2.T5.3.1" class="ltx_text ltx_font_bold" style="font-size:90%;">Image Denoising –<span id="A2.T5.3.1.1" class="ltx_text ltx_font_medium"> Reconstruction PSNR results for IREM and baseline comparisons. The <math id="A2.T5.3.1.1.m1.1" class="ltx_Math" alttext="*" display="inline"><semantics id="A2.T5.3.1.1.m1.1b"><mo id="A2.T5.3.1.1.m1.1.1" xref="A2.T5.3.1.1.m1.1.1.cmml">∗</mo><annotation-xml encoding="MathML-Content" id="A2.T5.3.1.1.m1.1c"><times id="A2.T5.3.1.1.m1.1.1.cmml" xref="A2.T5.3.1.1.m1.1.1"></times></annotation-xml><annotation encoding="application/x-tex" id="A2.T5.3.1.1.m1.1d">*</annotation></semantics></math> indicates noise levels of 65 and 75 which are not in the training set. Numbers for baselines directly reported from <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib10" title="" class="ltx_ref">2020</a>)</cite>. IREM generalizes substantially better to more complex denoising tasks. </span></span></figcaption>
</figure>
<div id="A2.p2" class="ltx_para">
<p id="A2.p2.7" class="ltx_p">We then define our predicted solution</p>
<table id="A2.E8" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="A2.E8.m1.4" class="ltx_Math" alttext="\hat{{\bm{y}}}=\operatorname*{arg\,min}_{{\bm{y}}}\min_{{\bm{z}}}E_{\theta}({\bm{x}},{\bm{y}},{\bm{z}})," display="block"><semantics id="A2.E8.m1.4a"><mrow id="A2.E8.m1.4.4.1" xref="A2.E8.m1.4.4.1.1.cmml"><mrow id="A2.E8.m1.4.4.1.1" xref="A2.E8.m1.4.4.1.1.cmml"><mover accent="true" id="A2.E8.m1.4.4.1.1.2" xref="A2.E8.m1.4.4.1.1.2.cmml"><mi id="A2.E8.m1.4.4.1.1.2.2" xref="A2.E8.m1.4.4.1.1.2.2.cmml">𝒚</mi><mo id="A2.E8.m1.4.4.1.1.2.1" xref="A2.E8.m1.4.4.1.1.2.1.cmml">^</mo></mover><mo id="A2.E8.m1.4.4.1.1.1" xref="A2.E8.m1.4.4.1.1.1.cmml">=</mo><mrow id="A2.E8.m1.4.4.1.1.3" xref="A2.E8.m1.4.4.1.1.3.cmml"><mrow id="A2.E8.m1.4.4.1.1.3.2" xref="A2.E8.m1.4.4.1.1.3.2.cmml"><mrow id="A2.E8.m1.4.4.1.1.3.2.1" xref="A2.E8.m1.4.4.1.1.3.2.1.cmml"><munder id="A2.E8.m1.4.4.1.1.3.2.1.1" xref="A2.E8.m1.4.4.1.1.3.2.1.1.cmml"><mrow id="A2.E8.m1.4.4.1.1.3.2.1.1.2" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.cmml"><mi id="A2.E8.m1.4.4.1.1.3.2.1.1.2.2" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="A2.E8.m1.4.4.1.1.3.2.1.1.2.1" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.1.cmml">​</mo><mi id="A2.E8.m1.4.4.1.1.3.2.1.1.2.3" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.3.cmml">min</mi></mrow><mi id="A2.E8.m1.4.4.1.1.3.2.1.1.3" xref="A2.E8.m1.4.4.1.1.3.2.1.1.3.cmml">𝒚</mi></munder><mo lspace="0.167em" id="A2.E8.m1.4.4.1.1.3.2.1a" xref="A2.E8.m1.4.4.1.1.3.2.1.cmml">⁡</mo><munder id="A2.E8.m1.4.4.1.1.3.2.1.2" xref="A2.E8.m1.4.4.1.1.3.2.1.2.cmml"><mi id="A2.E8.m1.4.4.1.1.3.2.1.2.2" xref="A2.E8.m1.4.4.1.1.3.2.1.2.2.cmml">min</mi><mi id="A2.E8.m1.4.4.1.1.3.2.1.2.3" xref="A2.E8.m1.4.4.1.1.3.2.1.2.3.cmml">𝒛</mi></munder></mrow><mo lspace="0.167em" id="A2.E8.m1.4.4.1.1.3.2a" xref="A2.E8.m1.4.4.1.1.3.2.cmml">⁡</mo><msub id="A2.E8.m1.4.4.1.1.3.2.2" xref="A2.E8.m1.4.4.1.1.3.2.2.cmml"><mi id="A2.E8.m1.4.4.1.1.3.2.2.2" xref="A2.E8.m1.4.4.1.1.3.2.2.2.cmml">E</mi><mi id="A2.E8.m1.4.4.1.1.3.2.2.3" xref="A2.E8.m1.4.4.1.1.3.2.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="A2.E8.m1.4.4.1.1.3.1" xref="A2.E8.m1.4.4.1.1.3.1.cmml">​</mo><mrow id="A2.E8.m1.4.4.1.1.3.3.2" xref="A2.E8.m1.4.4.1.1.3.3.1.cmml"><mo stretchy="false" id="A2.E8.m1.4.4.1.1.3.3.2.1" xref="A2.E8.m1.4.4.1.1.3.3.1.cmml">(</mo><mi id="A2.E8.m1.1.1" xref="A2.E8.m1.1.1.cmml">𝒙</mi><mo id="A2.E8.m1.4.4.1.1.3.3.2.2" xref="A2.E8.m1.4.4.1.1.3.3.1.cmml">,</mo><mi id="A2.E8.m1.2.2" xref="A2.E8.m1.2.2.cmml">𝒚</mi><mo id="A2.E8.m1.4.4.1.1.3.3.2.3" xref="A2.E8.m1.4.4.1.1.3.3.1.cmml">,</mo><mi id="A2.E8.m1.3.3" xref="A2.E8.m1.3.3.cmml">𝒛</mi><mo stretchy="false" id="A2.E8.m1.4.4.1.1.3.3.2.4" xref="A2.E8.m1.4.4.1.1.3.3.1.cmml">)</mo></mrow></mrow></mrow><mo id="A2.E8.m1.4.4.1.2" xref="A2.E8.m1.4.4.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="A2.E8.m1.4b"><apply id="A2.E8.m1.4.4.1.1.cmml" xref="A2.E8.m1.4.4.1"><eq id="A2.E8.m1.4.4.1.1.1.cmml" xref="A2.E8.m1.4.4.1.1.1"></eq><apply id="A2.E8.m1.4.4.1.1.2.cmml" xref="A2.E8.m1.4.4.1.1.2"><ci id="A2.E8.m1.4.4.1.1.2.1.cmml" xref="A2.E8.m1.4.4.1.1.2.1">^</ci><ci id="A2.E8.m1.4.4.1.1.2.2.cmml" xref="A2.E8.m1.4.4.1.1.2.2">𝒚</ci></apply><apply id="A2.E8.m1.4.4.1.1.3.cmml" xref="A2.E8.m1.4.4.1.1.3"><times id="A2.E8.m1.4.4.1.1.3.1.cmml" xref="A2.E8.m1.4.4.1.1.3.1"></times><apply id="A2.E8.m1.4.4.1.1.3.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2"><apply id="A2.E8.m1.4.4.1.1.3.2.1.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1"><apply id="A2.E8.m1.4.4.1.1.3.2.1.1.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1"><csymbol cd="ambiguous" id="A2.E8.m1.4.4.1.1.3.2.1.1.1.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1">subscript</csymbol><apply id="A2.E8.m1.4.4.1.1.3.2.1.1.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2"><times id="A2.E8.m1.4.4.1.1.3.2.1.1.2.1.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.1"></times><ci id="A2.E8.m1.4.4.1.1.3.2.1.1.2.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.2">arg</ci><ci id="A2.E8.m1.4.4.1.1.3.2.1.1.2.3.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1.2.3">min</ci></apply><ci id="A2.E8.m1.4.4.1.1.3.2.1.1.3.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.1.3">𝒚</ci></apply><apply id="A2.E8.m1.4.4.1.1.3.2.1.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.2"><csymbol cd="ambiguous" id="A2.E8.m1.4.4.1.1.3.2.1.2.1.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.2">subscript</csymbol><min id="A2.E8.m1.4.4.1.1.3.2.1.2.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.2.2"></min><ci id="A2.E8.m1.4.4.1.1.3.2.1.2.3.cmml" xref="A2.E8.m1.4.4.1.1.3.2.1.2.3">𝒛</ci></apply></apply><apply id="A2.E8.m1.4.4.1.1.3.2.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2.2"><csymbol cd="ambiguous" id="A2.E8.m1.4.4.1.1.3.2.2.1.cmml" xref="A2.E8.m1.4.4.1.1.3.2.2">subscript</csymbol><ci id="A2.E8.m1.4.4.1.1.3.2.2.2.cmml" xref="A2.E8.m1.4.4.1.1.3.2.2.2">𝐸</ci><ci id="A2.E8.m1.4.4.1.1.3.2.2.3.cmml" xref="A2.E8.m1.4.4.1.1.3.2.2.3">𝜃</ci></apply></apply><vector id="A2.E8.m1.4.4.1.1.3.3.1.cmml" xref="A2.E8.m1.4.4.1.1.3.3.2"><ci id="A2.E8.m1.1.1.cmml" xref="A2.E8.m1.1.1">𝒙</ci><ci id="A2.E8.m1.2.2.cmml" xref="A2.E8.m1.2.2">𝒚</ci><ci id="A2.E8.m1.3.3.cmml" xref="A2.E8.m1.3.3">𝒛</ci></vector></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.E8.m1.4c">\hat{{\bm{y}}}=\operatorname*{arg\,min}_{{\bm{y}}}\min_{{\bm{z}}}E_{\theta}({\bm{x}},{\bm{y}},{\bm{z}}),</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(8)</span></td>
</tr></tbody>
</table>
<p id="A2.p2.6" class="ltx_p">where optimize over both output solutions <math id="A2.p2.1.m1.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="A2.p2.1.m1.1a"><mi id="A2.p2.1.m1.1.1" xref="A2.p2.1.m1.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="A2.p2.1.m1.1b"><ci id="A2.p2.1.m1.1.1.cmml" xref="A2.p2.1.m1.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p2.1.m1.1c">{\bm{y}}</annotation></semantics></math> and an external scratchpad <math id="A2.p2.2.m2.1" class="ltx_Math" alttext="{\bm{z}}" display="inline"><semantics id="A2.p2.2.m2.1a"><mi id="A2.p2.2.m2.1.1" xref="A2.p2.2.m2.1.1.cmml">𝒛</mi><annotation-xml encoding="MathML-Content" id="A2.p2.2.m2.1b"><ci id="A2.p2.2.m2.1.1.cmml" xref="A2.p2.2.m2.1.1">𝒛</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p2.2.m2.1c">{\bm{z}}</annotation></semantics></math> at prediction time. Analogous to the training procedure in the main paper, we may obtain a fast approximation of <math id="A2.p2.3.m3.1" class="ltx_Math" alttext="\hat{{\bm{y}}}" display="inline"><semantics id="A2.p2.3.m3.1a"><mover accent="true" id="A2.p2.3.m3.1.1" xref="A2.p2.3.m3.1.1.cmml"><mi id="A2.p2.3.m3.1.1.2" xref="A2.p2.3.m3.1.1.2.cmml">𝒚</mi><mo id="A2.p2.3.m3.1.1.1" xref="A2.p2.3.m3.1.1.1.cmml">^</mo></mover><annotation-xml encoding="MathML-Content" id="A2.p2.3.m3.1b"><apply id="A2.p2.3.m3.1.1.cmml" xref="A2.p2.3.m3.1.1"><ci id="A2.p2.3.m3.1.1.1.cmml" xref="A2.p2.3.m3.1.1.1">^</ci><ci id="A2.p2.3.m3.1.1.2.cmml" xref="A2.p2.3.m3.1.1.2">𝒚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.p2.3.m3.1c">\hat{{\bm{y}}}</annotation></semantics></math> as <math id="A2.p2.4.m4.1" class="ltx_Math" alttext="{\bm{y}}^{N}" display="inline"><semantics id="A2.p2.4.m4.1a"><msup id="A2.p2.4.m4.1.1" xref="A2.p2.4.m4.1.1.cmml"><mi id="A2.p2.4.m4.1.1.2" xref="A2.p2.4.m4.1.1.2.cmml">𝒚</mi><mi id="A2.p2.4.m4.1.1.3" xref="A2.p2.4.m4.1.1.3.cmml">N</mi></msup><annotation-xml encoding="MathML-Content" id="A2.p2.4.m4.1b"><apply id="A2.p2.4.m4.1.1.cmml" xref="A2.p2.4.m4.1.1"><csymbol cd="ambiguous" id="A2.p2.4.m4.1.1.1.cmml" xref="A2.p2.4.m4.1.1">superscript</csymbol><ci id="A2.p2.4.m4.1.1.2.cmml" xref="A2.p2.4.m4.1.1.2">𝒚</ci><ci id="A2.p2.4.m4.1.1.3.cmml" xref="A2.p2.4.m4.1.1.3">𝑁</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.p2.4.m4.1c">{\bm{y}}^{N}</annotation></semantics></math> training time by jointly optimizing <math id="A2.p2.5.m5.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="A2.p2.5.m5.1a"><mi id="A2.p2.5.m5.1.1" xref="A2.p2.5.m5.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="A2.p2.5.m5.1b"><ci id="A2.p2.5.m5.1.1.cmml" xref="A2.p2.5.m5.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p2.5.m5.1c">{\bm{y}}</annotation></semantics></math> and <math id="A2.p2.6.m6.1" class="ltx_Math" alttext="{\bm{z}}" display="inline"><semantics id="A2.p2.6.m6.1a"><mi id="A2.p2.6.m6.1.1" xref="A2.p2.6.m6.1.1.cmml">𝒛</mi><annotation-xml encoding="MathML-Content" id="A2.p2.6.m6.1b"><ci id="A2.p2.6.m6.1.1.cmml" xref="A2.p2.6.m6.1.1">𝒛</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p2.6.m6.1c">{\bm{z}}</annotation></semantics></math></p>
<table id="A2.E9" class="ltx_equationgroup ltx_eqn_table">
<tbody>
<tr id="A2.E9X" class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="A2.E9X.2.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{\bm{y}}_{i}^{n}" display="inline"><semantics id="A2.E9X.2.1.1.m1.1a"><msubsup id="A2.E9X.2.1.1.m1.1.1" xref="A2.E9X.2.1.1.m1.1.1.cmml"><mi id="A2.E9X.2.1.1.m1.1.1.2.2" xref="A2.E9X.2.1.1.m1.1.1.2.2.cmml">𝒚</mi><mi id="A2.E9X.2.1.1.m1.1.1.2.3" xref="A2.E9X.2.1.1.m1.1.1.2.3.cmml">i</mi><mi id="A2.E9X.2.1.1.m1.1.1.3" xref="A2.E9X.2.1.1.m1.1.1.3.cmml">n</mi></msubsup><annotation-xml encoding="MathML-Content" id="A2.E9X.2.1.1.m1.1b"><apply id="A2.E9X.2.1.1.m1.1.1.cmml" xref="A2.E9X.2.1.1.m1.1.1"><csymbol cd="ambiguous" id="A2.E9X.2.1.1.m1.1.1.1.cmml" xref="A2.E9X.2.1.1.m1.1.1">superscript</csymbol><apply id="A2.E9X.2.1.1.m1.1.1.2.cmml" xref="A2.E9X.2.1.1.m1.1.1"><csymbol cd="ambiguous" id="A2.E9X.2.1.1.m1.1.1.2.1.cmml" xref="A2.E9X.2.1.1.m1.1.1">subscript</csymbol><ci id="A2.E9X.2.1.1.m1.1.1.2.2.cmml" xref="A2.E9X.2.1.1.m1.1.1.2.2">𝒚</ci><ci id="A2.E9X.2.1.1.m1.1.1.2.3.cmml" xref="A2.E9X.2.1.1.m1.1.1.2.3">𝑖</ci></apply><ci id="A2.E9X.2.1.1.m1.1.1.3.cmml" xref="A2.E9X.2.1.1.m1.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.E9X.2.1.1.m1.1c">\displaystyle{\bm{y}}_{i}^{n}</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="A2.E9X.3.2.2.m1.3" class="ltx_Math" alttext="\displaystyle={\bm{y}}_{i}^{n-1}-\lambda\nabla_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}}_{i}^{n-1},{\bm{z}}_{i}^{n-1})" display="inline"><semantics id="A2.E9X.3.2.2.m1.3a"><mrow id="A2.E9X.3.2.2.m1.3.3" xref="A2.E9X.3.2.2.m1.3.3.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.5" xref="A2.E9X.3.2.2.m1.3.3.5.cmml"></mi><mo id="A2.E9X.3.2.2.m1.3.3.4" xref="A2.E9X.3.2.2.m1.3.3.4.cmml">=</mo><mrow id="A2.E9X.3.2.2.m1.3.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.cmml"><msubsup id="A2.E9X.3.2.2.m1.3.3.3.5" xref="A2.E9X.3.2.2.m1.3.3.3.5.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.3.5.2.2" xref="A2.E9X.3.2.2.m1.3.3.3.5.2.2.cmml">𝒚</mi><mi id="A2.E9X.3.2.2.m1.3.3.3.5.2.3" xref="A2.E9X.3.2.2.m1.3.3.3.5.2.3.cmml">i</mi><mrow id="A2.E9X.3.2.2.m1.3.3.3.5.3" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.3.5.3.2" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.2.cmml">n</mi><mo id="A2.E9X.3.2.2.m1.3.3.3.5.3.1" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.1.cmml">−</mo><mn id="A2.E9X.3.2.2.m1.3.3.3.5.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.3.cmml">1</mn></mrow></msubsup><mo id="A2.E9X.3.2.2.m1.3.3.3.4" xref="A2.E9X.3.2.2.m1.3.3.3.4.cmml">−</mo><mrow id="A2.E9X.3.2.2.m1.3.3.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.3.3.5" xref="A2.E9X.3.2.2.m1.3.3.3.3.5.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="A2.E9X.3.2.2.m1.3.3.3.3.4" xref="A2.E9X.3.2.2.m1.3.3.3.3.4.cmml">​</mo><mrow id="A2.E9X.3.2.2.m1.3.3.3.3.6" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.cmml"><msub id="A2.E9X.3.2.2.m1.3.3.3.3.6.1" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1.cmml"><mo rspace="0.167em" id="A2.E9X.3.2.2.m1.3.3.3.3.6.1.2" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1.2.cmml">∇</mo><mi id="A2.E9X.3.2.2.m1.3.3.3.3.6.1.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1.3.cmml">𝒚</mi></msub><msub id="A2.E9X.3.2.2.m1.3.3.3.3.6.2" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.3.3.6.2.2" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2.2.cmml">E</mi><mi id="A2.E9X.3.2.2.m1.3.3.3.3.6.2.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="A2.E9X.3.2.2.m1.3.3.3.3.4a" xref="A2.E9X.3.2.2.m1.3.3.3.3.4.cmml">​</mo><mrow id="A2.E9X.3.2.2.m1.3.3.3.3.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.4.cmml"><mo stretchy="false" id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.4" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.4.cmml">(</mo><msub id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.cmml"><mi id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.2" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.3" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.5" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.4.cmml">,</mo><msubsup id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.cmml"><mi id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.2" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mi id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.3" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.3.cmml">i</mi><mrow id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.cmml"><mi id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.2" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.2.cmml">n</mi><mo id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.1" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.1.cmml">−</mo><mn id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.3" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.3.cmml">1</mn></mrow></msubsup><mo id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.6" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.4.cmml">,</mo><msubsup id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.2" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.2.cmml">𝒛</mi><mi id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.3.cmml">i</mi><mrow id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.cmml"><mi id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.2" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.2.cmml">n</mi><mo id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.1" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.1.cmml">−</mo><mn id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.3" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.3.cmml">1</mn></mrow></msubsup><mo stretchy="false" id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.7" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.4.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A2.E9X.3.2.2.m1.3b"><apply id="A2.E9X.3.2.2.m1.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3"><eq id="A2.E9X.3.2.2.m1.3.3.4.cmml" xref="A2.E9X.3.2.2.m1.3.3.4"></eq><csymbol cd="latexml" id="A2.E9X.3.2.2.m1.3.3.5.cmml" xref="A2.E9X.3.2.2.m1.3.3.5">absent</csymbol><apply id="A2.E9X.3.2.2.m1.3.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3"><minus id="A2.E9X.3.2.2.m1.3.3.3.4.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.4"></minus><apply id="A2.E9X.3.2.2.m1.3.3.3.5.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.3.3.3.5.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5">superscript</csymbol><apply id="A2.E9X.3.2.2.m1.3.3.3.5.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.3.3.3.5.2.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5">subscript</csymbol><ci id="A2.E9X.3.2.2.m1.3.3.3.5.2.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5.2.2">𝒚</ci><ci id="A2.E9X.3.2.2.m1.3.3.3.5.2.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5.2.3">𝑖</ci></apply><apply id="A2.E9X.3.2.2.m1.3.3.3.5.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5.3"><minus id="A2.E9X.3.2.2.m1.3.3.3.5.3.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.1"></minus><ci id="A2.E9X.3.2.2.m1.3.3.3.5.3.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.2">𝑛</ci><cn type="integer" id="A2.E9X.3.2.2.m1.3.3.3.5.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.5.3.3">1</cn></apply></apply><apply id="A2.E9X.3.2.2.m1.3.3.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3"><times id="A2.E9X.3.2.2.m1.3.3.3.3.4.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.4"></times><ci id="A2.E9X.3.2.2.m1.3.3.3.3.5.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.5">𝜆</ci><apply id="A2.E9X.3.2.2.m1.3.3.3.3.6.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6"><apply id="A2.E9X.3.2.2.m1.3.3.3.3.6.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.3.3.3.3.6.1.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1">subscript</csymbol><ci id="A2.E9X.3.2.2.m1.3.3.3.3.6.1.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1.2">∇</ci><ci id="A2.E9X.3.2.2.m1.3.3.3.3.6.1.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.1.3">𝒚</ci></apply><apply id="A2.E9X.3.2.2.m1.3.3.3.3.6.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.3.3.3.3.6.2.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2">subscript</csymbol><ci id="A2.E9X.3.2.2.m1.3.3.3.3.6.2.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2.2">𝐸</ci><ci id="A2.E9X.3.2.2.m1.3.3.3.3.6.2.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.6.2.3">𝜃</ci></apply></apply><vector id="A2.E9X.3.2.2.m1.3.3.3.3.3.4.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3"><apply id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.cmml" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.1.cmml" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.2.cmml" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.3.cmml" xref="A2.E9X.3.2.2.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.1.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2">superscript</csymbol><apply id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.1.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2">subscript</csymbol><ci id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.2.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.2">𝒚</ci><ci id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.3.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3"><minus id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.1.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.1"></minus><ci id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.2.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.2">𝑛</ci><cn type="integer" id="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.3.cmml" xref="A2.E9X.3.2.2.m1.2.2.2.2.2.2.2.3.3">1</cn></apply></apply><apply id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3">superscript</csymbol><apply id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3"><csymbol cd="ambiguous" id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3">subscript</csymbol><ci id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.2">𝒛</ci><ci id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.2.3">𝑖</ci></apply><apply id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3"><minus id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.1.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.1"></minus><ci id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.2.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.2">𝑛</ci><cn type="integer" id="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.3.cmml" xref="A2.E9X.3.2.2.m1.3.3.3.3.3.3.3.3.3">1</cn></apply></apply></vector></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.E9X.3.2.2.m1.3c">\displaystyle={\bm{y}}_{i}^{n-1}-\lambda\nabla_{{\bm{y}}}E_{\theta}({\bm{x}}_{i},{\bm{y}}_{i}^{n-1},{\bm{z}}_{i}^{n-1})</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="2" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equationgroup ltx_align_right">(9)</span></td>
</tr>
<tr id="A2.E9Xa" class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="A2.E9Xa.2.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{\bm{z}}_{i}^{n}" display="inline"><semantics id="A2.E9Xa.2.1.1.m1.1a"><msubsup id="A2.E9Xa.2.1.1.m1.1.1" xref="A2.E9Xa.2.1.1.m1.1.1.cmml"><mi id="A2.E9Xa.2.1.1.m1.1.1.2.2" xref="A2.E9Xa.2.1.1.m1.1.1.2.2.cmml">𝒛</mi><mi id="A2.E9Xa.2.1.1.m1.1.1.2.3" xref="A2.E9Xa.2.1.1.m1.1.1.2.3.cmml">i</mi><mi id="A2.E9Xa.2.1.1.m1.1.1.3" xref="A2.E9Xa.2.1.1.m1.1.1.3.cmml">n</mi></msubsup><annotation-xml encoding="MathML-Content" id="A2.E9Xa.2.1.1.m1.1b"><apply id="A2.E9Xa.2.1.1.m1.1.1.cmml" xref="A2.E9Xa.2.1.1.m1.1.1"><csymbol cd="ambiguous" id="A2.E9Xa.2.1.1.m1.1.1.1.cmml" xref="A2.E9Xa.2.1.1.m1.1.1">superscript</csymbol><apply id="A2.E9Xa.2.1.1.m1.1.1.2.cmml" xref="A2.E9Xa.2.1.1.m1.1.1"><csymbol cd="ambiguous" id="A2.E9Xa.2.1.1.m1.1.1.2.1.cmml" xref="A2.E9Xa.2.1.1.m1.1.1">subscript</csymbol><ci id="A2.E9Xa.2.1.1.m1.1.1.2.2.cmml" xref="A2.E9Xa.2.1.1.m1.1.1.2.2">𝒛</ci><ci id="A2.E9Xa.2.1.1.m1.1.1.2.3.cmml" xref="A2.E9Xa.2.1.1.m1.1.1.2.3">𝑖</ci></apply><ci id="A2.E9Xa.2.1.1.m1.1.1.3.cmml" xref="A2.E9Xa.2.1.1.m1.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.E9Xa.2.1.1.m1.1c">\displaystyle{\bm{z}}_{i}^{n}</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="A2.E9Xa.3.2.2.m1.1" class="ltx_Math" alttext="\displaystyle={\bm{z}}_{i}^{n-1}-\lambda\nabla_{{\bm{z}}}E_{\theta}({\bm{x}}_{i},{\bm{y}}_{i}^{n-1},{\bm{z}}_{i}^{n-1})." display="inline"><semantics id="A2.E9Xa.3.2.2.m1.1a"><mrow id="A2.E9Xa.3.2.2.m1.1.1.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.cmml"><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.5" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.5.cmml"></mi><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.4" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.4.cmml">=</mo><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.cmml"><msubsup id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.2.cmml">𝒛</mi><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.3.cmml">i</mi><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.2.cmml">n</mi><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.1.cmml">−</mo><mn id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.3.cmml">1</mn></mrow></msubsup><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.4" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.4.cmml">−</mo><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.5" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.5.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.4" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.4.cmml">​</mo><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.cmml"><msub id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.cmml"><mo rspace="0.167em" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.2.cmml">∇</mo><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.3.cmml">𝒛</mi></msub><msub id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.2.cmml">E</mi><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.4a" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.4.cmml">​</mo><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.4.cmml"><mo stretchy="false" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.4" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.4.cmml">(</mo><msub id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.5" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.4.cmml">,</mo><msubsup id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.2.cmml">𝒚</mi><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.3.cmml">i</mi><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.2.cmml">n</mi><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.1.cmml">−</mo><mn id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.3.cmml">1</mn></mrow></msubsup><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.6" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.4.cmml">,</mo><msubsup id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.2.cmml">𝒛</mi><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.3.cmml">i</mi><mrow id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.cmml"><mi id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.2.cmml">n</mi><mo id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.1" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.1.cmml">−</mo><mn id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.3" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.3.cmml">1</mn></mrow></msubsup><mo stretchy="false" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.7" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.4.cmml">)</mo></mrow></mrow></mrow></mrow><mo lspace="0em" id="A2.E9Xa.3.2.2.m1.1.1.1.2" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="A2.E9Xa.3.2.2.m1.1b"><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1"><eq id="A2.E9Xa.3.2.2.m1.1.1.1.1.4.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.4"></eq><csymbol cd="latexml" id="A2.E9Xa.3.2.2.m1.1.1.1.1.5.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.5">absent</csymbol><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3"><minus id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.4.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.4"></minus><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5">superscript</csymbol><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5">subscript</csymbol><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.2">𝒛</ci><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.2.3">𝑖</ci></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3"><minus id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.1"></minus><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.2">𝑛</ci><cn type="integer" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.5.3.3">1</cn></apply></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3"><times id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.4.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.4"></times><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.5.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.5">𝜆</ci><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6"><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1">subscript</csymbol><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.2">∇</ci><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.1.3">𝒛</ci></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2">subscript</csymbol><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.2">𝐸</ci><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.6.2.3">𝜃</ci></apply></apply><vector id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.4.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3"><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2">superscript</csymbol><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2">subscript</csymbol><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.2">𝒚</ci><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3"><minus id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.1"></minus><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.2">𝑛</ci><cn type="integer" id="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.2.2.2.2.2.3.3">1</cn></apply></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3">superscript</csymbol><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3"><csymbol cd="ambiguous" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3">subscript</csymbol><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.2">𝒛</ci><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.2.3">𝑖</ci></apply><apply id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3"><minus id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.1.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.1"></minus><ci id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.2.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.2">𝑛</ci><cn type="integer" id="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.3.cmml" xref="A2.E9Xa.3.2.2.m1.1.1.1.1.3.3.3.3.3.3.3">1</cn></apply></apply></vector></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.E9Xa.3.2.2.m1.1c">\displaystyle={\bm{z}}_{i}^{n-1}-\lambda\nabla_{{\bm{z}}}E_{\theta}({\bm{x}}_{i},{\bm{y}}_{i}^{n-1},{\bm{z}}_{i}^{n-1}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</tbody>
</table>
<p id="A2.p2.8" class="ltx_p">We then analogously minimize the corresponding loss:</p>
<table id="A2.E10" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="A2.E10.m1.2" class="ltx_Math" alttext="\mathcal{L}_{\text{MSE}}(\theta)=\|{\bm{y}}_{i}^{N}-{\bm{y}}_{i}\|^{2}," display="block"><semantics id="A2.E10.m1.2a"><mrow id="A2.E10.m1.2.2.1" xref="A2.E10.m1.2.2.1.1.cmml"><mrow id="A2.E10.m1.2.2.1.1" xref="A2.E10.m1.2.2.1.1.cmml"><mrow id="A2.E10.m1.2.2.1.1.3" xref="A2.E10.m1.2.2.1.1.3.cmml"><msub id="A2.E10.m1.2.2.1.1.3.2" xref="A2.E10.m1.2.2.1.1.3.2.cmml"><mi class="ltx_font_mathcaligraphic" id="A2.E10.m1.2.2.1.1.3.2.2" xref="A2.E10.m1.2.2.1.1.3.2.2.cmml">ℒ</mi><mtext id="A2.E10.m1.2.2.1.1.3.2.3" xref="A2.E10.m1.2.2.1.1.3.2.3a.cmml">MSE</mtext></msub><mo lspace="0em" rspace="0em" id="A2.E10.m1.2.2.1.1.3.1" xref="A2.E10.m1.2.2.1.1.3.1.cmml">​</mo><mrow id="A2.E10.m1.2.2.1.1.3.3.2" xref="A2.E10.m1.2.2.1.1.3.cmml"><mo stretchy="false" id="A2.E10.m1.2.2.1.1.3.3.2.1" xref="A2.E10.m1.2.2.1.1.3.cmml">(</mo><mi id="A2.E10.m1.1.1" xref="A2.E10.m1.1.1.cmml">θ</mi><mo stretchy="false" id="A2.E10.m1.2.2.1.1.3.3.2.2" xref="A2.E10.m1.2.2.1.1.3.cmml">)</mo></mrow></mrow><mo id="A2.E10.m1.2.2.1.1.2" xref="A2.E10.m1.2.2.1.1.2.cmml">=</mo><msup id="A2.E10.m1.2.2.1.1.1" xref="A2.E10.m1.2.2.1.1.1.cmml"><mrow id="A2.E10.m1.2.2.1.1.1.1.1" xref="A2.E10.m1.2.2.1.1.1.1.2.cmml"><mo stretchy="false" id="A2.E10.m1.2.2.1.1.1.1.1.2" xref="A2.E10.m1.2.2.1.1.1.1.2.1.cmml">‖</mo><mrow id="A2.E10.m1.2.2.1.1.1.1.1.1" xref="A2.E10.m1.2.2.1.1.1.1.1.1.cmml"><msubsup id="A2.E10.m1.2.2.1.1.1.1.1.1.2" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.cmml"><mi id="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.2" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.2.cmml">𝒚</mi><mi id="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.3" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.3.cmml">i</mi><mi id="A2.E10.m1.2.2.1.1.1.1.1.1.2.3" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.3.cmml">N</mi></msubsup><mo id="A2.E10.m1.2.2.1.1.1.1.1.1.1" xref="A2.E10.m1.2.2.1.1.1.1.1.1.1.cmml">−</mo><msub id="A2.E10.m1.2.2.1.1.1.1.1.1.3" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3.cmml"><mi id="A2.E10.m1.2.2.1.1.1.1.1.1.3.2" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3.2.cmml">𝒚</mi><mi id="A2.E10.m1.2.2.1.1.1.1.1.1.3.3" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo stretchy="false" id="A2.E10.m1.2.2.1.1.1.1.1.3" xref="A2.E10.m1.2.2.1.1.1.1.2.1.cmml">‖</mo></mrow><mn id="A2.E10.m1.2.2.1.1.1.3" xref="A2.E10.m1.2.2.1.1.1.3.cmml">2</mn></msup></mrow><mo id="A2.E10.m1.2.2.1.2" xref="A2.E10.m1.2.2.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="A2.E10.m1.2b"><apply id="A2.E10.m1.2.2.1.1.cmml" xref="A2.E10.m1.2.2.1"><eq id="A2.E10.m1.2.2.1.1.2.cmml" xref="A2.E10.m1.2.2.1.1.2"></eq><apply id="A2.E10.m1.2.2.1.1.3.cmml" xref="A2.E10.m1.2.2.1.1.3"><times id="A2.E10.m1.2.2.1.1.3.1.cmml" xref="A2.E10.m1.2.2.1.1.3.1"></times><apply id="A2.E10.m1.2.2.1.1.3.2.cmml" xref="A2.E10.m1.2.2.1.1.3.2"><csymbol cd="ambiguous" id="A2.E10.m1.2.2.1.1.3.2.1.cmml" xref="A2.E10.m1.2.2.1.1.3.2">subscript</csymbol><ci id="A2.E10.m1.2.2.1.1.3.2.2.cmml" xref="A2.E10.m1.2.2.1.1.3.2.2">ℒ</ci><ci id="A2.E10.m1.2.2.1.1.3.2.3a.cmml" xref="A2.E10.m1.2.2.1.1.3.2.3"><mtext mathsize="70%" id="A2.E10.m1.2.2.1.1.3.2.3.cmml" xref="A2.E10.m1.2.2.1.1.3.2.3">MSE</mtext></ci></apply><ci id="A2.E10.m1.1.1.cmml" xref="A2.E10.m1.1.1">𝜃</ci></apply><apply id="A2.E10.m1.2.2.1.1.1.cmml" xref="A2.E10.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="A2.E10.m1.2.2.1.1.1.2.cmml" xref="A2.E10.m1.2.2.1.1.1">superscript</csymbol><apply id="A2.E10.m1.2.2.1.1.1.1.2.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1"><csymbol cd="latexml" id="A2.E10.m1.2.2.1.1.1.1.2.1.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.2">norm</csymbol><apply id="A2.E10.m1.2.2.1.1.1.1.1.1.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1"><minus id="A2.E10.m1.2.2.1.1.1.1.1.1.1.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.1"></minus><apply id="A2.E10.m1.2.2.1.1.1.1.1.1.2.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="A2.E10.m1.2.2.1.1.1.1.1.1.2.1.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2">superscript</csymbol><apply id="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.1.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2">subscript</csymbol><ci id="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.2.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.2">𝒚</ci><ci id="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.3.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.2.3">𝑖</ci></apply><ci id="A2.E10.m1.2.2.1.1.1.1.1.1.2.3.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.2.3">𝑁</ci></apply><apply id="A2.E10.m1.2.2.1.1.1.1.1.1.3.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="A2.E10.m1.2.2.1.1.1.1.1.1.3.1.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3">subscript</csymbol><ci id="A2.E10.m1.2.2.1.1.1.1.1.1.3.2.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3.2">𝒚</ci><ci id="A2.E10.m1.2.2.1.1.1.1.1.1.3.3.cmml" xref="A2.E10.m1.2.2.1.1.1.1.1.1.3.3">𝑖</ci></apply></apply></apply><cn type="integer" id="A2.E10.m1.2.2.1.1.1.3.cmml" xref="A2.E10.m1.2.2.1.1.1.3">2</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.E10.m1.2c">\mathcal{L}_{\text{MSE}}(\theta)=\|{\bm{y}}_{i}^{N}-{\bm{y}}_{i}\|^{2},</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(10)</span></td>
</tr></tbody>
</table>
</div>
<div id="A2.p3" class="ltx_para">
<p id="A2.p3.1" class="ltx_p">We provide the overall pseudocode for training IREM with external memory in Algorithm  <a href="#alg3" title="Algorithm 3 ‣ Appendix B Iterative Reasoning with a Scratchpad ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>.</p>
</div>
<figure id="alg3" class="ltx_float ltx_float_algorithm ltx_framed ltx_framed_top">
<div id="alg3.2" class="ltx_listing ltx_listing">
<div id="alg2.l1" class="ltx_listingline">
<span id="alg2.l1.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg2.l1.2" class="ltx_text ltx_font_bold" style="font-size:90%;">Input:</span><span id="alg2.l1.3" class="ltx_text" style="font-size:90%;"> Data Dist </span><math id="alg2.l1.m1.2" class="ltx_Math" alttext="p_{D}({\bm{x}},{\bm{y}})" display="inline"><semantics id="alg2.l1.m1.2a"><mrow id="alg2.l1.m1.2.3" xref="alg2.l1.m1.2.3.cmml"><msub id="alg2.l1.m1.2.3.2" xref="alg2.l1.m1.2.3.2.cmml"><mi mathsize="90%" id="alg2.l1.m1.2.3.2.2" xref="alg2.l1.m1.2.3.2.2.cmml">p</mi><mi mathsize="90%" id="alg2.l1.m1.2.3.2.3" xref="alg2.l1.m1.2.3.2.3.cmml">D</mi></msub><mo lspace="0em" rspace="0em" id="alg2.l1.m1.2.3.1" xref="alg2.l1.m1.2.3.1.cmml">​</mo><mrow id="alg2.l1.m1.2.3.3.2" xref="alg2.l1.m1.2.3.3.1.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l1.m1.2.3.3.2.1" xref="alg2.l1.m1.2.3.3.1.cmml">(</mo><mi mathsize="90%" id="alg2.l1.m1.1.1" xref="alg2.l1.m1.1.1.cmml">𝒙</mi><mo mathsize="90%" id="alg2.l1.m1.2.3.3.2.2" xref="alg2.l1.m1.2.3.3.1.cmml">,</mo><mi mathsize="90%" id="alg2.l1.m1.2.2" xref="alg2.l1.m1.2.2.cmml">𝒚</mi><mo maxsize="90%" minsize="90%" id="alg2.l1.m1.2.3.3.2.3" xref="alg2.l1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l1.m1.2b"><apply id="alg2.l1.m1.2.3.cmml" xref="alg2.l1.m1.2.3"><times id="alg2.l1.m1.2.3.1.cmml" xref="alg2.l1.m1.2.3.1"></times><apply id="alg2.l1.m1.2.3.2.cmml" xref="alg2.l1.m1.2.3.2"><csymbol cd="ambiguous" id="alg2.l1.m1.2.3.2.1.cmml" xref="alg2.l1.m1.2.3.2">subscript</csymbol><ci id="alg2.l1.m1.2.3.2.2.cmml" xref="alg2.l1.m1.2.3.2.2">𝑝</ci><ci id="alg2.l1.m1.2.3.2.3.cmml" xref="alg2.l1.m1.2.3.2.3">𝐷</ci></apply><interval closure="open" id="alg2.l1.m1.2.3.3.1.cmml" xref="alg2.l1.m1.2.3.3.2"><ci id="alg2.l1.m1.1.1.cmml" xref="alg2.l1.m1.1.1">𝒙</ci><ci id="alg2.l1.m1.2.2.cmml" xref="alg2.l1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m1.2c">p_{D}({\bm{x}},{\bm{y}})</annotation></semantics></math><span id="alg2.l1.4" class="ltx_text" style="font-size:90%;">, Replay Buffer </span><math id="alg2.l1.m2.1" class="ltx_Math" alttext="\mathcal{B}" display="inline"><semantics id="alg2.l1.m2.1a"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg2.l1.m2.1.1" xref="alg2.l1.m2.1.1.cmml">ℬ</mi><annotation-xml encoding="MathML-Content" id="alg2.l1.m2.1b"><ci id="alg2.l1.m2.1.1.cmml" xref="alg2.l1.m2.1.1">ℬ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m2.1c">\mathcal{B}</annotation></semantics></math><span id="alg2.l1.5" class="ltx_text" style="font-size:90%;">, Step Size </span><math id="alg2.l1.m3.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="alg2.l1.m3.1a"><mi mathsize="90%" id="alg2.l1.m3.1.1" xref="alg2.l1.m3.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="alg2.l1.m3.1b"><ci id="alg2.l1.m3.1.1.cmml" xref="alg2.l1.m3.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m3.1c">\lambda</annotation></semantics></math><span id="alg2.l1.6" class="ltx_text" style="font-size:90%;">, Number of Steps </span><math id="alg2.l1.m4.1" class="ltx_Math" alttext="N" display="inline"><semantics id="alg2.l1.m4.1a"><mi mathsize="90%" id="alg2.l1.m4.1.1" xref="alg2.l1.m4.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="alg2.l1.m4.1b"><ci id="alg2.l1.m4.1.1.cmml" xref="alg2.l1.m4.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m4.1c">N</annotation></semantics></math><span id="alg2.l1.7" class="ltx_text" style="font-size:90%;">, EBM </span><math id="alg2.l1.m5.1" class="ltx_Math" alttext="E_{\theta}(\cdot)" display="inline"><semantics id="alg2.l1.m5.1a"><mrow id="alg2.l1.m5.1.2" xref="alg2.l1.m5.1.2.cmml"><msub id="alg2.l1.m5.1.2.2" xref="alg2.l1.m5.1.2.2.cmml"><mi mathsize="90%" id="alg2.l1.m5.1.2.2.2" xref="alg2.l1.m5.1.2.2.2.cmml">E</mi><mi mathsize="90%" id="alg2.l1.m5.1.2.2.3" xref="alg2.l1.m5.1.2.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="alg2.l1.m5.1.2.1" xref="alg2.l1.m5.1.2.1.cmml">​</mo><mrow id="alg2.l1.m5.1.2.3.2" xref="alg2.l1.m5.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l1.m5.1.2.3.2.1" xref="alg2.l1.m5.1.2.cmml">(</mo><mo lspace="0em" mathsize="90%" rspace="0em" id="alg2.l1.m5.1.1" xref="alg2.l1.m5.1.1.cmml">⋅</mo><mo maxsize="90%" minsize="90%" id="alg2.l1.m5.1.2.3.2.2" xref="alg2.l1.m5.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l1.m5.1b"><apply id="alg2.l1.m5.1.2.cmml" xref="alg2.l1.m5.1.2"><times id="alg2.l1.m5.1.2.1.cmml" xref="alg2.l1.m5.1.2.1"></times><apply id="alg2.l1.m5.1.2.2.cmml" xref="alg2.l1.m5.1.2.2"><csymbol cd="ambiguous" id="alg2.l1.m5.1.2.2.1.cmml" xref="alg2.l1.m5.1.2.2">subscript</csymbol><ci id="alg2.l1.m5.1.2.2.2.cmml" xref="alg2.l1.m5.1.2.2.2">𝐸</ci><ci id="alg2.l1.m5.1.2.2.3.cmml" xref="alg2.l1.m5.1.2.2.3">𝜃</ci></apply><ci id="alg2.l1.m5.1.1.cmml" xref="alg2.l1.m5.1.1">⋅</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m5.1c">E_{\theta}(\cdot)</annotation></semantics></math><span id="alg2.l1.8" class="ltx_text" style="font-size:90%;">, Uniform Distribution </span><math id="alg2.l1.m6.2" class="ltx_Math" alttext="U(-1,1)" display="inline"><semantics id="alg2.l1.m6.2a"><mrow id="alg2.l1.m6.2.2" xref="alg2.l1.m6.2.2.cmml"><mi mathsize="90%" id="alg2.l1.m6.2.2.3" xref="alg2.l1.m6.2.2.3.cmml">U</mi><mo lspace="0em" rspace="0em" id="alg2.l1.m6.2.2.2" xref="alg2.l1.m6.2.2.2.cmml">​</mo><mrow id="alg2.l1.m6.2.2.1.1" xref="alg2.l1.m6.2.2.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l1.m6.2.2.1.1.2" xref="alg2.l1.m6.2.2.1.2.cmml">(</mo><mrow id="alg2.l1.m6.2.2.1.1.1" xref="alg2.l1.m6.2.2.1.1.1.cmml"><mo mathsize="90%" id="alg2.l1.m6.2.2.1.1.1a" xref="alg2.l1.m6.2.2.1.1.1.cmml">−</mo><mn mathsize="90%" id="alg2.l1.m6.2.2.1.1.1.2" xref="alg2.l1.m6.2.2.1.1.1.2.cmml">1</mn></mrow><mo mathsize="90%" id="alg2.l1.m6.2.2.1.1.3" xref="alg2.l1.m6.2.2.1.2.cmml">,</mo><mn mathsize="90%" id="alg2.l1.m6.1.1" xref="alg2.l1.m6.1.1.cmml">1</mn><mo maxsize="90%" minsize="90%" id="alg2.l1.m6.2.2.1.1.4" xref="alg2.l1.m6.2.2.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l1.m6.2b"><apply id="alg2.l1.m6.2.2.cmml" xref="alg2.l1.m6.2.2"><times id="alg2.l1.m6.2.2.2.cmml" xref="alg2.l1.m6.2.2.2"></times><ci id="alg2.l1.m6.2.2.3.cmml" xref="alg2.l1.m6.2.2.3">𝑈</ci><interval closure="open" id="alg2.l1.m6.2.2.1.2.cmml" xref="alg2.l1.m6.2.2.1.1"><apply id="alg2.l1.m6.2.2.1.1.1.cmml" xref="alg2.l1.m6.2.2.1.1.1"><minus id="alg2.l1.m6.2.2.1.1.1.1.cmml" xref="alg2.l1.m6.2.2.1.1.1"></minus><cn type="integer" id="alg2.l1.m6.2.2.1.1.1.2.cmml" xref="alg2.l1.m6.2.2.1.1.1.2">1</cn></apply><cn type="integer" id="alg2.l1.m6.1.1.cmml" xref="alg2.l1.m6.1.1">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l1.m6.2c">U(-1,1)</annotation></semantics></math><span id="alg2.l1.9" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l2" class="ltx_listingline">
<span id="alg2.l2.1" class="ltx_text" style="font-size:90%;">  </span><math id="alg2.l2.m1.1" class="ltx_Math" alttext="\mathcal{B}\leftarrow\varnothing" display="inline"><semantics id="alg2.l2.m1.1a"><mrow id="alg2.l2.m1.1.1" xref="alg2.l2.m1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg2.l2.m1.1.1.2" xref="alg2.l2.m1.1.1.2.cmml">ℬ</mi><mo mathsize="90%" stretchy="false" id="alg2.l2.m1.1.1.1" xref="alg2.l2.m1.1.1.1.cmml">←</mo><mi mathsize="90%" mathvariant="normal" id="alg2.l2.m1.1.1.3" xref="alg2.l2.m1.1.1.3.cmml">∅</mi></mrow><annotation-xml encoding="MathML-Content" id="alg2.l2.m1.1b"><apply id="alg2.l2.m1.1.1.cmml" xref="alg2.l2.m1.1.1"><ci id="alg2.l2.m1.1.1.1.cmml" xref="alg2.l2.m1.1.1.1">←</ci><ci id="alg2.l2.m1.1.1.2.cmml" xref="alg2.l2.m1.1.1.2">ℬ</ci><emptyset id="alg2.l2.m1.1.1.3.cmml" xref="alg2.l2.m1.1.1.3"></emptyset></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l2.m1.1c">\mathcal{B}\leftarrow\varnothing</annotation></semantics></math><span id="alg2.l2.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l3" class="ltx_listingline">
<span id="alg2.l3.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg2.l3.2" class="ltx_text ltx_font_bold" style="font-size:90%;">while</span><span id="alg2.l3.3" class="ltx_text" style="font-size:90%;"> not converged </span><span id="alg2.l3.4" class="ltx_text ltx_font_bold" style="font-size:90%;">do</span><span id="alg2.l3.5" class="ltx_text" style="font-size:90%;">


</span>
</div>
<div id="alg2.l4" class="ltx_listingline">
<span id="alg2.l4.4" class="ltx_text" style="font-size:90%;">    </span><em id="alg2.l4.3" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg2.l4.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg2.l4.1.m1.1a"><mo id="alg2.l4.1.m1.1.1" xref="alg2.l4.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg2.l4.1.m1.1b"><ci id="alg2.l4.1.m1.1.1.cmml" xref="alg2.l4.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l4.1.m1.1c">\triangleright</annotation></semantics></math> Sample data and candidate solutions from <math id="alg2.l4.2.m2.1" class="ltx_Math" alttext="p_{d}" display="inline"><semantics id="alg2.l4.2.m2.1a"><msub id="alg2.l4.2.m2.1.1" xref="alg2.l4.2.m2.1.1.cmml"><mi id="alg2.l4.2.m2.1.1.2" xref="alg2.l4.2.m2.1.1.2.cmml">p</mi><mi id="alg2.l4.2.m2.1.1.3" xref="alg2.l4.2.m2.1.1.3.cmml">d</mi></msub><annotation-xml encoding="MathML-Content" id="alg2.l4.2.m2.1b"><apply id="alg2.l4.2.m2.1.1.cmml" xref="alg2.l4.2.m2.1.1"><csymbol cd="ambiguous" id="alg2.l4.2.m2.1.1.1.cmml" xref="alg2.l4.2.m2.1.1">subscript</csymbol><ci id="alg2.l4.2.m2.1.1.2.cmml" xref="alg2.l4.2.m2.1.1.2">𝑝</ci><ci id="alg2.l4.2.m2.1.1.3.cmml" xref="alg2.l4.2.m2.1.1.3">𝑑</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l4.2.m2.1c">p_{d}</annotation></semantics></math> and replay buffer <math id="alg2.l4.3.m3.1" class="ltx_Math" alttext="\mathcal{B}" display="inline"><semantics id="alg2.l4.3.m3.1a"><mi class="ltx_font_mathcaligraphic" id="alg2.l4.3.m3.1.1" xref="alg2.l4.3.m3.1.1.cmml">ℬ</mi><annotation-xml encoding="MathML-Content" id="alg2.l4.3.m3.1b"><ci id="alg2.l4.3.m3.1.1.cmml" xref="alg2.l4.3.m3.1.1">ℬ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l4.3.m3.1c">\mathcal{B}</annotation></semantics></math></em><span id="alg2.l4.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l5" class="ltx_listingline">
<span id="alg2.l5.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg2.l5.m1.2" class="ltx_Math" alttext="{\bm{x}}_{i},{\bm{y}}_{i}\sim p_{D}" display="inline"><semantics id="alg2.l5.m1.2a"><mrow id="alg2.l5.m1.2.2" xref="alg2.l5.m1.2.2.cmml"><mrow id="alg2.l5.m1.2.2.2.2" xref="alg2.l5.m1.2.2.2.3.cmml"><msub id="alg2.l5.m1.1.1.1.1.1" xref="alg2.l5.m1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg2.l5.m1.1.1.1.1.1.2" xref="alg2.l5.m1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l5.m1.1.1.1.1.1.3" xref="alg2.l5.m1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l5.m1.2.2.2.2.3" xref="alg2.l5.m1.2.2.2.3.cmml">,</mo><msub id="alg2.l5.m1.2.2.2.2.2" xref="alg2.l5.m1.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l5.m1.2.2.2.2.2.2" xref="alg2.l5.m1.2.2.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l5.m1.2.2.2.2.2.3" xref="alg2.l5.m1.2.2.2.2.2.3.cmml">i</mi></msub></mrow><mo mathsize="90%" id="alg2.l5.m1.2.2.3" xref="alg2.l5.m1.2.2.3.cmml">∼</mo><msub id="alg2.l5.m1.2.2.4" xref="alg2.l5.m1.2.2.4.cmml"><mi mathsize="90%" id="alg2.l5.m1.2.2.4.2" xref="alg2.l5.m1.2.2.4.2.cmml">p</mi><mi mathsize="90%" id="alg2.l5.m1.2.2.4.3" xref="alg2.l5.m1.2.2.4.3.cmml">D</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="alg2.l5.m1.2b"><apply id="alg2.l5.m1.2.2.cmml" xref="alg2.l5.m1.2.2"><csymbol cd="latexml" id="alg2.l5.m1.2.2.3.cmml" xref="alg2.l5.m1.2.2.3">similar-to</csymbol><list id="alg2.l5.m1.2.2.2.3.cmml" xref="alg2.l5.m1.2.2.2.2"><apply id="alg2.l5.m1.1.1.1.1.1.cmml" xref="alg2.l5.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l5.m1.1.1.1.1.1.1.cmml" xref="alg2.l5.m1.1.1.1.1.1">subscript</csymbol><ci id="alg2.l5.m1.1.1.1.1.1.2.cmml" xref="alg2.l5.m1.1.1.1.1.1.2">𝒙</ci><ci id="alg2.l5.m1.1.1.1.1.1.3.cmml" xref="alg2.l5.m1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg2.l5.m1.2.2.2.2.2.cmml" xref="alg2.l5.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l5.m1.2.2.2.2.2.1.cmml" xref="alg2.l5.m1.2.2.2.2.2">subscript</csymbol><ci id="alg2.l5.m1.2.2.2.2.2.2.cmml" xref="alg2.l5.m1.2.2.2.2.2.2">𝒚</ci><ci id="alg2.l5.m1.2.2.2.2.2.3.cmml" xref="alg2.l5.m1.2.2.2.2.2.3">𝑖</ci></apply></list><apply id="alg2.l5.m1.2.2.4.cmml" xref="alg2.l5.m1.2.2.4"><csymbol cd="ambiguous" id="alg2.l5.m1.2.2.4.1.cmml" xref="alg2.l5.m1.2.2.4">subscript</csymbol><ci id="alg2.l5.m1.2.2.4.2.cmml" xref="alg2.l5.m1.2.2.4.2">𝑝</ci><ci id="alg2.l5.m1.2.2.4.3.cmml" xref="alg2.l5.m1.2.2.4.3">𝐷</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l5.m1.2c">{\bm{x}}_{i},{\bm{y}}_{i}\sim p_{D}</annotation></semantics></math><span id="alg2.l5.2" class="ltx_text" style="font-size:90%;">, </span><math id="alg2.l5.m2.4" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}^{0},\tilde{{\bm{z}}}_{i}^{0}\sim\mathcal{U}(-1,1)" display="inline"><semantics id="alg2.l5.m2.4a"><mrow id="alg2.l5.m2.4.4" xref="alg2.l5.m2.4.4.cmml"><mrow id="alg2.l5.m2.3.3.2.2" xref="alg2.l5.m2.3.3.2.3.cmml"><msubsup id="alg2.l5.m2.2.2.1.1.1" xref="alg2.l5.m2.2.2.1.1.1.cmml"><mover accent="true" id="alg2.l5.m2.2.2.1.1.1.2.2" xref="alg2.l5.m2.2.2.1.1.1.2.2.cmml"><mi mathsize="90%" id="alg2.l5.m2.2.2.1.1.1.2.2.2" xref="alg2.l5.m2.2.2.1.1.1.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l5.m2.2.2.1.1.1.2.2.1" xref="alg2.l5.m2.2.2.1.1.1.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l5.m2.2.2.1.1.1.2.3" xref="alg2.l5.m2.2.2.1.1.1.2.3.cmml">i</mi><mn mathsize="90%" id="alg2.l5.m2.2.2.1.1.1.3" xref="alg2.l5.m2.2.2.1.1.1.3.cmml">0</mn></msubsup><mo mathsize="90%" id="alg2.l5.m2.3.3.2.2.3" xref="alg2.l5.m2.3.3.2.3.cmml">,</mo><msubsup id="alg2.l5.m2.3.3.2.2.2" xref="alg2.l5.m2.3.3.2.2.2.cmml"><mover accent="true" id="alg2.l5.m2.3.3.2.2.2.2.2" xref="alg2.l5.m2.3.3.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l5.m2.3.3.2.2.2.2.2.2" xref="alg2.l5.m2.3.3.2.2.2.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l5.m2.3.3.2.2.2.2.2.1" xref="alg2.l5.m2.3.3.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l5.m2.3.3.2.2.2.2.3" xref="alg2.l5.m2.3.3.2.2.2.2.3.cmml">i</mi><mn mathsize="90%" id="alg2.l5.m2.3.3.2.2.2.3" xref="alg2.l5.m2.3.3.2.2.2.3.cmml">0</mn></msubsup></mrow><mo mathsize="90%" id="alg2.l5.m2.4.4.4" xref="alg2.l5.m2.4.4.4.cmml">∼</mo><mrow id="alg2.l5.m2.4.4.3" xref="alg2.l5.m2.4.4.3.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg2.l5.m2.4.4.3.3" xref="alg2.l5.m2.4.4.3.3.cmml">𝒰</mi><mo lspace="0em" rspace="0em" id="alg2.l5.m2.4.4.3.2" xref="alg2.l5.m2.4.4.3.2.cmml">​</mo><mrow id="alg2.l5.m2.4.4.3.1.1" xref="alg2.l5.m2.4.4.3.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l5.m2.4.4.3.1.1.2" xref="alg2.l5.m2.4.4.3.1.2.cmml">(</mo><mrow id="alg2.l5.m2.4.4.3.1.1.1" xref="alg2.l5.m2.4.4.3.1.1.1.cmml"><mo mathsize="90%" id="alg2.l5.m2.4.4.3.1.1.1a" xref="alg2.l5.m2.4.4.3.1.1.1.cmml">−</mo><mn mathsize="90%" id="alg2.l5.m2.4.4.3.1.1.1.2" xref="alg2.l5.m2.4.4.3.1.1.1.2.cmml">1</mn></mrow><mo mathsize="90%" id="alg2.l5.m2.4.4.3.1.1.3" xref="alg2.l5.m2.4.4.3.1.2.cmml">,</mo><mn mathsize="90%" id="alg2.l5.m2.1.1" xref="alg2.l5.m2.1.1.cmml">1</mn><mo maxsize="90%" minsize="90%" id="alg2.l5.m2.4.4.3.1.1.4" xref="alg2.l5.m2.4.4.3.1.2.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l5.m2.4b"><apply id="alg2.l5.m2.4.4.cmml" xref="alg2.l5.m2.4.4"><csymbol cd="latexml" id="alg2.l5.m2.4.4.4.cmml" xref="alg2.l5.m2.4.4.4">similar-to</csymbol><list id="alg2.l5.m2.3.3.2.3.cmml" xref="alg2.l5.m2.3.3.2.2"><apply id="alg2.l5.m2.2.2.1.1.1.cmml" xref="alg2.l5.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="alg2.l5.m2.2.2.1.1.1.1.cmml" xref="alg2.l5.m2.2.2.1.1.1">superscript</csymbol><apply id="alg2.l5.m2.2.2.1.1.1.2.cmml" xref="alg2.l5.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="alg2.l5.m2.2.2.1.1.1.2.1.cmml" xref="alg2.l5.m2.2.2.1.1.1">subscript</csymbol><apply id="alg2.l5.m2.2.2.1.1.1.2.2.cmml" xref="alg2.l5.m2.2.2.1.1.1.2.2"><ci id="alg2.l5.m2.2.2.1.1.1.2.2.1.cmml" xref="alg2.l5.m2.2.2.1.1.1.2.2.1">~</ci><ci id="alg2.l5.m2.2.2.1.1.1.2.2.2.cmml" xref="alg2.l5.m2.2.2.1.1.1.2.2.2">𝒚</ci></apply><ci id="alg2.l5.m2.2.2.1.1.1.2.3.cmml" xref="alg2.l5.m2.2.2.1.1.1.2.3">𝑖</ci></apply><cn type="integer" id="alg2.l5.m2.2.2.1.1.1.3.cmml" xref="alg2.l5.m2.2.2.1.1.1.3">0</cn></apply><apply id="alg2.l5.m2.3.3.2.2.2.cmml" xref="alg2.l5.m2.3.3.2.2.2"><csymbol cd="ambiguous" id="alg2.l5.m2.3.3.2.2.2.1.cmml" xref="alg2.l5.m2.3.3.2.2.2">superscript</csymbol><apply id="alg2.l5.m2.3.3.2.2.2.2.cmml" xref="alg2.l5.m2.3.3.2.2.2"><csymbol cd="ambiguous" id="alg2.l5.m2.3.3.2.2.2.2.1.cmml" xref="alg2.l5.m2.3.3.2.2.2">subscript</csymbol><apply id="alg2.l5.m2.3.3.2.2.2.2.2.cmml" xref="alg2.l5.m2.3.3.2.2.2.2.2"><ci id="alg2.l5.m2.3.3.2.2.2.2.2.1.cmml" xref="alg2.l5.m2.3.3.2.2.2.2.2.1">~</ci><ci id="alg2.l5.m2.3.3.2.2.2.2.2.2.cmml" xref="alg2.l5.m2.3.3.2.2.2.2.2.2">𝒛</ci></apply><ci id="alg2.l5.m2.3.3.2.2.2.2.3.cmml" xref="alg2.l5.m2.3.3.2.2.2.2.3">𝑖</ci></apply><cn type="integer" id="alg2.l5.m2.3.3.2.2.2.3.cmml" xref="alg2.l5.m2.3.3.2.2.2.3">0</cn></apply></list><apply id="alg2.l5.m2.4.4.3.cmml" xref="alg2.l5.m2.4.4.3"><times id="alg2.l5.m2.4.4.3.2.cmml" xref="alg2.l5.m2.4.4.3.2"></times><ci id="alg2.l5.m2.4.4.3.3.cmml" xref="alg2.l5.m2.4.4.3.3">𝒰</ci><interval closure="open" id="alg2.l5.m2.4.4.3.1.2.cmml" xref="alg2.l5.m2.4.4.3.1.1"><apply id="alg2.l5.m2.4.4.3.1.1.1.cmml" xref="alg2.l5.m2.4.4.3.1.1.1"><minus id="alg2.l5.m2.4.4.3.1.1.1.1.cmml" xref="alg2.l5.m2.4.4.3.1.1.1"></minus><cn type="integer" id="alg2.l5.m2.4.4.3.1.1.1.2.cmml" xref="alg2.l5.m2.4.4.3.1.1.1.2">1</cn></apply><cn type="integer" id="alg2.l5.m2.1.1.cmml" xref="alg2.l5.m2.1.1">1</cn></interval></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l5.m2.4c">\tilde{{\bm{y}}}_{i}^{0},\tilde{{\bm{z}}}_{i}^{0}\sim\mathcal{U}(-1,1)</annotation></semantics></math><span id="alg2.l5.3" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l6" class="ltx_listingline">
<span id="alg2.l6.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg2.l6.m1.4" class="ltx_Math" alttext="{\bm{x}}_{i}^{b},{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{b},\tilde{{\bm{z}}}_{i}^{b}\sim B" display="inline"><semantics id="alg2.l6.m1.4a"><mrow id="alg2.l6.m1.4.4" xref="alg2.l6.m1.4.4.cmml"><mrow id="alg2.l6.m1.4.4.4.4" xref="alg2.l6.m1.4.4.4.5.cmml"><msubsup id="alg2.l6.m1.1.1.1.1.1" xref="alg2.l6.m1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg2.l6.m1.1.1.1.1.1.2.2" xref="alg2.l6.m1.1.1.1.1.1.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l6.m1.1.1.1.1.1.2.3" xref="alg2.l6.m1.1.1.1.1.1.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l6.m1.1.1.1.1.1.3" xref="alg2.l6.m1.1.1.1.1.1.3.cmml">b</mi></msubsup><mo mathsize="90%" id="alg2.l6.m1.4.4.4.4.5" xref="alg2.l6.m1.4.4.4.5.cmml">,</mo><msubsup id="alg2.l6.m1.2.2.2.2.2" xref="alg2.l6.m1.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l6.m1.2.2.2.2.2.2.2" xref="alg2.l6.m1.2.2.2.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l6.m1.2.2.2.2.2.2.3" xref="alg2.l6.m1.2.2.2.2.2.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l6.m1.2.2.2.2.2.3" xref="alg2.l6.m1.2.2.2.2.2.3.cmml">b</mi></msubsup><mo mathsize="90%" id="alg2.l6.m1.4.4.4.4.6" xref="alg2.l6.m1.4.4.4.5.cmml">,</mo><msubsup id="alg2.l6.m1.3.3.3.3.3" xref="alg2.l6.m1.3.3.3.3.3.cmml"><mover accent="true" id="alg2.l6.m1.3.3.3.3.3.2.2" xref="alg2.l6.m1.3.3.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg2.l6.m1.3.3.3.3.3.2.2.2" xref="alg2.l6.m1.3.3.3.3.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l6.m1.3.3.3.3.3.2.2.1" xref="alg2.l6.m1.3.3.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l6.m1.3.3.3.3.3.2.3" xref="alg2.l6.m1.3.3.3.3.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l6.m1.3.3.3.3.3.3" xref="alg2.l6.m1.3.3.3.3.3.3.cmml">b</mi></msubsup><mo mathsize="90%" id="alg2.l6.m1.4.4.4.4.7" xref="alg2.l6.m1.4.4.4.5.cmml">,</mo><msubsup id="alg2.l6.m1.4.4.4.4.4" xref="alg2.l6.m1.4.4.4.4.4.cmml"><mover accent="true" id="alg2.l6.m1.4.4.4.4.4.2.2" xref="alg2.l6.m1.4.4.4.4.4.2.2.cmml"><mi mathsize="90%" id="alg2.l6.m1.4.4.4.4.4.2.2.2" xref="alg2.l6.m1.4.4.4.4.4.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l6.m1.4.4.4.4.4.2.2.1" xref="alg2.l6.m1.4.4.4.4.4.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l6.m1.4.4.4.4.4.2.3" xref="alg2.l6.m1.4.4.4.4.4.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l6.m1.4.4.4.4.4.3" xref="alg2.l6.m1.4.4.4.4.4.3.cmml">b</mi></msubsup></mrow><mo mathsize="90%" id="alg2.l6.m1.4.4.5" xref="alg2.l6.m1.4.4.5.cmml">∼</mo><mi mathsize="90%" id="alg2.l6.m1.4.4.6" xref="alg2.l6.m1.4.4.6.cmml">B</mi></mrow><annotation-xml encoding="MathML-Content" id="alg2.l6.m1.4b"><apply id="alg2.l6.m1.4.4.cmml" xref="alg2.l6.m1.4.4"><csymbol cd="latexml" id="alg2.l6.m1.4.4.5.cmml" xref="alg2.l6.m1.4.4.5">similar-to</csymbol><list id="alg2.l6.m1.4.4.4.5.cmml" xref="alg2.l6.m1.4.4.4.4"><apply id="alg2.l6.m1.1.1.1.1.1.cmml" xref="alg2.l6.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.1.1.1.1.cmml" xref="alg2.l6.m1.1.1.1.1.1">superscript</csymbol><apply id="alg2.l6.m1.1.1.1.1.1.2.cmml" xref="alg2.l6.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l6.m1.1.1.1.1.1.2.1.cmml" xref="alg2.l6.m1.1.1.1.1.1">subscript</csymbol><ci id="alg2.l6.m1.1.1.1.1.1.2.2.cmml" xref="alg2.l6.m1.1.1.1.1.1.2.2">𝒙</ci><ci id="alg2.l6.m1.1.1.1.1.1.2.3.cmml" xref="alg2.l6.m1.1.1.1.1.1.2.3">𝑖</ci></apply><ci id="alg2.l6.m1.1.1.1.1.1.3.cmml" xref="alg2.l6.m1.1.1.1.1.1.3">𝑏</ci></apply><apply id="alg2.l6.m1.2.2.2.2.2.cmml" xref="alg2.l6.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l6.m1.2.2.2.2.2.1.cmml" xref="alg2.l6.m1.2.2.2.2.2">superscript</csymbol><apply id="alg2.l6.m1.2.2.2.2.2.2.cmml" xref="alg2.l6.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l6.m1.2.2.2.2.2.2.1.cmml" xref="alg2.l6.m1.2.2.2.2.2">subscript</csymbol><ci id="alg2.l6.m1.2.2.2.2.2.2.2.cmml" xref="alg2.l6.m1.2.2.2.2.2.2.2">𝒚</ci><ci id="alg2.l6.m1.2.2.2.2.2.2.3.cmml" xref="alg2.l6.m1.2.2.2.2.2.2.3">𝑖</ci></apply><ci id="alg2.l6.m1.2.2.2.2.2.3.cmml" xref="alg2.l6.m1.2.2.2.2.2.3">𝑏</ci></apply><apply id="alg2.l6.m1.3.3.3.3.3.cmml" xref="alg2.l6.m1.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l6.m1.3.3.3.3.3.1.cmml" xref="alg2.l6.m1.3.3.3.3.3">superscript</csymbol><apply id="alg2.l6.m1.3.3.3.3.3.2.cmml" xref="alg2.l6.m1.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l6.m1.3.3.3.3.3.2.1.cmml" xref="alg2.l6.m1.3.3.3.3.3">subscript</csymbol><apply id="alg2.l6.m1.3.3.3.3.3.2.2.cmml" xref="alg2.l6.m1.3.3.3.3.3.2.2"><ci id="alg2.l6.m1.3.3.3.3.3.2.2.1.cmml" xref="alg2.l6.m1.3.3.3.3.3.2.2.1">~</ci><ci id="alg2.l6.m1.3.3.3.3.3.2.2.2.cmml" xref="alg2.l6.m1.3.3.3.3.3.2.2.2">𝒚</ci></apply><ci id="alg2.l6.m1.3.3.3.3.3.2.3.cmml" xref="alg2.l6.m1.3.3.3.3.3.2.3">𝑖</ci></apply><ci id="alg2.l6.m1.3.3.3.3.3.3.cmml" xref="alg2.l6.m1.3.3.3.3.3.3">𝑏</ci></apply><apply id="alg2.l6.m1.4.4.4.4.4.cmml" xref="alg2.l6.m1.4.4.4.4.4"><csymbol cd="ambiguous" id="alg2.l6.m1.4.4.4.4.4.1.cmml" xref="alg2.l6.m1.4.4.4.4.4">superscript</csymbol><apply id="alg2.l6.m1.4.4.4.4.4.2.cmml" xref="alg2.l6.m1.4.4.4.4.4"><csymbol cd="ambiguous" id="alg2.l6.m1.4.4.4.4.4.2.1.cmml" xref="alg2.l6.m1.4.4.4.4.4">subscript</csymbol><apply id="alg2.l6.m1.4.4.4.4.4.2.2.cmml" xref="alg2.l6.m1.4.4.4.4.4.2.2"><ci id="alg2.l6.m1.4.4.4.4.4.2.2.1.cmml" xref="alg2.l6.m1.4.4.4.4.4.2.2.1">~</ci><ci id="alg2.l6.m1.4.4.4.4.4.2.2.2.cmml" xref="alg2.l6.m1.4.4.4.4.4.2.2.2">𝒛</ci></apply><ci id="alg2.l6.m1.4.4.4.4.4.2.3.cmml" xref="alg2.l6.m1.4.4.4.4.4.2.3">𝑖</ci></apply><ci id="alg2.l6.m1.4.4.4.4.4.3.cmml" xref="alg2.l6.m1.4.4.4.4.4.3">𝑏</ci></apply></list><ci id="alg2.l6.m1.4.4.6.cmml" xref="alg2.l6.m1.4.4.6">𝐵</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l6.m1.4c">{\bm{x}}_{i}^{b},{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{b},\tilde{{\bm{z}}}_{i}^{b}\sim B</annotation></semantics></math><span id="alg2.l6.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l7" class="ltx_listingline">
<span id="alg2.l7.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg2.l7.m1.2" class="ltx_Math" alttext="{\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{0},\tilde{{\bm{z}}}_{i}^{0}\leftarrow{\bm{x}}_{i}\cup{\bm{x}}_{i}^{b},{\bm{y}}_{i}\cup{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{0}\cup\tilde{{\bm{y}}}_{i}^{b},\tilde{{\bm{z}}}_{i}^{0}\cup\tilde{{\bm{z}}}_{i}^{b}" display="inline"><semantics id="alg2.l7.m1.2a"><mrow id="alg2.l7.m1.2.2.2" xref="alg2.l7.m1.2.2.3.cmml"><mrow id="alg2.l7.m1.1.1.1.1" xref="alg2.l7.m1.1.1.1.1.cmml"><mrow id="alg2.l7.m1.1.1.1.1.4.4" xref="alg2.l7.m1.1.1.1.1.4.5.cmml"><msub id="alg2.l7.m1.1.1.1.1.1.1.1" xref="alg2.l7.m1.1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.1.1.1.2" xref="alg2.l7.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.1.1.1.3" xref="alg2.l7.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.5" xref="alg2.l7.m1.1.1.1.1.4.5.cmml">,</mo><msub id="alg2.l7.m1.1.1.1.1.2.2.2" xref="alg2.l7.m1.1.1.1.1.2.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.2.2.2.2" xref="alg2.l7.m1.1.1.1.1.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.2.2.2.3" xref="alg2.l7.m1.1.1.1.1.2.2.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.6" xref="alg2.l7.m1.1.1.1.1.4.5.cmml">,</mo><msubsup id="alg2.l7.m1.1.1.1.1.3.3.3" xref="alg2.l7.m1.1.1.1.1.3.3.3.cmml"><mover accent="true" id="alg2.l7.m1.1.1.1.1.3.3.3.2.2" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.3.3.3.2.2.2" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l7.m1.1.1.1.1.3.3.3.2.2.1" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.3.3.3.2.3" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.3.cmml">i</mi><mn mathsize="90%" id="alg2.l7.m1.1.1.1.1.3.3.3.3" xref="alg2.l7.m1.1.1.1.1.3.3.3.3.cmml">0</mn></msubsup><mo mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.7" xref="alg2.l7.m1.1.1.1.1.4.5.cmml">,</mo><msubsup id="alg2.l7.m1.1.1.1.1.4.4.4" xref="alg2.l7.m1.1.1.1.1.4.4.4.cmml"><mover accent="true" id="alg2.l7.m1.1.1.1.1.4.4.4.2.2" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.4.2.2.2" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.4.2.2.1" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.4.2.3" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.3.cmml">i</mi><mn mathsize="90%" id="alg2.l7.m1.1.1.1.1.4.4.4.3" xref="alg2.l7.m1.1.1.1.1.4.4.4.3.cmml">0</mn></msubsup></mrow><mo mathsize="90%" stretchy="false" id="alg2.l7.m1.1.1.1.1.5" xref="alg2.l7.m1.1.1.1.1.5.cmml">←</mo><mrow id="alg2.l7.m1.1.1.1.1.6" xref="alg2.l7.m1.1.1.1.1.6.cmml"><msub id="alg2.l7.m1.1.1.1.1.6.2" xref="alg2.l7.m1.1.1.1.1.6.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.6.2.2" xref="alg2.l7.m1.1.1.1.1.6.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.6.2.3" xref="alg2.l7.m1.1.1.1.1.6.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l7.m1.1.1.1.1.6.1" xref="alg2.l7.m1.1.1.1.1.6.1.cmml">∪</mo><msubsup id="alg2.l7.m1.1.1.1.1.6.3" xref="alg2.l7.m1.1.1.1.1.6.3.cmml"><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.6.3.2.2" xref="alg2.l7.m1.1.1.1.1.6.3.2.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.6.3.2.3" xref="alg2.l7.m1.1.1.1.1.6.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l7.m1.1.1.1.1.6.3.3" xref="alg2.l7.m1.1.1.1.1.6.3.3.cmml">b</mi></msubsup></mrow></mrow><mo mathsize="90%" id="alg2.l7.m1.2.2.2.3" xref="alg2.l7.m1.2.2.3a.cmml">,</mo><mrow id="alg2.l7.m1.2.2.2.2.3" xref="alg2.l7.m1.2.2.2.2.4.cmml"><mrow id="alg2.l7.m1.2.2.2.2.1.1" xref="alg2.l7.m1.2.2.2.2.1.1.cmml"><msub id="alg2.l7.m1.2.2.2.2.1.1.2" xref="alg2.l7.m1.2.2.2.2.1.1.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.1.1.2.2" xref="alg2.l7.m1.2.2.2.2.1.1.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.1.1.2.3" xref="alg2.l7.m1.2.2.2.2.1.1.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.1.1.1" xref="alg2.l7.m1.2.2.2.2.1.1.1.cmml">∪</mo><msubsup id="alg2.l7.m1.2.2.2.2.1.1.3" xref="alg2.l7.m1.2.2.2.2.1.1.3.cmml"><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.1.1.3.2.2" xref="alg2.l7.m1.2.2.2.2.1.1.3.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.1.1.3.2.3" xref="alg2.l7.m1.2.2.2.2.1.1.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.1.1.3.3" xref="alg2.l7.m1.2.2.2.2.1.1.3.3.cmml">b</mi></msubsup></mrow><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.4" xref="alg2.l7.m1.2.2.2.2.4.cmml">,</mo><mrow id="alg2.l7.m1.2.2.2.2.2.2" xref="alg2.l7.m1.2.2.2.2.2.2.cmml"><msubsup id="alg2.l7.m1.2.2.2.2.2.2.2" xref="alg2.l7.m1.2.2.2.2.2.2.2.cmml"><mover accent="true" id="alg2.l7.m1.2.2.2.2.2.2.2.2.2" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.2.2.2.2" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.2.2.2.1" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.2.2.3" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.3.cmml">i</mi><mn mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.2.3" xref="alg2.l7.m1.2.2.2.2.2.2.2.3.cmml">0</mn></msubsup><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.1" xref="alg2.l7.m1.2.2.2.2.2.2.1.cmml">∪</mo><msubsup id="alg2.l7.m1.2.2.2.2.2.2.3" xref="alg2.l7.m1.2.2.2.2.2.2.3.cmml"><mover accent="true" id="alg2.l7.m1.2.2.2.2.2.2.3.2.2" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.3.2.2.2" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.3.2.2.1" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.3.2.3" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.2.2.3.3" xref="alg2.l7.m1.2.2.2.2.2.2.3.3.cmml">b</mi></msubsup></mrow><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.5" xref="alg2.l7.m1.2.2.2.2.4.cmml">,</mo><mrow id="alg2.l7.m1.2.2.2.2.3.3" xref="alg2.l7.m1.2.2.2.2.3.3.cmml"><msubsup id="alg2.l7.m1.2.2.2.2.3.3.2" xref="alg2.l7.m1.2.2.2.2.3.3.2.cmml"><mover accent="true" id="alg2.l7.m1.2.2.2.2.3.3.2.2.2" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.2.2.2.2" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.2.2.2.1" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.2.2.3" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.3.cmml">i</mi><mn mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.2.3" xref="alg2.l7.m1.2.2.2.2.3.3.2.3.cmml">0</mn></msubsup><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.1" xref="alg2.l7.m1.2.2.2.2.3.3.1.cmml">∪</mo><msubsup id="alg2.l7.m1.2.2.2.2.3.3.3" xref="alg2.l7.m1.2.2.2.2.3.3.3.cmml"><mover accent="true" id="alg2.l7.m1.2.2.2.2.3.3.3.2.2" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.3.2.2.2" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.3.2.2.1" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.3.2.3" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l7.m1.2.2.2.2.3.3.3.3" xref="alg2.l7.m1.2.2.2.2.3.3.3.3.cmml">b</mi></msubsup></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l7.m1.2b"><apply id="alg2.l7.m1.2.2.3.cmml" xref="alg2.l7.m1.2.2.2"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.3a.cmml" xref="alg2.l7.m1.2.2.2.3">formulae-sequence</csymbol><apply id="alg2.l7.m1.1.1.1.1.cmml" xref="alg2.l7.m1.1.1.1.1"><ci id="alg2.l7.m1.1.1.1.1.5.cmml" xref="alg2.l7.m1.1.1.1.1.5">←</ci><list id="alg2.l7.m1.1.1.1.1.4.5.cmml" xref="alg2.l7.m1.1.1.1.1.4.4"><apply id="alg2.l7.m1.1.1.1.1.1.1.1.cmml" xref="alg2.l7.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.1.1.1.1.cmml" xref="alg2.l7.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="alg2.l7.m1.1.1.1.1.1.1.1.2.cmml" xref="alg2.l7.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg2.l7.m1.1.1.1.1.1.1.1.3.cmml" xref="alg2.l7.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg2.l7.m1.1.1.1.1.2.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.2.2.2"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.2.2.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.2.2.2">subscript</csymbol><ci id="alg2.l7.m1.1.1.1.1.2.2.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.2.2.2.2">𝒚</ci><ci id="alg2.l7.m1.1.1.1.1.2.2.2.3.cmml" xref="alg2.l7.m1.1.1.1.1.2.2.2.3">𝑖</ci></apply><apply id="alg2.l7.m1.1.1.1.1.3.3.3.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.3.3.3.1.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3">superscript</csymbol><apply id="alg2.l7.m1.1.1.1.1.3.3.3.2.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.3.3.3.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3">subscript</csymbol><apply id="alg2.l7.m1.1.1.1.1.3.3.3.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.2"><ci id="alg2.l7.m1.1.1.1.1.3.3.3.2.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.2.1">~</ci><ci id="alg2.l7.m1.1.1.1.1.3.3.3.2.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.2.2">𝒚</ci></apply><ci id="alg2.l7.m1.1.1.1.1.3.3.3.2.3.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3.2.3">𝑖</ci></apply><cn type="integer" id="alg2.l7.m1.1.1.1.1.3.3.3.3.cmml" xref="alg2.l7.m1.1.1.1.1.3.3.3.3">0</cn></apply><apply id="alg2.l7.m1.1.1.1.1.4.4.4.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.4.4.4.1.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4">superscript</csymbol><apply id="alg2.l7.m1.1.1.1.1.4.4.4.2.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.4.4.4.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4">subscript</csymbol><apply id="alg2.l7.m1.1.1.1.1.4.4.4.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.2"><ci id="alg2.l7.m1.1.1.1.1.4.4.4.2.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.2.1">~</ci><ci id="alg2.l7.m1.1.1.1.1.4.4.4.2.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.2.2">𝒛</ci></apply><ci id="alg2.l7.m1.1.1.1.1.4.4.4.2.3.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4.2.3">𝑖</ci></apply><cn type="integer" id="alg2.l7.m1.1.1.1.1.4.4.4.3.cmml" xref="alg2.l7.m1.1.1.1.1.4.4.4.3">0</cn></apply></list><apply id="alg2.l7.m1.1.1.1.1.6.cmml" xref="alg2.l7.m1.1.1.1.1.6"><union id="alg2.l7.m1.1.1.1.1.6.1.cmml" xref="alg2.l7.m1.1.1.1.1.6.1"></union><apply id="alg2.l7.m1.1.1.1.1.6.2.cmml" xref="alg2.l7.m1.1.1.1.1.6.2"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.6.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.6.2">subscript</csymbol><ci id="alg2.l7.m1.1.1.1.1.6.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.6.2.2">𝒙</ci><ci id="alg2.l7.m1.1.1.1.1.6.2.3.cmml" xref="alg2.l7.m1.1.1.1.1.6.2.3">𝑖</ci></apply><apply id="alg2.l7.m1.1.1.1.1.6.3.cmml" xref="alg2.l7.m1.1.1.1.1.6.3"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.6.3.1.cmml" xref="alg2.l7.m1.1.1.1.1.6.3">superscript</csymbol><apply id="alg2.l7.m1.1.1.1.1.6.3.2.cmml" xref="alg2.l7.m1.1.1.1.1.6.3"><csymbol cd="ambiguous" id="alg2.l7.m1.1.1.1.1.6.3.2.1.cmml" xref="alg2.l7.m1.1.1.1.1.6.3">subscript</csymbol><ci id="alg2.l7.m1.1.1.1.1.6.3.2.2.cmml" xref="alg2.l7.m1.1.1.1.1.6.3.2.2">𝒙</ci><ci id="alg2.l7.m1.1.1.1.1.6.3.2.3.cmml" xref="alg2.l7.m1.1.1.1.1.6.3.2.3">𝑖</ci></apply><ci id="alg2.l7.m1.1.1.1.1.6.3.3.cmml" xref="alg2.l7.m1.1.1.1.1.6.3.3">𝑏</ci></apply></apply></apply><list id="alg2.l7.m1.2.2.2.2.4.cmml" xref="alg2.l7.m1.2.2.2.2.3"><apply id="alg2.l7.m1.2.2.2.2.1.1.cmml" xref="alg2.l7.m1.2.2.2.2.1.1"><union id="alg2.l7.m1.2.2.2.2.1.1.1.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.1"></union><apply id="alg2.l7.m1.2.2.2.2.1.1.2.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.2"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.1.1.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.2">subscript</csymbol><ci id="alg2.l7.m1.2.2.2.2.1.1.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.2.2">𝒚</ci><ci id="alg2.l7.m1.2.2.2.2.1.1.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.2.3">𝑖</ci></apply><apply id="alg2.l7.m1.2.2.2.2.1.1.3.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.1.1.3.1.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3">superscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.1.1.3.2.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.1.1.3.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3">subscript</csymbol><ci id="alg2.l7.m1.2.2.2.2.1.1.3.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3.2.2">𝒚</ci><ci id="alg2.l7.m1.2.2.2.2.1.1.3.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3.2.3">𝑖</ci></apply><ci id="alg2.l7.m1.2.2.2.2.1.1.3.3.cmml" xref="alg2.l7.m1.2.2.2.2.1.1.3.3">𝑏</ci></apply></apply><apply id="alg2.l7.m1.2.2.2.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2"><union id="alg2.l7.m1.2.2.2.2.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.1"></union><apply id="alg2.l7.m1.2.2.2.2.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.2.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2">superscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.2.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.2.2.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2">subscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.2.2.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.2"><ci id="alg2.l7.m1.2.2.2.2.2.2.2.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.2.1">~</ci><ci id="alg2.l7.m1.2.2.2.2.2.2.2.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.2.2">𝒚</ci></apply><ci id="alg2.l7.m1.2.2.2.2.2.2.2.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2.2.3">𝑖</ci></apply><cn type="integer" id="alg2.l7.m1.2.2.2.2.2.2.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.2.3">0</cn></apply><apply id="alg2.l7.m1.2.2.2.2.2.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.2.2.3.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3">superscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.2.2.3.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.2.2.3.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3">subscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.2.2.3.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.2"><ci id="alg2.l7.m1.2.2.2.2.2.2.3.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.2.1">~</ci><ci id="alg2.l7.m1.2.2.2.2.2.2.3.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.2.2">𝒚</ci></apply><ci id="alg2.l7.m1.2.2.2.2.2.2.3.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3.2.3">𝑖</ci></apply><ci id="alg2.l7.m1.2.2.2.2.2.2.3.3.cmml" xref="alg2.l7.m1.2.2.2.2.2.2.3.3">𝑏</ci></apply></apply><apply id="alg2.l7.m1.2.2.2.2.3.3.cmml" xref="alg2.l7.m1.2.2.2.2.3.3"><union id="alg2.l7.m1.2.2.2.2.3.3.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.1"></union><apply id="alg2.l7.m1.2.2.2.2.3.3.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.3.3.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2">superscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.3.3.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.3.3.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2">subscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.3.3.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.2"><ci id="alg2.l7.m1.2.2.2.2.3.3.2.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.2.1">~</ci><ci id="alg2.l7.m1.2.2.2.2.3.3.2.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.2.2">𝒛</ci></apply><ci id="alg2.l7.m1.2.2.2.2.3.3.2.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2.2.3">𝑖</ci></apply><cn type="integer" id="alg2.l7.m1.2.2.2.2.3.3.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.2.3">0</cn></apply><apply id="alg2.l7.m1.2.2.2.2.3.3.3.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.3.3.3.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3">superscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.3.3.3.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3"><csymbol cd="ambiguous" id="alg2.l7.m1.2.2.2.2.3.3.3.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3">subscript</csymbol><apply id="alg2.l7.m1.2.2.2.2.3.3.3.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.2"><ci id="alg2.l7.m1.2.2.2.2.3.3.3.2.2.1.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.2.1">~</ci><ci id="alg2.l7.m1.2.2.2.2.3.3.3.2.2.2.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.2.2">𝒛</ci></apply><ci id="alg2.l7.m1.2.2.2.2.3.3.3.2.3.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3.2.3">𝑖</ci></apply><ci id="alg2.l7.m1.2.2.2.2.3.3.3.3.cmml" xref="alg2.l7.m1.2.2.2.2.3.3.3.3">𝑏</ci></apply></apply></list></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l7.m1.2c">{\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{0},\tilde{{\bm{z}}}_{i}^{0}\leftarrow{\bm{x}}_{i}\cup{\bm{x}}_{i}^{b},{\bm{y}}_{i}\cup{\bm{y}}_{i}^{b},\tilde{{\bm{y}}}_{i}^{0}\cup\tilde{{\bm{y}}}_{i}^{b},\tilde{{\bm{z}}}_{i}^{0}\cup\tilde{{\bm{z}}}_{i}^{b}</annotation></semantics></math><span id="alg2.l7.2" class="ltx_text" style="font-size:90%;">

</span>
</div>
<div id="alg2.l8" class="ltx_listingline">
<span id="alg2.l8.2" class="ltx_text" style="font-size:90%;">    </span><em id="alg2.l8.1" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg2.l8.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg2.l8.1.m1.1a"><mo id="alg2.l8.1.m1.1.1" xref="alg2.l8.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg2.l8.1.m1.1b"><ci id="alg2.l8.1.m1.1.1.cmml" xref="alg2.l8.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l8.1.m1.1c">\triangleright</annotation></semantics></math> Generate low energy solutions through optimization:</em><span id="alg2.l8.3" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l9" class="ltx_listingline">
<span id="alg2.l9.1" class="ltx_text" style="font-size:90%;">    </span><span id="alg2.l9.2" class="ltx_text ltx_font_bold" style="font-size:90%;">for</span><span id="alg2.l9.3" class="ltx_text" style="font-size:90%;"> sample step </span><math id="alg2.l9.m1.1" class="ltx_Math" alttext="n=1" display="inline"><semantics id="alg2.l9.m1.1a"><mrow id="alg2.l9.m1.1.1" xref="alg2.l9.m1.1.1.cmml"><mi mathsize="90%" id="alg2.l9.m1.1.1.2" xref="alg2.l9.m1.1.1.2.cmml">n</mi><mo mathsize="90%" id="alg2.l9.m1.1.1.1" xref="alg2.l9.m1.1.1.1.cmml">=</mo><mn mathsize="90%" id="alg2.l9.m1.1.1.3" xref="alg2.l9.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="alg2.l9.m1.1b"><apply id="alg2.l9.m1.1.1.cmml" xref="alg2.l9.m1.1.1"><eq id="alg2.l9.m1.1.1.1.cmml" xref="alg2.l9.m1.1.1.1"></eq><ci id="alg2.l9.m1.1.1.2.cmml" xref="alg2.l9.m1.1.1.2">𝑛</ci><cn type="integer" id="alg2.l9.m1.1.1.3.cmml" xref="alg2.l9.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l9.m1.1c">n=1</annotation></semantics></math><span id="alg2.l9.4" class="ltx_text" style="font-size:90%;"> to </span><math id="alg2.l9.m2.1" class="ltx_Math" alttext="N" display="inline"><semantics id="alg2.l9.m2.1a"><mi mathsize="90%" id="alg2.l9.m2.1.1" xref="alg2.l9.m2.1.1.cmml">N</mi><annotation-xml encoding="MathML-Content" id="alg2.l9.m2.1b"><ci id="alg2.l9.m2.1.1.cmml" xref="alg2.l9.m2.1.1">𝑁</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l9.m2.1c">N</annotation></semantics></math><span id="alg2.l9.5" class="ltx_text" style="font-size:90%;"> </span><span id="alg2.l9.6" class="ltx_text ltx_font_bold" style="font-size:90%;">do</span><span id="alg2.l9.7" class="ltx_text" style="font-size:90%;">


</span>
</div>
<div id="alg2.l10" class="ltx_listingline">
<span id="alg2.l10.1" class="ltx_text" style="font-size:90%;">       </span><math id="alg2.l10.m1.3" class="ltx_Math" alttext="\tilde{{\bm{y}}}_{i}^{n}\leftarrow\tilde{{\bm{y}}}_{i}^{n-1}-\lambda\nabla_{\bm{y}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i}^{n-1},\tilde{{\bm{z}}}_{i}^{n-1})" display="inline"><semantics id="alg2.l10.m1.3a"><mrow id="alg2.l10.m1.3.3" xref="alg2.l10.m1.3.3.cmml"><msubsup id="alg2.l10.m1.3.3.5" xref="alg2.l10.m1.3.3.5.cmml"><mover accent="true" id="alg2.l10.m1.3.3.5.2.2" xref="alg2.l10.m1.3.3.5.2.2.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.5.2.2.2" xref="alg2.l10.m1.3.3.5.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l10.m1.3.3.5.2.2.1" xref="alg2.l10.m1.3.3.5.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l10.m1.3.3.5.2.3" xref="alg2.l10.m1.3.3.5.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l10.m1.3.3.5.3" xref="alg2.l10.m1.3.3.5.3.cmml">n</mi></msubsup><mo mathsize="90%" stretchy="false" id="alg2.l10.m1.3.3.4" xref="alg2.l10.m1.3.3.4.cmml">←</mo><mrow id="alg2.l10.m1.3.3.3" xref="alg2.l10.m1.3.3.3.cmml"><msubsup id="alg2.l10.m1.3.3.3.5" xref="alg2.l10.m1.3.3.3.5.cmml"><mover accent="true" id="alg2.l10.m1.3.3.3.5.2.2" xref="alg2.l10.m1.3.3.3.5.2.2.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.3.5.2.2.2" xref="alg2.l10.m1.3.3.3.5.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l10.m1.3.3.3.5.2.2.1" xref="alg2.l10.m1.3.3.3.5.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l10.m1.3.3.3.5.2.3" xref="alg2.l10.m1.3.3.3.5.2.3.cmml">i</mi><mrow id="alg2.l10.m1.3.3.3.5.3" xref="alg2.l10.m1.3.3.3.5.3.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.3.5.3.2" xref="alg2.l10.m1.3.3.3.5.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l10.m1.3.3.3.5.3.1" xref="alg2.l10.m1.3.3.3.5.3.1.cmml">−</mo><mn mathsize="90%" id="alg2.l10.m1.3.3.3.5.3.3" xref="alg2.l10.m1.3.3.3.5.3.3.cmml">1</mn></mrow></msubsup><mo mathsize="90%" id="alg2.l10.m1.3.3.3.4" xref="alg2.l10.m1.3.3.3.4.cmml">−</mo><mrow id="alg2.l10.m1.3.3.3.3" xref="alg2.l10.m1.3.3.3.3.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.5" xref="alg2.l10.m1.3.3.3.3.5.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="alg2.l10.m1.3.3.3.3.4" xref="alg2.l10.m1.3.3.3.3.4.cmml">​</mo><mrow id="alg2.l10.m1.3.3.3.3.6" xref="alg2.l10.m1.3.3.3.3.6.cmml"><msub id="alg2.l10.m1.3.3.3.3.6.1" xref="alg2.l10.m1.3.3.3.3.6.1.cmml"><mo mathsize="90%" rspace="0.167em" id="alg2.l10.m1.3.3.3.3.6.1.2" xref="alg2.l10.m1.3.3.3.3.6.1.2.cmml">∇</mo><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.6.1.3" xref="alg2.l10.m1.3.3.3.3.6.1.3.cmml">𝒚</mi></msub><msub id="alg2.l10.m1.3.3.3.3.6.2" xref="alg2.l10.m1.3.3.3.3.6.2.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.6.2.2" xref="alg2.l10.m1.3.3.3.3.6.2.2.cmml">E</mi><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.6.2.3" xref="alg2.l10.m1.3.3.3.3.6.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="alg2.l10.m1.3.3.3.3.4a" xref="alg2.l10.m1.3.3.3.3.4.cmml">​</mo><mrow id="alg2.l10.m1.3.3.3.3.3.3" xref="alg2.l10.m1.3.3.3.3.3.4.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.4" xref="alg2.l10.m1.3.3.3.3.3.4.cmml">(</mo><msub id="alg2.l10.m1.1.1.1.1.1.1.1" xref="alg2.l10.m1.1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg2.l10.m1.1.1.1.1.1.1.1.2" xref="alg2.l10.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l10.m1.1.1.1.1.1.1.1.3" xref="alg2.l10.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.5" xref="alg2.l10.m1.3.3.3.3.3.4.cmml">,</mo><msubsup id="alg2.l10.m1.2.2.2.2.2.2.2" xref="alg2.l10.m1.2.2.2.2.2.2.2.cmml"><mover accent="true" id="alg2.l10.m1.2.2.2.2.2.2.2.2.2" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l10.m1.2.2.2.2.2.2.2.2.2.2" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l10.m1.2.2.2.2.2.2.2.2.2.1" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l10.m1.2.2.2.2.2.2.2.2.3" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.3.cmml">i</mi><mrow id="alg2.l10.m1.2.2.2.2.2.2.2.3" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.cmml"><mi mathsize="90%" id="alg2.l10.m1.2.2.2.2.2.2.2.3.2" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l10.m1.2.2.2.2.2.2.2.3.1" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.1.cmml">−</mo><mn mathsize="90%" id="alg2.l10.m1.2.2.2.2.2.2.2.3.3" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.3.cmml">1</mn></mrow></msubsup><mo mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.6" xref="alg2.l10.m1.3.3.3.3.3.4.cmml">,</mo><msubsup id="alg2.l10.m1.3.3.3.3.3.3.3" xref="alg2.l10.m1.3.3.3.3.3.3.3.cmml"><mover accent="true" id="alg2.l10.m1.3.3.3.3.3.3.3.2.2" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.3.2.2.2" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.3.2.2.1" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.3.2.3" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.3.cmml">i</mi><mrow id="alg2.l10.m1.3.3.3.3.3.3.3.3" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.cmml"><mi mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.3.3.2" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.3.3.1" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.1.cmml">−</mo><mn mathsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.3.3.3" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.3.cmml">1</mn></mrow></msubsup><mo maxsize="90%" minsize="90%" id="alg2.l10.m1.3.3.3.3.3.3.7" xref="alg2.l10.m1.3.3.3.3.3.4.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l10.m1.3b"><apply id="alg2.l10.m1.3.3.cmml" xref="alg2.l10.m1.3.3"><ci id="alg2.l10.m1.3.3.4.cmml" xref="alg2.l10.m1.3.3.4">←</ci><apply id="alg2.l10.m1.3.3.5.cmml" xref="alg2.l10.m1.3.3.5"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.5.1.cmml" xref="alg2.l10.m1.3.3.5">superscript</csymbol><apply id="alg2.l10.m1.3.3.5.2.cmml" xref="alg2.l10.m1.3.3.5"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.5.2.1.cmml" xref="alg2.l10.m1.3.3.5">subscript</csymbol><apply id="alg2.l10.m1.3.3.5.2.2.cmml" xref="alg2.l10.m1.3.3.5.2.2"><ci id="alg2.l10.m1.3.3.5.2.2.1.cmml" xref="alg2.l10.m1.3.3.5.2.2.1">~</ci><ci id="alg2.l10.m1.3.3.5.2.2.2.cmml" xref="alg2.l10.m1.3.3.5.2.2.2">𝒚</ci></apply><ci id="alg2.l10.m1.3.3.5.2.3.cmml" xref="alg2.l10.m1.3.3.5.2.3">𝑖</ci></apply><ci id="alg2.l10.m1.3.3.5.3.cmml" xref="alg2.l10.m1.3.3.5.3">𝑛</ci></apply><apply id="alg2.l10.m1.3.3.3.cmml" xref="alg2.l10.m1.3.3.3"><minus id="alg2.l10.m1.3.3.3.4.cmml" xref="alg2.l10.m1.3.3.3.4"></minus><apply id="alg2.l10.m1.3.3.3.5.cmml" xref="alg2.l10.m1.3.3.3.5"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.3.5.1.cmml" xref="alg2.l10.m1.3.3.3.5">superscript</csymbol><apply id="alg2.l10.m1.3.3.3.5.2.cmml" xref="alg2.l10.m1.3.3.3.5"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.3.5.2.1.cmml" xref="alg2.l10.m1.3.3.3.5">subscript</csymbol><apply id="alg2.l10.m1.3.3.3.5.2.2.cmml" xref="alg2.l10.m1.3.3.3.5.2.2"><ci id="alg2.l10.m1.3.3.3.5.2.2.1.cmml" xref="alg2.l10.m1.3.3.3.5.2.2.1">~</ci><ci id="alg2.l10.m1.3.3.3.5.2.2.2.cmml" xref="alg2.l10.m1.3.3.3.5.2.2.2">𝒚</ci></apply><ci id="alg2.l10.m1.3.3.3.5.2.3.cmml" xref="alg2.l10.m1.3.3.3.5.2.3">𝑖</ci></apply><apply id="alg2.l10.m1.3.3.3.5.3.cmml" xref="alg2.l10.m1.3.3.3.5.3"><minus id="alg2.l10.m1.3.3.3.5.3.1.cmml" xref="alg2.l10.m1.3.3.3.5.3.1"></minus><ci id="alg2.l10.m1.3.3.3.5.3.2.cmml" xref="alg2.l10.m1.3.3.3.5.3.2">𝑛</ci><cn type="integer" id="alg2.l10.m1.3.3.3.5.3.3.cmml" xref="alg2.l10.m1.3.3.3.5.3.3">1</cn></apply></apply><apply id="alg2.l10.m1.3.3.3.3.cmml" xref="alg2.l10.m1.3.3.3.3"><times id="alg2.l10.m1.3.3.3.3.4.cmml" xref="alg2.l10.m1.3.3.3.3.4"></times><ci id="alg2.l10.m1.3.3.3.3.5.cmml" xref="alg2.l10.m1.3.3.3.3.5">𝜆</ci><apply id="alg2.l10.m1.3.3.3.3.6.cmml" xref="alg2.l10.m1.3.3.3.3.6"><apply id="alg2.l10.m1.3.3.3.3.6.1.cmml" xref="alg2.l10.m1.3.3.3.3.6.1"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.3.3.6.1.1.cmml" xref="alg2.l10.m1.3.3.3.3.6.1">subscript</csymbol><ci id="alg2.l10.m1.3.3.3.3.6.1.2.cmml" xref="alg2.l10.m1.3.3.3.3.6.1.2">∇</ci><ci id="alg2.l10.m1.3.3.3.3.6.1.3.cmml" xref="alg2.l10.m1.3.3.3.3.6.1.3">𝒚</ci></apply><apply id="alg2.l10.m1.3.3.3.3.6.2.cmml" xref="alg2.l10.m1.3.3.3.3.6.2"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.3.3.6.2.1.cmml" xref="alg2.l10.m1.3.3.3.3.6.2">subscript</csymbol><ci id="alg2.l10.m1.3.3.3.3.6.2.2.cmml" xref="alg2.l10.m1.3.3.3.3.6.2.2">𝐸</ci><ci id="alg2.l10.m1.3.3.3.3.6.2.3.cmml" xref="alg2.l10.m1.3.3.3.3.6.2.3">𝜃</ci></apply></apply><vector id="alg2.l10.m1.3.3.3.3.3.4.cmml" xref="alg2.l10.m1.3.3.3.3.3.3"><apply id="alg2.l10.m1.1.1.1.1.1.1.1.cmml" xref="alg2.l10.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l10.m1.1.1.1.1.1.1.1.1.cmml" xref="alg2.l10.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="alg2.l10.m1.1.1.1.1.1.1.1.2.cmml" xref="alg2.l10.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg2.l10.m1.1.1.1.1.1.1.1.3.cmml" xref="alg2.l10.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg2.l10.m1.2.2.2.2.2.2.2.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l10.m1.2.2.2.2.2.2.2.1.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2">superscript</csymbol><apply id="alg2.l10.m1.2.2.2.2.2.2.2.2.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l10.m1.2.2.2.2.2.2.2.2.1.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2">subscript</csymbol><apply id="alg2.l10.m1.2.2.2.2.2.2.2.2.2.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.2"><ci id="alg2.l10.m1.2.2.2.2.2.2.2.2.2.1.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.2.1">~</ci><ci id="alg2.l10.m1.2.2.2.2.2.2.2.2.2.2.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.2.2">𝒚</ci></apply><ci id="alg2.l10.m1.2.2.2.2.2.2.2.2.3.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="alg2.l10.m1.2.2.2.2.2.2.2.3.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.3"><minus id="alg2.l10.m1.2.2.2.2.2.2.2.3.1.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.1"></minus><ci id="alg2.l10.m1.2.2.2.2.2.2.2.3.2.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.2">𝑛</ci><cn type="integer" id="alg2.l10.m1.2.2.2.2.2.2.2.3.3.cmml" xref="alg2.l10.m1.2.2.2.2.2.2.2.3.3">1</cn></apply></apply><apply id="alg2.l10.m1.3.3.3.3.3.3.3.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.3.3.3.3.3.1.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3">superscript</csymbol><apply id="alg2.l10.m1.3.3.3.3.3.3.3.2.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l10.m1.3.3.3.3.3.3.3.2.1.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3">subscript</csymbol><apply id="alg2.l10.m1.3.3.3.3.3.3.3.2.2.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.2"><ci id="alg2.l10.m1.3.3.3.3.3.3.3.2.2.1.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.2.1">~</ci><ci id="alg2.l10.m1.3.3.3.3.3.3.3.2.2.2.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.2.2">𝒛</ci></apply><ci id="alg2.l10.m1.3.3.3.3.3.3.3.2.3.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.2.3">𝑖</ci></apply><apply id="alg2.l10.m1.3.3.3.3.3.3.3.3.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.3"><minus id="alg2.l10.m1.3.3.3.3.3.3.3.3.1.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.1"></minus><ci id="alg2.l10.m1.3.3.3.3.3.3.3.3.2.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.2">𝑛</ci><cn type="integer" id="alg2.l10.m1.3.3.3.3.3.3.3.3.3.cmml" xref="alg2.l10.m1.3.3.3.3.3.3.3.3.3">1</cn></apply></apply></vector></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l10.m1.3c">\tilde{{\bm{y}}}_{i}^{n}\leftarrow\tilde{{\bm{y}}}_{i}^{n-1}-\lambda\nabla_{\bm{y}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i}^{n-1},\tilde{{\bm{z}}}_{i}^{n-1})</annotation></semantics></math><span id="alg2.l10.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l11" class="ltx_listingline">
<span id="alg2.l11.1" class="ltx_text" style="font-size:90%;">       </span><math id="alg2.l11.m1.3" class="ltx_Math" alttext="\tilde{{\bm{z}}}_{i}^{n}\leftarrow\tilde{{\bm{z}}}_{i}^{n-1}-\lambda\nabla_{\bm{z}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i}^{n-1},\tilde{{\bm{z}}}_{i}^{n-1})" display="inline"><semantics id="alg2.l11.m1.3a"><mrow id="alg2.l11.m1.3.3" xref="alg2.l11.m1.3.3.cmml"><msubsup id="alg2.l11.m1.3.3.5" xref="alg2.l11.m1.3.3.5.cmml"><mover accent="true" id="alg2.l11.m1.3.3.5.2.2" xref="alg2.l11.m1.3.3.5.2.2.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.5.2.2.2" xref="alg2.l11.m1.3.3.5.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l11.m1.3.3.5.2.2.1" xref="alg2.l11.m1.3.3.5.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l11.m1.3.3.5.2.3" xref="alg2.l11.m1.3.3.5.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l11.m1.3.3.5.3" xref="alg2.l11.m1.3.3.5.3.cmml">n</mi></msubsup><mo mathsize="90%" stretchy="false" id="alg2.l11.m1.3.3.4" xref="alg2.l11.m1.3.3.4.cmml">←</mo><mrow id="alg2.l11.m1.3.3.3" xref="alg2.l11.m1.3.3.3.cmml"><msubsup id="alg2.l11.m1.3.3.3.5" xref="alg2.l11.m1.3.3.3.5.cmml"><mover accent="true" id="alg2.l11.m1.3.3.3.5.2.2" xref="alg2.l11.m1.3.3.3.5.2.2.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.3.5.2.2.2" xref="alg2.l11.m1.3.3.3.5.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l11.m1.3.3.3.5.2.2.1" xref="alg2.l11.m1.3.3.3.5.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l11.m1.3.3.3.5.2.3" xref="alg2.l11.m1.3.3.3.5.2.3.cmml">i</mi><mrow id="alg2.l11.m1.3.3.3.5.3" xref="alg2.l11.m1.3.3.3.5.3.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.3.5.3.2" xref="alg2.l11.m1.3.3.3.5.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l11.m1.3.3.3.5.3.1" xref="alg2.l11.m1.3.3.3.5.3.1.cmml">−</mo><mn mathsize="90%" id="alg2.l11.m1.3.3.3.5.3.3" xref="alg2.l11.m1.3.3.3.5.3.3.cmml">1</mn></mrow></msubsup><mo mathsize="90%" id="alg2.l11.m1.3.3.3.4" xref="alg2.l11.m1.3.3.3.4.cmml">−</mo><mrow id="alg2.l11.m1.3.3.3.3" xref="alg2.l11.m1.3.3.3.3.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.5" xref="alg2.l11.m1.3.3.3.3.5.cmml">λ</mi><mo lspace="0.167em" rspace="0em" id="alg2.l11.m1.3.3.3.3.4" xref="alg2.l11.m1.3.3.3.3.4.cmml">​</mo><mrow id="alg2.l11.m1.3.3.3.3.6" xref="alg2.l11.m1.3.3.3.3.6.cmml"><msub id="alg2.l11.m1.3.3.3.3.6.1" xref="alg2.l11.m1.3.3.3.3.6.1.cmml"><mo mathsize="90%" rspace="0.167em" id="alg2.l11.m1.3.3.3.3.6.1.2" xref="alg2.l11.m1.3.3.3.3.6.1.2.cmml">∇</mo><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.6.1.3" xref="alg2.l11.m1.3.3.3.3.6.1.3.cmml">𝒛</mi></msub><msub id="alg2.l11.m1.3.3.3.3.6.2" xref="alg2.l11.m1.3.3.3.3.6.2.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.6.2.2" xref="alg2.l11.m1.3.3.3.3.6.2.2.cmml">E</mi><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.6.2.3" xref="alg2.l11.m1.3.3.3.3.6.2.3.cmml">θ</mi></msub></mrow><mo lspace="0em" rspace="0em" id="alg2.l11.m1.3.3.3.3.4a" xref="alg2.l11.m1.3.3.3.3.4.cmml">​</mo><mrow id="alg2.l11.m1.3.3.3.3.3.3" xref="alg2.l11.m1.3.3.3.3.3.4.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.4" xref="alg2.l11.m1.3.3.3.3.3.4.cmml">(</mo><msub id="alg2.l11.m1.1.1.1.1.1.1.1" xref="alg2.l11.m1.1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg2.l11.m1.1.1.1.1.1.1.1.2" xref="alg2.l11.m1.1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l11.m1.1.1.1.1.1.1.1.3" xref="alg2.l11.m1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.5" xref="alg2.l11.m1.3.3.3.3.3.4.cmml">,</mo><msubsup id="alg2.l11.m1.2.2.2.2.2.2.2" xref="alg2.l11.m1.2.2.2.2.2.2.2.cmml"><mover accent="true" id="alg2.l11.m1.2.2.2.2.2.2.2.2.2" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l11.m1.2.2.2.2.2.2.2.2.2.2" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l11.m1.2.2.2.2.2.2.2.2.2.1" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l11.m1.2.2.2.2.2.2.2.2.3" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.3.cmml">i</mi><mrow id="alg2.l11.m1.2.2.2.2.2.2.2.3" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.cmml"><mi mathsize="90%" id="alg2.l11.m1.2.2.2.2.2.2.2.3.2" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l11.m1.2.2.2.2.2.2.2.3.1" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.1.cmml">−</mo><mn mathsize="90%" id="alg2.l11.m1.2.2.2.2.2.2.2.3.3" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.3.cmml">1</mn></mrow></msubsup><mo mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.6" xref="alg2.l11.m1.3.3.3.3.3.4.cmml">,</mo><msubsup id="alg2.l11.m1.3.3.3.3.3.3.3" xref="alg2.l11.m1.3.3.3.3.3.3.3.cmml"><mover accent="true" id="alg2.l11.m1.3.3.3.3.3.3.3.2.2" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.3.2.2.2" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.3.2.2.1" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.3.2.3" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.3.cmml">i</mi><mrow id="alg2.l11.m1.3.3.3.3.3.3.3.3" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.cmml"><mi mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.3.3.2" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.3.3.1" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.1.cmml">−</mo><mn mathsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.3.3.3" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.3.cmml">1</mn></mrow></msubsup><mo maxsize="90%" minsize="90%" id="alg2.l11.m1.3.3.3.3.3.3.7" xref="alg2.l11.m1.3.3.3.3.3.4.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l11.m1.3b"><apply id="alg2.l11.m1.3.3.cmml" xref="alg2.l11.m1.3.3"><ci id="alg2.l11.m1.3.3.4.cmml" xref="alg2.l11.m1.3.3.4">←</ci><apply id="alg2.l11.m1.3.3.5.cmml" xref="alg2.l11.m1.3.3.5"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.5.1.cmml" xref="alg2.l11.m1.3.3.5">superscript</csymbol><apply id="alg2.l11.m1.3.3.5.2.cmml" xref="alg2.l11.m1.3.3.5"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.5.2.1.cmml" xref="alg2.l11.m1.3.3.5">subscript</csymbol><apply id="alg2.l11.m1.3.3.5.2.2.cmml" xref="alg2.l11.m1.3.3.5.2.2"><ci id="alg2.l11.m1.3.3.5.2.2.1.cmml" xref="alg2.l11.m1.3.3.5.2.2.1">~</ci><ci id="alg2.l11.m1.3.3.5.2.2.2.cmml" xref="alg2.l11.m1.3.3.5.2.2.2">𝒛</ci></apply><ci id="alg2.l11.m1.3.3.5.2.3.cmml" xref="alg2.l11.m1.3.3.5.2.3">𝑖</ci></apply><ci id="alg2.l11.m1.3.3.5.3.cmml" xref="alg2.l11.m1.3.3.5.3">𝑛</ci></apply><apply id="alg2.l11.m1.3.3.3.cmml" xref="alg2.l11.m1.3.3.3"><minus id="alg2.l11.m1.3.3.3.4.cmml" xref="alg2.l11.m1.3.3.3.4"></minus><apply id="alg2.l11.m1.3.3.3.5.cmml" xref="alg2.l11.m1.3.3.3.5"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.3.5.1.cmml" xref="alg2.l11.m1.3.3.3.5">superscript</csymbol><apply id="alg2.l11.m1.3.3.3.5.2.cmml" xref="alg2.l11.m1.3.3.3.5"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.3.5.2.1.cmml" xref="alg2.l11.m1.3.3.3.5">subscript</csymbol><apply id="alg2.l11.m1.3.3.3.5.2.2.cmml" xref="alg2.l11.m1.3.3.3.5.2.2"><ci id="alg2.l11.m1.3.3.3.5.2.2.1.cmml" xref="alg2.l11.m1.3.3.3.5.2.2.1">~</ci><ci id="alg2.l11.m1.3.3.3.5.2.2.2.cmml" xref="alg2.l11.m1.3.3.3.5.2.2.2">𝒛</ci></apply><ci id="alg2.l11.m1.3.3.3.5.2.3.cmml" xref="alg2.l11.m1.3.3.3.5.2.3">𝑖</ci></apply><apply id="alg2.l11.m1.3.3.3.5.3.cmml" xref="alg2.l11.m1.3.3.3.5.3"><minus id="alg2.l11.m1.3.3.3.5.3.1.cmml" xref="alg2.l11.m1.3.3.3.5.3.1"></minus><ci id="alg2.l11.m1.3.3.3.5.3.2.cmml" xref="alg2.l11.m1.3.3.3.5.3.2">𝑛</ci><cn type="integer" id="alg2.l11.m1.3.3.3.5.3.3.cmml" xref="alg2.l11.m1.3.3.3.5.3.3">1</cn></apply></apply><apply id="alg2.l11.m1.3.3.3.3.cmml" xref="alg2.l11.m1.3.3.3.3"><times id="alg2.l11.m1.3.3.3.3.4.cmml" xref="alg2.l11.m1.3.3.3.3.4"></times><ci id="alg2.l11.m1.3.3.3.3.5.cmml" xref="alg2.l11.m1.3.3.3.3.5">𝜆</ci><apply id="alg2.l11.m1.3.3.3.3.6.cmml" xref="alg2.l11.m1.3.3.3.3.6"><apply id="alg2.l11.m1.3.3.3.3.6.1.cmml" xref="alg2.l11.m1.3.3.3.3.6.1"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.3.3.6.1.1.cmml" xref="alg2.l11.m1.3.3.3.3.6.1">subscript</csymbol><ci id="alg2.l11.m1.3.3.3.3.6.1.2.cmml" xref="alg2.l11.m1.3.3.3.3.6.1.2">∇</ci><ci id="alg2.l11.m1.3.3.3.3.6.1.3.cmml" xref="alg2.l11.m1.3.3.3.3.6.1.3">𝒛</ci></apply><apply id="alg2.l11.m1.3.3.3.3.6.2.cmml" xref="alg2.l11.m1.3.3.3.3.6.2"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.3.3.6.2.1.cmml" xref="alg2.l11.m1.3.3.3.3.6.2">subscript</csymbol><ci id="alg2.l11.m1.3.3.3.3.6.2.2.cmml" xref="alg2.l11.m1.3.3.3.3.6.2.2">𝐸</ci><ci id="alg2.l11.m1.3.3.3.3.6.2.3.cmml" xref="alg2.l11.m1.3.3.3.3.6.2.3">𝜃</ci></apply></apply><vector id="alg2.l11.m1.3.3.3.3.3.4.cmml" xref="alg2.l11.m1.3.3.3.3.3.3"><apply id="alg2.l11.m1.1.1.1.1.1.1.1.cmml" xref="alg2.l11.m1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l11.m1.1.1.1.1.1.1.1.1.cmml" xref="alg2.l11.m1.1.1.1.1.1.1.1">subscript</csymbol><ci id="alg2.l11.m1.1.1.1.1.1.1.1.2.cmml" xref="alg2.l11.m1.1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg2.l11.m1.1.1.1.1.1.1.1.3.cmml" xref="alg2.l11.m1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg2.l11.m1.2.2.2.2.2.2.2.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l11.m1.2.2.2.2.2.2.2.1.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2">superscript</csymbol><apply id="alg2.l11.m1.2.2.2.2.2.2.2.2.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l11.m1.2.2.2.2.2.2.2.2.1.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2">subscript</csymbol><apply id="alg2.l11.m1.2.2.2.2.2.2.2.2.2.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.2"><ci id="alg2.l11.m1.2.2.2.2.2.2.2.2.2.1.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.2.1">~</ci><ci id="alg2.l11.m1.2.2.2.2.2.2.2.2.2.2.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.2.2">𝒚</ci></apply><ci id="alg2.l11.m1.2.2.2.2.2.2.2.2.3.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="alg2.l11.m1.2.2.2.2.2.2.2.3.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.3"><minus id="alg2.l11.m1.2.2.2.2.2.2.2.3.1.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.1"></minus><ci id="alg2.l11.m1.2.2.2.2.2.2.2.3.2.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.2">𝑛</ci><cn type="integer" id="alg2.l11.m1.2.2.2.2.2.2.2.3.3.cmml" xref="alg2.l11.m1.2.2.2.2.2.2.2.3.3">1</cn></apply></apply><apply id="alg2.l11.m1.3.3.3.3.3.3.3.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.3.3.3.3.3.1.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3">superscript</csymbol><apply id="alg2.l11.m1.3.3.3.3.3.3.3.2.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l11.m1.3.3.3.3.3.3.3.2.1.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3">subscript</csymbol><apply id="alg2.l11.m1.3.3.3.3.3.3.3.2.2.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.2"><ci id="alg2.l11.m1.3.3.3.3.3.3.3.2.2.1.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.2.1">~</ci><ci id="alg2.l11.m1.3.3.3.3.3.3.3.2.2.2.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.2.2">𝒛</ci></apply><ci id="alg2.l11.m1.3.3.3.3.3.3.3.2.3.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.2.3">𝑖</ci></apply><apply id="alg2.l11.m1.3.3.3.3.3.3.3.3.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.3"><minus id="alg2.l11.m1.3.3.3.3.3.3.3.3.1.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.1"></minus><ci id="alg2.l11.m1.3.3.3.3.3.3.3.3.2.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.2">𝑛</ci><cn type="integer" id="alg2.l11.m1.3.3.3.3.3.3.3.3.3.cmml" xref="alg2.l11.m1.3.3.3.3.3.3.3.3.3">1</cn></apply></apply></vector></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l11.m1.3c">\tilde{{\bm{z}}}_{i}^{n}\leftarrow\tilde{{\bm{z}}}_{i}^{n-1}-\lambda\nabla_{\bm{z}}E_{\theta}({\bm{x}}_{i},\tilde{{\bm{y}}}_{i}^{n-1},\tilde{{\bm{z}}}_{i}^{n-1})</annotation></semantics></math><span id="alg2.l11.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l12" class="ltx_listingline">
<span id="alg2.l12.1" class="ltx_text" style="font-size:90%;">    </span><span id="alg2.l12.2" class="ltx_text ltx_font_bold" style="font-size:90%;">end</span><span id="alg2.l12.3" class="ltx_text" style="font-size:90%;"> </span><span id="alg2.l12.4" class="ltx_text ltx_font_bold" style="font-size:90%;">for</span><span id="alg2.l12.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l13" class="ltx_listingline">
<span id="alg2.l13.4" class="ltx_text" style="font-size:90%;">    </span><em id="alg2.l13.3" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg2.l13.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg2.l13.1.m1.1a"><mo id="alg2.l13.1.m1.1.1" xref="alg2.l13.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg2.l13.1.m1.1b"><ci id="alg2.l13.1.m1.1.1.cmml" xref="alg2.l13.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l13.1.m1.1c">\triangleright</annotation></semantics></math> Optimize objective <math id="alg2.l13.2.m2.1" class="ltx_Math" alttext="\mathcal{L}_{\text{MSE}}" display="inline"><semantics id="alg2.l13.2.m2.1a"><msub id="alg2.l13.2.m2.1.1" xref="alg2.l13.2.m2.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg2.l13.2.m2.1.1.2" xref="alg2.l13.2.m2.1.1.2.cmml">ℒ</mi><mtext class="ltx_mathvariant_italic" id="alg2.l13.2.m2.1.1.3" xref="alg2.l13.2.m2.1.1.3b.cmml"><em id="alg2.l13.2.m2.1.1.3.1nest" class="ltx_emph" style="font-size:70%;">MSE</em></mtext></msub><annotation-xml encoding="MathML-Content" id="alg2.l13.2.m2.1b"><apply id="alg2.l13.2.m2.1.1.cmml" xref="alg2.l13.2.m2.1.1"><csymbol cd="ambiguous" id="alg2.l13.2.m2.1.1.1.cmml" xref="alg2.l13.2.m2.1.1">subscript</csymbol><ci id="alg2.l13.2.m2.1.1.2.cmml" xref="alg2.l13.2.m2.1.1.2">ℒ</ci><ci id="alg2.l13.2.m2.1.1.3b.cmml" xref="alg2.l13.2.m2.1.1.3"><mtext class="ltx_mathvariant_italic" mathsize="70%" id="alg2.l13.2.m2.1.1.3.cmml" xref="alg2.l13.2.m2.1.1.3"><em id="alg2.l13.2.m2.1.1.3.1anest" class="ltx_emph" style="font-size:70%;">MSE</em></mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l13.2.m2.1c">\mathcal{L}_{\text{MSE}}</annotation></semantics></math> wrt <math id="alg2.l13.3.m3.1" class="ltx_Math" alttext="\theta" display="inline"><semantics id="alg2.l13.3.m3.1a"><mi id="alg2.l13.3.m3.1.1" xref="alg2.l13.3.m3.1.1.cmml">θ</mi><annotation-xml encoding="MathML-Content" id="alg2.l13.3.m3.1b"><ci id="alg2.l13.3.m3.1.1.cmml" xref="alg2.l13.3.m3.1.1">𝜃</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l13.3.m3.1c">\theta</annotation></semantics></math>:</em><span id="alg2.l13.5" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l14" class="ltx_listingline">
<span id="alg2.l14.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg2.l14.m1.1" class="ltx_Math" alttext="\Delta\theta\leftarrow\nabla_{\theta}\sum_{n=1}^{N}\|\tilde{{\bm{y}}}^{N}_{i}-{\bm{y}}_{i}\|^{2}" display="inline"><semantics id="alg2.l14.m1.1a"><mrow id="alg2.l14.m1.1.1" xref="alg2.l14.m1.1.1.cmml"><mrow id="alg2.l14.m1.1.1.3" xref="alg2.l14.m1.1.1.3.cmml"><mi mathsize="90%" mathvariant="normal" id="alg2.l14.m1.1.1.3.2" xref="alg2.l14.m1.1.1.3.2.cmml">Δ</mi><mo lspace="0em" rspace="0em" id="alg2.l14.m1.1.1.3.1" xref="alg2.l14.m1.1.1.3.1.cmml">​</mo><mi mathsize="90%" id="alg2.l14.m1.1.1.3.3" xref="alg2.l14.m1.1.1.3.3.cmml">θ</mi></mrow><mo mathsize="90%" stretchy="false" id="alg2.l14.m1.1.1.2" xref="alg2.l14.m1.1.1.2.cmml">←</mo><mrow id="alg2.l14.m1.1.1.1" xref="alg2.l14.m1.1.1.1.cmml"><msub id="alg2.l14.m1.1.1.1.3" xref="alg2.l14.m1.1.1.1.3.cmml"><mo mathsize="90%" id="alg2.l14.m1.1.1.1.3.2" xref="alg2.l14.m1.1.1.1.3.2.cmml">∇</mo><mi mathsize="90%" id="alg2.l14.m1.1.1.1.3.3" xref="alg2.l14.m1.1.1.1.3.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="alg2.l14.m1.1.1.1.2" xref="alg2.l14.m1.1.1.1.2.cmml">​</mo><mrow id="alg2.l14.m1.1.1.1.1" xref="alg2.l14.m1.1.1.1.1.cmml"><msubsup id="alg2.l14.m1.1.1.1.1.2" xref="alg2.l14.m1.1.1.1.1.2.cmml"><mo maxsize="90%" minsize="90%" rspace="0em" stretchy="true" id="alg2.l14.m1.1.1.1.1.2.2.2" xref="alg2.l14.m1.1.1.1.1.2.2.2.cmml">∑</mo><mrow id="alg2.l14.m1.1.1.1.1.2.2.3" xref="alg2.l14.m1.1.1.1.1.2.2.3.cmml"><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.2.2.3.2" xref="alg2.l14.m1.1.1.1.1.2.2.3.2.cmml">n</mi><mo mathsize="90%" id="alg2.l14.m1.1.1.1.1.2.2.3.1" xref="alg2.l14.m1.1.1.1.1.2.2.3.1.cmml">=</mo><mn mathsize="90%" id="alg2.l14.m1.1.1.1.1.2.2.3.3" xref="alg2.l14.m1.1.1.1.1.2.2.3.3.cmml">1</mn></mrow><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.2.3" xref="alg2.l14.m1.1.1.1.1.2.3.cmml">N</mi></msubsup><msup id="alg2.l14.m1.1.1.1.1.1" xref="alg2.l14.m1.1.1.1.1.1.cmml"><mrow id="alg2.l14.m1.1.1.1.1.1.1.1" xref="alg2.l14.m1.1.1.1.1.1.1.2.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.2" xref="alg2.l14.m1.1.1.1.1.1.1.2.1.cmml">‖</mo><mrow id="alg2.l14.m1.1.1.1.1.1.1.1.1" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.cmml"><msubsup id="alg2.l14.m1.1.1.1.1.1.1.1.1.2" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.cmml"><mover accent="true" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.cmml"><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.2" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.1" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.3" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.3" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.3.cmml">N</mi></msubsup><mo mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.1" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.1.cmml">−</mo><msub id="alg2.l14.m1.1.1.1.1.1.1.1.1.3" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3.cmml"><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.3.2" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.1.3.3" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo maxsize="90%" minsize="90%" id="alg2.l14.m1.1.1.1.1.1.1.1.3" xref="alg2.l14.m1.1.1.1.1.1.1.2.1.cmml">‖</mo></mrow><mn mathsize="90%" id="alg2.l14.m1.1.1.1.1.1.3" xref="alg2.l14.m1.1.1.1.1.1.3.cmml">2</mn></msup></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l14.m1.1b"><apply id="alg2.l14.m1.1.1.cmml" xref="alg2.l14.m1.1.1"><ci id="alg2.l14.m1.1.1.2.cmml" xref="alg2.l14.m1.1.1.2">←</ci><apply id="alg2.l14.m1.1.1.3.cmml" xref="alg2.l14.m1.1.1.3"><times id="alg2.l14.m1.1.1.3.1.cmml" xref="alg2.l14.m1.1.1.3.1"></times><ci id="alg2.l14.m1.1.1.3.2.cmml" xref="alg2.l14.m1.1.1.3.2">Δ</ci><ci id="alg2.l14.m1.1.1.3.3.cmml" xref="alg2.l14.m1.1.1.3.3">𝜃</ci></apply><apply id="alg2.l14.m1.1.1.1.cmml" xref="alg2.l14.m1.1.1.1"><times id="alg2.l14.m1.1.1.1.2.cmml" xref="alg2.l14.m1.1.1.1.2"></times><apply id="alg2.l14.m1.1.1.1.3.cmml" xref="alg2.l14.m1.1.1.1.3"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.3.1.cmml" xref="alg2.l14.m1.1.1.1.3">subscript</csymbol><ci id="alg2.l14.m1.1.1.1.3.2.cmml" xref="alg2.l14.m1.1.1.1.3.2">∇</ci><ci id="alg2.l14.m1.1.1.1.3.3.cmml" xref="alg2.l14.m1.1.1.1.3.3">𝜃</ci></apply><apply id="alg2.l14.m1.1.1.1.1.cmml" xref="alg2.l14.m1.1.1.1.1"><apply id="alg2.l14.m1.1.1.1.1.2.cmml" xref="alg2.l14.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.1.2.1.cmml" xref="alg2.l14.m1.1.1.1.1.2">superscript</csymbol><apply id="alg2.l14.m1.1.1.1.1.2.2.cmml" xref="alg2.l14.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.1.2.2.1.cmml" xref="alg2.l14.m1.1.1.1.1.2">subscript</csymbol><sum id="alg2.l14.m1.1.1.1.1.2.2.2.cmml" xref="alg2.l14.m1.1.1.1.1.2.2.2"></sum><apply id="alg2.l14.m1.1.1.1.1.2.2.3.cmml" xref="alg2.l14.m1.1.1.1.1.2.2.3"><eq id="alg2.l14.m1.1.1.1.1.2.2.3.1.cmml" xref="alg2.l14.m1.1.1.1.1.2.2.3.1"></eq><ci id="alg2.l14.m1.1.1.1.1.2.2.3.2.cmml" xref="alg2.l14.m1.1.1.1.1.2.2.3.2">𝑛</ci><cn type="integer" id="alg2.l14.m1.1.1.1.1.2.2.3.3.cmml" xref="alg2.l14.m1.1.1.1.1.2.2.3.3">1</cn></apply></apply><ci id="alg2.l14.m1.1.1.1.1.2.3.cmml" xref="alg2.l14.m1.1.1.1.1.2.3">𝑁</ci></apply><apply id="alg2.l14.m1.1.1.1.1.1.cmml" xref="alg2.l14.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.1.1.2.cmml" xref="alg2.l14.m1.1.1.1.1.1">superscript</csymbol><apply id="alg2.l14.m1.1.1.1.1.1.1.2.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1"><csymbol cd="latexml" id="alg2.l14.m1.1.1.1.1.1.1.2.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.2">norm</csymbol><apply id="alg2.l14.m1.1.1.1.1.1.1.1.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1"><minus id="alg2.l14.m1.1.1.1.1.1.1.1.1.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.1"></minus><apply id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2">subscript</csymbol><apply id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2">superscript</csymbol><apply id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2"><ci id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.1">~</ci><ci id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.2.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.2.2">𝒚</ci></apply><ci id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.3.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.2.3">𝑁</ci></apply><ci id="alg2.l14.m1.1.1.1.1.1.1.1.1.2.3.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="alg2.l14.m1.1.1.1.1.1.1.1.1.3.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="alg2.l14.m1.1.1.1.1.1.1.1.1.3.1.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3">subscript</csymbol><ci id="alg2.l14.m1.1.1.1.1.1.1.1.1.3.2.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3.2">𝒚</ci><ci id="alg2.l14.m1.1.1.1.1.1.1.1.1.3.3.cmml" xref="alg2.l14.m1.1.1.1.1.1.1.1.1.3.3">𝑖</ci></apply></apply></apply><cn type="integer" id="alg2.l14.m1.1.1.1.1.1.3.cmml" xref="alg2.l14.m1.1.1.1.1.1.3">2</cn></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l14.m1.1c">\Delta\theta\leftarrow\nabla_{\theta}\sum_{n=1}^{N}\|\tilde{{\bm{y}}}^{N}_{i}-{\bm{y}}_{i}\|^{2}</annotation></semantics></math><span id="alg2.l14.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l15" class="ltx_listingline">
<span id="alg2.l15.1" class="ltx_text" style="font-size:90%;">    Update </span><math id="alg2.l15.m1.1" class="ltx_Math" alttext="\theta" display="inline"><semantics id="alg2.l15.m1.1a"><mi mathsize="90%" id="alg2.l15.m1.1.1" xref="alg2.l15.m1.1.1.cmml">θ</mi><annotation-xml encoding="MathML-Content" id="alg2.l15.m1.1b"><ci id="alg2.l15.m1.1.1.cmml" xref="alg2.l15.m1.1.1">𝜃</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l15.m1.1c">\theta</annotation></semantics></math><span id="alg2.l15.2" class="ltx_text" style="font-size:90%;"> based on </span><math id="alg2.l15.m2.1" class="ltx_Math" alttext="\Delta\theta" display="inline"><semantics id="alg2.l15.m2.1a"><mrow id="alg2.l15.m2.1.1" xref="alg2.l15.m2.1.1.cmml"><mi mathsize="90%" mathvariant="normal" id="alg2.l15.m2.1.1.2" xref="alg2.l15.m2.1.1.2.cmml">Δ</mi><mo lspace="0em" rspace="0em" id="alg2.l15.m2.1.1.1" xref="alg2.l15.m2.1.1.1.cmml">​</mo><mi mathsize="90%" id="alg2.l15.m2.1.1.3" xref="alg2.l15.m2.1.1.3.cmml">θ</mi></mrow><annotation-xml encoding="MathML-Content" id="alg2.l15.m2.1b"><apply id="alg2.l15.m2.1.1.cmml" xref="alg2.l15.m2.1.1"><times id="alg2.l15.m2.1.1.1.cmml" xref="alg2.l15.m2.1.1.1"></times><ci id="alg2.l15.m2.1.1.2.cmml" xref="alg2.l15.m2.1.1.2">Δ</ci><ci id="alg2.l15.m2.1.1.3.cmml" xref="alg2.l15.m2.1.1.3">𝜃</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l15.m2.1c">\Delta\theta</annotation></semantics></math><span id="alg2.l15.3" class="ltx_text" style="font-size:90%;"> using Adam optimizer

</span>
</div>
<div id="alg2.l16" class="ltx_listingline">
<span id="alg2.l16.3" class="ltx_text" style="font-size:90%;">    </span><em id="alg2.l16.2" class="ltx_emph ltx_font_italic" style="font-size:90%;"><math id="alg2.l16.1.m1.1" class="ltx_Math" alttext="\triangleright" display="inline"><semantics id="alg2.l16.1.m1.1a"><mo id="alg2.l16.1.m1.1.1" xref="alg2.l16.1.m1.1.1.cmml">▷</mo><annotation-xml encoding="MathML-Content" id="alg2.l16.1.m1.1b"><ci id="alg2.l16.1.m1.1.1.cmml" xref="alg2.l16.1.m1.1.1">▷</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l16.1.m1.1c">\triangleright</annotation></semantics></math> Update replay buffer <math id="alg2.l16.2.m2.1" class="ltx_Math" alttext="\mathcal{B}" display="inline"><semantics id="alg2.l16.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="alg2.l16.2.m2.1.1" xref="alg2.l16.2.m2.1.1.cmml">ℬ</mi><annotation-xml encoding="MathML-Content" id="alg2.l16.2.m2.1b"><ci id="alg2.l16.2.m2.1.1.cmml" xref="alg2.l16.2.m2.1.1">ℬ</ci></annotation-xml><annotation encoding="application/x-tex" id="alg2.l16.2.m2.1c">\mathcal{B}</annotation></semantics></math></em><span id="alg2.l16.4" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l17" class="ltx_listingline">
<span id="alg2.l17.1" class="ltx_text" style="font-size:90%;">    </span><math id="alg2.l17.m1.4" class="ltx_Math" alttext="\mathcal{B}\leftarrow\mathcal{B}\cup({\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{N},\tilde{{\bm{z}}}_{i}^{N})" display="inline"><semantics id="alg2.l17.m1.4a"><mrow id="alg2.l17.m1.4.4" xref="alg2.l17.m1.4.4.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg2.l17.m1.4.4.6" xref="alg2.l17.m1.4.4.6.cmml">ℬ</mi><mo mathsize="90%" stretchy="false" id="alg2.l17.m1.4.4.5" xref="alg2.l17.m1.4.4.5.cmml">←</mo><mrow id="alg2.l17.m1.4.4.4" xref="alg2.l17.m1.4.4.4.cmml"><mi class="ltx_font_mathcaligraphic" mathsize="90%" id="alg2.l17.m1.4.4.4.6" xref="alg2.l17.m1.4.4.4.6.cmml">ℬ</mi><mo mathsize="90%" id="alg2.l17.m1.4.4.4.5" xref="alg2.l17.m1.4.4.4.5.cmml">∪</mo><mrow id="alg2.l17.m1.4.4.4.4.4" xref="alg2.l17.m1.4.4.4.4.5.cmml"><mo maxsize="90%" minsize="90%" id="alg2.l17.m1.4.4.4.4.4.5" xref="alg2.l17.m1.4.4.4.4.5.cmml">(</mo><msub id="alg2.l17.m1.1.1.1.1.1.1" xref="alg2.l17.m1.1.1.1.1.1.1.cmml"><mi mathsize="90%" id="alg2.l17.m1.1.1.1.1.1.1.2" xref="alg2.l17.m1.1.1.1.1.1.1.2.cmml">𝒙</mi><mi mathsize="90%" id="alg2.l17.m1.1.1.1.1.1.1.3" xref="alg2.l17.m1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.6" xref="alg2.l17.m1.4.4.4.4.5.cmml">,</mo><msub id="alg2.l17.m1.2.2.2.2.2.2" xref="alg2.l17.m1.2.2.2.2.2.2.cmml"><mi mathsize="90%" id="alg2.l17.m1.2.2.2.2.2.2.2" xref="alg2.l17.m1.2.2.2.2.2.2.2.cmml">𝒚</mi><mi mathsize="90%" id="alg2.l17.m1.2.2.2.2.2.2.3" xref="alg2.l17.m1.2.2.2.2.2.2.3.cmml">i</mi></msub><mo mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.7" xref="alg2.l17.m1.4.4.4.4.5.cmml">,</mo><msubsup id="alg2.l17.m1.3.3.3.3.3.3" xref="alg2.l17.m1.3.3.3.3.3.3.cmml"><mover accent="true" id="alg2.l17.m1.3.3.3.3.3.3.2.2" xref="alg2.l17.m1.3.3.3.3.3.3.2.2.cmml"><mi mathsize="90%" id="alg2.l17.m1.3.3.3.3.3.3.2.2.2" xref="alg2.l17.m1.3.3.3.3.3.3.2.2.2.cmml">𝒚</mi><mo mathsize="90%" id="alg2.l17.m1.3.3.3.3.3.3.2.2.1" xref="alg2.l17.m1.3.3.3.3.3.3.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l17.m1.3.3.3.3.3.3.2.3" xref="alg2.l17.m1.3.3.3.3.3.3.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l17.m1.3.3.3.3.3.3.3" xref="alg2.l17.m1.3.3.3.3.3.3.3.cmml">N</mi></msubsup><mo mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.8" xref="alg2.l17.m1.4.4.4.4.5.cmml">,</mo><msubsup id="alg2.l17.m1.4.4.4.4.4.4" xref="alg2.l17.m1.4.4.4.4.4.4.cmml"><mover accent="true" id="alg2.l17.m1.4.4.4.4.4.4.2.2" xref="alg2.l17.m1.4.4.4.4.4.4.2.2.cmml"><mi mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.4.2.2.2" xref="alg2.l17.m1.4.4.4.4.4.4.2.2.2.cmml">𝒛</mi><mo mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.4.2.2.1" xref="alg2.l17.m1.4.4.4.4.4.4.2.2.1.cmml">~</mo></mover><mi mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.4.2.3" xref="alg2.l17.m1.4.4.4.4.4.4.2.3.cmml">i</mi><mi mathsize="90%" id="alg2.l17.m1.4.4.4.4.4.4.3" xref="alg2.l17.m1.4.4.4.4.4.4.3.cmml">N</mi></msubsup><mo maxsize="90%" minsize="90%" id="alg2.l17.m1.4.4.4.4.4.9" xref="alg2.l17.m1.4.4.4.4.5.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg2.l17.m1.4b"><apply id="alg2.l17.m1.4.4.cmml" xref="alg2.l17.m1.4.4"><ci id="alg2.l17.m1.4.4.5.cmml" xref="alg2.l17.m1.4.4.5">←</ci><ci id="alg2.l17.m1.4.4.6.cmml" xref="alg2.l17.m1.4.4.6">ℬ</ci><apply id="alg2.l17.m1.4.4.4.cmml" xref="alg2.l17.m1.4.4.4"><union id="alg2.l17.m1.4.4.4.5.cmml" xref="alg2.l17.m1.4.4.4.5"></union><ci id="alg2.l17.m1.4.4.4.6.cmml" xref="alg2.l17.m1.4.4.4.6">ℬ</ci><vector id="alg2.l17.m1.4.4.4.4.5.cmml" xref="alg2.l17.m1.4.4.4.4.4"><apply id="alg2.l17.m1.1.1.1.1.1.1.cmml" xref="alg2.l17.m1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg2.l17.m1.1.1.1.1.1.1.1.cmml" xref="alg2.l17.m1.1.1.1.1.1.1">subscript</csymbol><ci id="alg2.l17.m1.1.1.1.1.1.1.2.cmml" xref="alg2.l17.m1.1.1.1.1.1.1.2">𝒙</ci><ci id="alg2.l17.m1.1.1.1.1.1.1.3.cmml" xref="alg2.l17.m1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="alg2.l17.m1.2.2.2.2.2.2.cmml" xref="alg2.l17.m1.2.2.2.2.2.2"><csymbol cd="ambiguous" id="alg2.l17.m1.2.2.2.2.2.2.1.cmml" xref="alg2.l17.m1.2.2.2.2.2.2">subscript</csymbol><ci id="alg2.l17.m1.2.2.2.2.2.2.2.cmml" xref="alg2.l17.m1.2.2.2.2.2.2.2">𝒚</ci><ci id="alg2.l17.m1.2.2.2.2.2.2.3.cmml" xref="alg2.l17.m1.2.2.2.2.2.2.3">𝑖</ci></apply><apply id="alg2.l17.m1.3.3.3.3.3.3.cmml" xref="alg2.l17.m1.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l17.m1.3.3.3.3.3.3.1.cmml" xref="alg2.l17.m1.3.3.3.3.3.3">superscript</csymbol><apply id="alg2.l17.m1.3.3.3.3.3.3.2.cmml" xref="alg2.l17.m1.3.3.3.3.3.3"><csymbol cd="ambiguous" id="alg2.l17.m1.3.3.3.3.3.3.2.1.cmml" xref="alg2.l17.m1.3.3.3.3.3.3">subscript</csymbol><apply id="alg2.l17.m1.3.3.3.3.3.3.2.2.cmml" xref="alg2.l17.m1.3.3.3.3.3.3.2.2"><ci id="alg2.l17.m1.3.3.3.3.3.3.2.2.1.cmml" xref="alg2.l17.m1.3.3.3.3.3.3.2.2.1">~</ci><ci id="alg2.l17.m1.3.3.3.3.3.3.2.2.2.cmml" xref="alg2.l17.m1.3.3.3.3.3.3.2.2.2">𝒚</ci></apply><ci id="alg2.l17.m1.3.3.3.3.3.3.2.3.cmml" xref="alg2.l17.m1.3.3.3.3.3.3.2.3">𝑖</ci></apply><ci id="alg2.l17.m1.3.3.3.3.3.3.3.cmml" xref="alg2.l17.m1.3.3.3.3.3.3.3">𝑁</ci></apply><apply id="alg2.l17.m1.4.4.4.4.4.4.cmml" xref="alg2.l17.m1.4.4.4.4.4.4"><csymbol cd="ambiguous" id="alg2.l17.m1.4.4.4.4.4.4.1.cmml" xref="alg2.l17.m1.4.4.4.4.4.4">superscript</csymbol><apply id="alg2.l17.m1.4.4.4.4.4.4.2.cmml" xref="alg2.l17.m1.4.4.4.4.4.4"><csymbol cd="ambiguous" id="alg2.l17.m1.4.4.4.4.4.4.2.1.cmml" xref="alg2.l17.m1.4.4.4.4.4.4">subscript</csymbol><apply id="alg2.l17.m1.4.4.4.4.4.4.2.2.cmml" xref="alg2.l17.m1.4.4.4.4.4.4.2.2"><ci id="alg2.l17.m1.4.4.4.4.4.4.2.2.1.cmml" xref="alg2.l17.m1.4.4.4.4.4.4.2.2.1">~</ci><ci id="alg2.l17.m1.4.4.4.4.4.4.2.2.2.cmml" xref="alg2.l17.m1.4.4.4.4.4.4.2.2.2">𝒛</ci></apply><ci id="alg2.l17.m1.4.4.4.4.4.4.2.3.cmml" xref="alg2.l17.m1.4.4.4.4.4.4.2.3">𝑖</ci></apply><ci id="alg2.l17.m1.4.4.4.4.4.4.3.cmml" xref="alg2.l17.m1.4.4.4.4.4.4.3">𝑁</ci></apply></vector></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg2.l17.m1.4c">\mathcal{B}\leftarrow\mathcal{B}\cup({\bm{x}}_{i},{\bm{y}}_{i},\tilde{{\bm{y}}}_{i}^{N},\tilde{{\bm{z}}}_{i}^{N})</annotation></semantics></math><span id="alg2.l17.2" class="ltx_text" style="font-size:90%;">
</span>
</div>
<div id="alg2.l18" class="ltx_listingline">
<span id="alg2.l18.1" class="ltx_text" style="font-size:90%;">  </span><span id="alg2.l18.2" class="ltx_text ltx_font_bold" style="font-size:90%;">end</span><span id="alg2.l18.3" class="ltx_text" style="font-size:90%;"> </span><span id="alg2.l18.4" class="ltx_text ltx_font_bold" style="font-size:90%;">while</span>
</div>
</div>
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_float"><span id="alg3.5.1.1" class="ltx_text ltx_font_bold">Algorithm 3</span> </span> IREM Training with External Memory</figcaption>
</figure>
</section>
<section id="A3" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix C </span>Experimental Details</h2>

<div id="A3.p1" class="ltx_para">
<p id="A3.p1.1" class="ltx_p"><span id="A3.p1.1.1" class="ltx_text ltx_font_bold">Graphical Algorithmic Computation</span>  Models were trained in approximately 2 hours on a single Nvidia Titan X GPU using a training batch size of 64 and the Adam optimizer with learning rate 1e-4. Each model was trained for 10,000 iterations and evaluated on 1000 test problems. Each model was trained with five steps of iterative computation, with PonderNet trained with a halting geometric distribution of 0.8. Below, we provide additional numerical details about each of the evaluated algorithmic tasks.</p>
</div>
<div id="A3.p2" class="ltx_para">
<ol id="A3.I1" class="ltx_enumerate">
<li id="A3.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="A3.I1.i1.p1" class="ltx_para">
<p id="A3.I1.i1.p1.1" class="ltx_p"><span id="A3.I1.i1.p1.1.1" class="ltx_text ltx_font_italic">Edge Copy:</span> We randomly sample a value for each edge in a fully connected graph with a uniform value between -1 and 1. Models are then tasked with replicating the value of each individual edge in the graph in the final output prediction.</p>
</div>
</li>
<li id="A3.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="A3.I1.i2.p1" class="ltx_para">
<p id="A3.I1.i2.p1.1" class="ltx_p"><span id="A3.I1.i2.p1.1.1" class="ltx_text ltx_font_italic">Connected Components:</span> We randomly zero-out 95% of the edges in a fully connected graph. Models are then tasked with predicting the pairwise connectively of all possible pairs of nodes in the graph.</p>
</div>
</li>
<li id="A3.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="A3.I1.i3.p1" class="ltx_para">
<p id="A3.I1.i3.p1.1" class="ltx_p"><span id="A3.I1.i3.p1.1.1" class="ltx_text ltx_font_italic">Shortest Path:</span> We randomly sample an edge distance between 0 and 1 for each edge in a fully connected graph. Models are then tasked with predicting the pairwise shortest distance between all possible pairs of nodes in the graph.</p>
</div>
</li>
</ol>
</div>
<div id="A3.p3" class="ltx_para">
<p id="A3.p3.1" class="ltx_p"><span id="A3.p3.1.1" class="ltx_text ltx_font_bold">Continuous Algorithmic Computation</span>  Models were trained in approximately 2 hours on a single Nvidia Titan X GPU using a training batch size of 128 and the Adam optimizer with learning rate 1e-4. Each model was trained for 10,000 iterations and evaluated on 1000 test problems. Each model was trained with five steps of iterative computation, with PonderNet trained with a halting geometric distribution of 0.8. We further provide individual dataset details below.</p>
</div>
<div id="A3.p4" class="ltx_para">
<ol id="A3.I2" class="ltx_enumerate">
<li id="A3.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="A3.I2.i1.p1" class="ltx_para">
<p id="A3.I2.i1.p1.1" class="ltx_p"><span id="A3.I2.i1.p1.1.1" class="ltx_text ltx_font_italic">Addition: </span> We randomly construct two separate vectors, each with 400 elements, with each element in the vector randomly sampled between -1 and 1. Models are then tasked with summing up the elements in each vector element-wise. When constructing more difficult addition problems at test time, each element in the vector is randomly sampled between -2.5 and 2.5.</p>
</div>
</li>
<li id="A3.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="A3.I2.i2.p1" class="ltx_para">
<p id="A3.I2.i2.p1.13" class="ltx_p"><span id="A3.I2.i2.p1.13.1" class="ltx_text ltx_font_italic">Matrix Completion:</span> We randomly construct a low-rank matrix <math id="A3.I2.i2.p1.1.m1.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A3.I2.i2.p1.1.m1.1a"><mi id="A3.I2.i2.p1.1.m1.1.1" xref="A3.I2.i2.p1.1.m1.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.1.m1.1b"><ci id="A3.I2.i2.p1.1.m1.1.1.cmml" xref="A3.I2.i2.p1.1.m1.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.1.m1.1c">M</annotation></semantics></math> represented as the <math id="A3.I2.i2.p1.2.m2.2" class="ltx_Math" alttext="M=U^{T}V+0.1\mathcal{N}(0,1)" display="inline"><semantics id="A3.I2.i2.p1.2.m2.2a"><mrow id="A3.I2.i2.p1.2.m2.2.3" xref="A3.I2.i2.p1.2.m2.2.3.cmml"><mi id="A3.I2.i2.p1.2.m2.2.3.2" xref="A3.I2.i2.p1.2.m2.2.3.2.cmml">M</mi><mo id="A3.I2.i2.p1.2.m2.2.3.1" xref="A3.I2.i2.p1.2.m2.2.3.1.cmml">=</mo><mrow id="A3.I2.i2.p1.2.m2.2.3.3" xref="A3.I2.i2.p1.2.m2.2.3.3.cmml"><mrow id="A3.I2.i2.p1.2.m2.2.3.3.2" xref="A3.I2.i2.p1.2.m2.2.3.3.2.cmml"><msup id="A3.I2.i2.p1.2.m2.2.3.3.2.2" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2.cmml"><mi id="A3.I2.i2.p1.2.m2.2.3.3.2.2.2" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2.2.cmml">U</mi><mi id="A3.I2.i2.p1.2.m2.2.3.3.2.2.3" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2.3.cmml">T</mi></msup><mo lspace="0em" rspace="0em" id="A3.I2.i2.p1.2.m2.2.3.3.2.1" xref="A3.I2.i2.p1.2.m2.2.3.3.2.1.cmml">​</mo><mi id="A3.I2.i2.p1.2.m2.2.3.3.2.3" xref="A3.I2.i2.p1.2.m2.2.3.3.2.3.cmml">V</mi></mrow><mo id="A3.I2.i2.p1.2.m2.2.3.3.1" xref="A3.I2.i2.p1.2.m2.2.3.3.1.cmml">+</mo><mrow id="A3.I2.i2.p1.2.m2.2.3.3.3" xref="A3.I2.i2.p1.2.m2.2.3.3.3.cmml"><mn id="A3.I2.i2.p1.2.m2.2.3.3.3.2" xref="A3.I2.i2.p1.2.m2.2.3.3.3.2.cmml">0.1</mn><mo lspace="0em" rspace="0em" id="A3.I2.i2.p1.2.m2.2.3.3.3.1" xref="A3.I2.i2.p1.2.m2.2.3.3.3.1.cmml">​</mo><mi class="ltx_font_mathcaligraphic" id="A3.I2.i2.p1.2.m2.2.3.3.3.3" xref="A3.I2.i2.p1.2.m2.2.3.3.3.3.cmml">𝒩</mi><mo lspace="0em" rspace="0em" id="A3.I2.i2.p1.2.m2.2.3.3.3.1a" xref="A3.I2.i2.p1.2.m2.2.3.3.3.1.cmml">​</mo><mrow id="A3.I2.i2.p1.2.m2.2.3.3.3.4.2" xref="A3.I2.i2.p1.2.m2.2.3.3.3.4.1.cmml"><mo stretchy="false" id="A3.I2.i2.p1.2.m2.2.3.3.3.4.2.1" xref="A3.I2.i2.p1.2.m2.2.3.3.3.4.1.cmml">(</mo><mn id="A3.I2.i2.p1.2.m2.1.1" xref="A3.I2.i2.p1.2.m2.1.1.cmml">0</mn><mo id="A3.I2.i2.p1.2.m2.2.3.3.3.4.2.2" xref="A3.I2.i2.p1.2.m2.2.3.3.3.4.1.cmml">,</mo><mn id="A3.I2.i2.p1.2.m2.2.2" xref="A3.I2.i2.p1.2.m2.2.2.cmml">1</mn><mo stretchy="false" id="A3.I2.i2.p1.2.m2.2.3.3.3.4.2.3" xref="A3.I2.i2.p1.2.m2.2.3.3.3.4.1.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.2.m2.2b"><apply id="A3.I2.i2.p1.2.m2.2.3.cmml" xref="A3.I2.i2.p1.2.m2.2.3"><eq id="A3.I2.i2.p1.2.m2.2.3.1.cmml" xref="A3.I2.i2.p1.2.m2.2.3.1"></eq><ci id="A3.I2.i2.p1.2.m2.2.3.2.cmml" xref="A3.I2.i2.p1.2.m2.2.3.2">𝑀</ci><apply id="A3.I2.i2.p1.2.m2.2.3.3.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3"><plus id="A3.I2.i2.p1.2.m2.2.3.3.1.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.1"></plus><apply id="A3.I2.i2.p1.2.m2.2.3.3.2.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2"><times id="A3.I2.i2.p1.2.m2.2.3.3.2.1.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2.1"></times><apply id="A3.I2.i2.p1.2.m2.2.3.3.2.2.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2"><csymbol cd="ambiguous" id="A3.I2.i2.p1.2.m2.2.3.3.2.2.1.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2">superscript</csymbol><ci id="A3.I2.i2.p1.2.m2.2.3.3.2.2.2.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2.2">𝑈</ci><ci id="A3.I2.i2.p1.2.m2.2.3.3.2.2.3.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2.2.3">𝑇</ci></apply><ci id="A3.I2.i2.p1.2.m2.2.3.3.2.3.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.2.3">𝑉</ci></apply><apply id="A3.I2.i2.p1.2.m2.2.3.3.3.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.3"><times id="A3.I2.i2.p1.2.m2.2.3.3.3.1.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.3.1"></times><cn type="float" id="A3.I2.i2.p1.2.m2.2.3.3.3.2.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.3.2">0.1</cn><ci id="A3.I2.i2.p1.2.m2.2.3.3.3.3.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.3.3">𝒩</ci><interval closure="open" id="A3.I2.i2.p1.2.m2.2.3.3.3.4.1.cmml" xref="A3.I2.i2.p1.2.m2.2.3.3.3.4.2"><cn type="integer" id="A3.I2.i2.p1.2.m2.1.1.cmml" xref="A3.I2.i2.p1.2.m2.1.1">0</cn><cn type="integer" id="A3.I2.i2.p1.2.m2.2.2.cmml" xref="A3.I2.i2.p1.2.m2.2.2">1</cn></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.2.m2.2c">M=U^{T}V+0.1\mathcal{N}(0,1)</annotation></semantics></math>, where <math id="A3.I2.i2.p1.3.m3.1" class="ltx_Math" alttext="U" display="inline"><semantics id="A3.I2.i2.p1.3.m3.1a"><mi id="A3.I2.i2.p1.3.m3.1.1" xref="A3.I2.i2.p1.3.m3.1.1.cmml">U</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.3.m3.1b"><ci id="A3.I2.i2.p1.3.m3.1.1.cmml" xref="A3.I2.i2.p1.3.m3.1.1">𝑈</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.3.m3.1c">U</annotation></semantics></math> and <math id="A3.I2.i2.p1.4.m4.1" class="ltx_Math" alttext="V" display="inline"><semantics id="A3.I2.i2.p1.4.m4.1a"><mi id="A3.I2.i2.p1.4.m4.1.1" xref="A3.I2.i2.p1.4.m4.1.1.cmml">V</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.4.m4.1b"><ci id="A3.I2.i2.p1.4.m4.1.1.cmml" xref="A3.I2.i2.p1.4.m4.1.1">𝑉</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.4.m4.1c">V</annotation></semantics></math> are <math id="A3.I2.i2.p1.5.m5.1" class="ltx_Math" alttext="10\times 20" display="inline"><semantics id="A3.I2.i2.p1.5.m5.1a"><mrow id="A3.I2.i2.p1.5.m5.1.1" xref="A3.I2.i2.p1.5.m5.1.1.cmml"><mn id="A3.I2.i2.p1.5.m5.1.1.2" xref="A3.I2.i2.p1.5.m5.1.1.2.cmml">10</mn><mo lspace="0.222em" rspace="0.222em" id="A3.I2.i2.p1.5.m5.1.1.1" xref="A3.I2.i2.p1.5.m5.1.1.1.cmml">×</mo><mn id="A3.I2.i2.p1.5.m5.1.1.3" xref="A3.I2.i2.p1.5.m5.1.1.3.cmml">20</mn></mrow><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.5.m5.1b"><apply id="A3.I2.i2.p1.5.m5.1.1.cmml" xref="A3.I2.i2.p1.5.m5.1.1"><times id="A3.I2.i2.p1.5.m5.1.1.1.cmml" xref="A3.I2.i2.p1.5.m5.1.1.1"></times><cn type="integer" id="A3.I2.i2.p1.5.m5.1.1.2.cmml" xref="A3.I2.i2.p1.5.m5.1.1.2">10</cn><cn type="integer" id="A3.I2.i2.p1.5.m5.1.1.3.cmml" xref="A3.I2.i2.p1.5.m5.1.1.3">20</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.5.m5.1c">10\times 20</annotation></semantics></math> matrices, with each individual elements in <math id="A3.I2.i2.p1.6.m6.1" class="ltx_Math" alttext="U" display="inline"><semantics id="A3.I2.i2.p1.6.m6.1a"><mi id="A3.I2.i2.p1.6.m6.1.1" xref="A3.I2.i2.p1.6.m6.1.1.cmml">U</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.6.m6.1b"><ci id="A3.I2.i2.p1.6.m6.1.1.cmml" xref="A3.I2.i2.p1.6.m6.1.1">𝑈</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.6.m6.1c">U</annotation></semantics></math> and <math id="A3.I2.i2.p1.7.m7.1" class="ltx_Math" alttext="V" display="inline"><semantics id="A3.I2.i2.p1.7.m7.1a"><mi id="A3.I2.i2.p1.7.m7.1.1" xref="A3.I2.i2.p1.7.m7.1.1.cmml">V</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.7.m7.1b"><ci id="A3.I2.i2.p1.7.m7.1.1.cmml" xref="A3.I2.i2.p1.7.m7.1.1">𝑉</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.7.m7.1c">V</annotation></semantics></math> sampled from <math id="A3.I2.i2.p1.8.m8.2" class="ltx_Math" alttext="\mathcal{N}(0,0.22)" display="inline"><semantics id="A3.I2.i2.p1.8.m8.2a"><mrow id="A3.I2.i2.p1.8.m8.2.3" xref="A3.I2.i2.p1.8.m8.2.3.cmml"><mi class="ltx_font_mathcaligraphic" id="A3.I2.i2.p1.8.m8.2.3.2" xref="A3.I2.i2.p1.8.m8.2.3.2.cmml">𝒩</mi><mo lspace="0em" rspace="0em" id="A3.I2.i2.p1.8.m8.2.3.1" xref="A3.I2.i2.p1.8.m8.2.3.1.cmml">​</mo><mrow id="A3.I2.i2.p1.8.m8.2.3.3.2" xref="A3.I2.i2.p1.8.m8.2.3.3.1.cmml"><mo stretchy="false" id="A3.I2.i2.p1.8.m8.2.3.3.2.1" xref="A3.I2.i2.p1.8.m8.2.3.3.1.cmml">(</mo><mn id="A3.I2.i2.p1.8.m8.1.1" xref="A3.I2.i2.p1.8.m8.1.1.cmml">0</mn><mo id="A3.I2.i2.p1.8.m8.2.3.3.2.2" xref="A3.I2.i2.p1.8.m8.2.3.3.1.cmml">,</mo><mn id="A3.I2.i2.p1.8.m8.2.2" xref="A3.I2.i2.p1.8.m8.2.2.cmml">0.22</mn><mo stretchy="false" id="A3.I2.i2.p1.8.m8.2.3.3.2.3" xref="A3.I2.i2.p1.8.m8.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.8.m8.2b"><apply id="A3.I2.i2.p1.8.m8.2.3.cmml" xref="A3.I2.i2.p1.8.m8.2.3"><times id="A3.I2.i2.p1.8.m8.2.3.1.cmml" xref="A3.I2.i2.p1.8.m8.2.3.1"></times><ci id="A3.I2.i2.p1.8.m8.2.3.2.cmml" xref="A3.I2.i2.p1.8.m8.2.3.2">𝒩</ci><interval closure="open" id="A3.I2.i2.p1.8.m8.2.3.3.1.cmml" xref="A3.I2.i2.p1.8.m8.2.3.3.2"><cn type="integer" id="A3.I2.i2.p1.8.m8.1.1.cmml" xref="A3.I2.i2.p1.8.m8.1.1">0</cn><cn type="float" id="A3.I2.i2.p1.8.m8.2.2.cmml" xref="A3.I2.i2.p1.8.m8.2.2">0.22</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.8.m8.2c">\mathcal{N}(0,0.22)</annotation></semantics></math> . Models are given 50% of the entries of <math id="A3.I2.i2.p1.9.m9.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A3.I2.i2.p1.9.m9.1a"><mi id="A3.I2.i2.p1.9.m9.1.1" xref="A3.I2.i2.p1.9.m9.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.9.m9.1b"><ci id="A3.I2.i2.p1.9.m9.1.1.cmml" xref="A3.I2.i2.p1.9.m9.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.9.m9.1c">M</annotation></semantics></math> are tasked with recovering all entries of <math id="A3.I2.i2.p1.10.m10.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A3.I2.i2.p1.10.m10.1a"><mi id="A3.I2.i2.p1.10.m10.1.1" xref="A3.I2.i2.p1.10.m10.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.10.m10.1b"><ci id="A3.I2.i2.p1.10.m10.1.1.cmml" xref="A3.I2.i2.p1.10.m10.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.10.m10.1c">M</annotation></semantics></math>. When constructing more difficult matrix completion problems at test time, each element in <math id="A3.I2.i2.p1.11.m11.1" class="ltx_Math" alttext="U" display="inline"><semantics id="A3.I2.i2.p1.11.m11.1a"><mi id="A3.I2.i2.p1.11.m11.1.1" xref="A3.I2.i2.p1.11.m11.1.1.cmml">U</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.11.m11.1b"><ci id="A3.I2.i2.p1.11.m11.1.1.cmml" xref="A3.I2.i2.p1.11.m11.1.1">𝑈</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.11.m11.1c">U</annotation></semantics></math> and <math id="A3.I2.i2.p1.12.m12.1" class="ltx_Math" alttext="V" display="inline"><semantics id="A3.I2.i2.p1.12.m12.1a"><mi id="A3.I2.i2.p1.12.m12.1.1" xref="A3.I2.i2.p1.12.m12.1.1.cmml">V</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.12.m12.1b"><ci id="A3.I2.i2.p1.12.m12.1.1.cmml" xref="A3.I2.i2.p1.12.m12.1.1">𝑉</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.12.m12.1c">V</annotation></semantics></math> are sampled from <math id="A3.I2.i2.p1.13.m13.2" class="ltx_Math" alttext="\mathcal{N}(0,0.47)" display="inline"><semantics id="A3.I2.i2.p1.13.m13.2a"><mrow id="A3.I2.i2.p1.13.m13.2.3" xref="A3.I2.i2.p1.13.m13.2.3.cmml"><mi class="ltx_font_mathcaligraphic" id="A3.I2.i2.p1.13.m13.2.3.2" xref="A3.I2.i2.p1.13.m13.2.3.2.cmml">𝒩</mi><mo lspace="0em" rspace="0em" id="A3.I2.i2.p1.13.m13.2.3.1" xref="A3.I2.i2.p1.13.m13.2.3.1.cmml">​</mo><mrow id="A3.I2.i2.p1.13.m13.2.3.3.2" xref="A3.I2.i2.p1.13.m13.2.3.3.1.cmml"><mo stretchy="false" id="A3.I2.i2.p1.13.m13.2.3.3.2.1" xref="A3.I2.i2.p1.13.m13.2.3.3.1.cmml">(</mo><mn id="A3.I2.i2.p1.13.m13.1.1" xref="A3.I2.i2.p1.13.m13.1.1.cmml">0</mn><mo id="A3.I2.i2.p1.13.m13.2.3.3.2.2" xref="A3.I2.i2.p1.13.m13.2.3.3.1.cmml">,</mo><mn id="A3.I2.i2.p1.13.m13.2.2" xref="A3.I2.i2.p1.13.m13.2.2.cmml">0.47</mn><mo stretchy="false" id="A3.I2.i2.p1.13.m13.2.3.3.2.3" xref="A3.I2.i2.p1.13.m13.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A3.I2.i2.p1.13.m13.2b"><apply id="A3.I2.i2.p1.13.m13.2.3.cmml" xref="A3.I2.i2.p1.13.m13.2.3"><times id="A3.I2.i2.p1.13.m13.2.3.1.cmml" xref="A3.I2.i2.p1.13.m13.2.3.1"></times><ci id="A3.I2.i2.p1.13.m13.2.3.2.cmml" xref="A3.I2.i2.p1.13.m13.2.3.2">𝒩</ci><interval closure="open" id="A3.I2.i2.p1.13.m13.2.3.3.1.cmml" xref="A3.I2.i2.p1.13.m13.2.3.3.2"><cn type="integer" id="A3.I2.i2.p1.13.m13.1.1.cmml" xref="A3.I2.i2.p1.13.m13.1.1">0</cn><cn type="float" id="A3.I2.i2.p1.13.m13.2.2.cmml" xref="A3.I2.i2.p1.13.m13.2.2">0.47</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i2.p1.13.m13.2c">\mathcal{N}(0,0.47)</annotation></semantics></math>.</p>
</div>
</li>
<li id="A3.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="A3.I2.i3.p1" class="ltx_para">
<p id="A3.I2.i3.p1.5" class="ltx_p"><span id="A3.I2.i3.p1.5.1" class="ltx_text ltx_font_italic">Matrix Inverse:</span> We randomly construct a well conditioned invertible matrix <math id="A3.I2.i3.p1.1.m1.1" class="ltx_Math" alttext="M=R+R^{T}+0.5*I" display="inline"><semantics id="A3.I2.i3.p1.1.m1.1a"><mrow id="A3.I2.i3.p1.1.m1.1.1" xref="A3.I2.i3.p1.1.m1.1.1.cmml"><mi id="A3.I2.i3.p1.1.m1.1.1.2" xref="A3.I2.i3.p1.1.m1.1.1.2.cmml">M</mi><mo id="A3.I2.i3.p1.1.m1.1.1.1" xref="A3.I2.i3.p1.1.m1.1.1.1.cmml">=</mo><mrow id="A3.I2.i3.p1.1.m1.1.1.3" xref="A3.I2.i3.p1.1.m1.1.1.3.cmml"><mi id="A3.I2.i3.p1.1.m1.1.1.3.2" xref="A3.I2.i3.p1.1.m1.1.1.3.2.cmml">R</mi><mo id="A3.I2.i3.p1.1.m1.1.1.3.1" xref="A3.I2.i3.p1.1.m1.1.1.3.1.cmml">+</mo><msup id="A3.I2.i3.p1.1.m1.1.1.3.3" xref="A3.I2.i3.p1.1.m1.1.1.3.3.cmml"><mi id="A3.I2.i3.p1.1.m1.1.1.3.3.2" xref="A3.I2.i3.p1.1.m1.1.1.3.3.2.cmml">R</mi><mi id="A3.I2.i3.p1.1.m1.1.1.3.3.3" xref="A3.I2.i3.p1.1.m1.1.1.3.3.3.cmml">T</mi></msup><mo id="A3.I2.i3.p1.1.m1.1.1.3.1a" xref="A3.I2.i3.p1.1.m1.1.1.3.1.cmml">+</mo><mrow id="A3.I2.i3.p1.1.m1.1.1.3.4" xref="A3.I2.i3.p1.1.m1.1.1.3.4.cmml"><mn id="A3.I2.i3.p1.1.m1.1.1.3.4.2" xref="A3.I2.i3.p1.1.m1.1.1.3.4.2.cmml">0.5</mn><mo lspace="0.222em" rspace="0.222em" id="A3.I2.i3.p1.1.m1.1.1.3.4.1" xref="A3.I2.i3.p1.1.m1.1.1.3.4.1.cmml">∗</mo><mi id="A3.I2.i3.p1.1.m1.1.1.3.4.3" xref="A3.I2.i3.p1.1.m1.1.1.3.4.3.cmml">I</mi></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A3.I2.i3.p1.1.m1.1b"><apply id="A3.I2.i3.p1.1.m1.1.1.cmml" xref="A3.I2.i3.p1.1.m1.1.1"><eq id="A3.I2.i3.p1.1.m1.1.1.1.cmml" xref="A3.I2.i3.p1.1.m1.1.1.1"></eq><ci id="A3.I2.i3.p1.1.m1.1.1.2.cmml" xref="A3.I2.i3.p1.1.m1.1.1.2">𝑀</ci><apply id="A3.I2.i3.p1.1.m1.1.1.3.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3"><plus id="A3.I2.i3.p1.1.m1.1.1.3.1.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.1"></plus><ci id="A3.I2.i3.p1.1.m1.1.1.3.2.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.2">𝑅</ci><apply id="A3.I2.i3.p1.1.m1.1.1.3.3.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.3"><csymbol cd="ambiguous" id="A3.I2.i3.p1.1.m1.1.1.3.3.1.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.3">superscript</csymbol><ci id="A3.I2.i3.p1.1.m1.1.1.3.3.2.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.3.2">𝑅</ci><ci id="A3.I2.i3.p1.1.m1.1.1.3.3.3.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.3.3">𝑇</ci></apply><apply id="A3.I2.i3.p1.1.m1.1.1.3.4.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.4"><times id="A3.I2.i3.p1.1.m1.1.1.3.4.1.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.4.1"></times><cn type="float" id="A3.I2.i3.p1.1.m1.1.1.3.4.2.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.4.2">0.5</cn><ci id="A3.I2.i3.p1.1.m1.1.1.3.4.3.cmml" xref="A3.I2.i3.p1.1.m1.1.1.3.4.3">𝐼</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i3.p1.1.m1.1c">M=R+R^{T}+0.5*I</annotation></semantics></math>, where <math id="A3.I2.i3.p1.2.m2.1" class="ltx_Math" alttext="R" display="inline"><semantics id="A3.I2.i3.p1.2.m2.1a"><mi id="A3.I2.i3.p1.2.m2.1.1" xref="A3.I2.i3.p1.2.m2.1.1.cmml">R</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i3.p1.2.m2.1b"><ci id="A3.I2.i3.p1.2.m2.1.1.cmml" xref="A3.I2.i3.p1.2.m2.1.1">𝑅</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i3.p1.2.m2.1c">R</annotation></semantics></math> is a random matrix, with individual elements sampled between -1 and 1. Models are tasked with computing the matrix inverse of <math id="A3.I2.i3.p1.3.m3.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A3.I2.i3.p1.3.m3.1a"><mi id="A3.I2.i3.p1.3.m3.1.1" xref="A3.I2.i3.p1.3.m3.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i3.p1.3.m3.1b"><ci id="A3.I2.i3.p1.3.m3.1.1.cmml" xref="A3.I2.i3.p1.3.m3.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i3.p1.3.m3.1c">M</annotation></semantics></math>. When constructing more difficult matrix inversion problems at test time, we make <math id="A3.I2.i3.p1.4.m4.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A3.I2.i3.p1.4.m4.1a"><mi id="A3.I2.i3.p1.4.m4.1.1" xref="A3.I2.i3.p1.4.m4.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A3.I2.i3.p1.4.m4.1b"><ci id="A3.I2.i3.p1.4.m4.1.1.cmml" xref="A3.I2.i3.p1.4.m4.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i3.p1.4.m4.1c">M</annotation></semantics></math> less well-conditioned by setting <math id="A3.I2.i3.p1.5.m5.1" class="ltx_Math" alttext="M=R+R^{T}+0.1*I" display="inline"><semantics id="A3.I2.i3.p1.5.m5.1a"><mrow id="A3.I2.i3.p1.5.m5.1.1" xref="A3.I2.i3.p1.5.m5.1.1.cmml"><mi id="A3.I2.i3.p1.5.m5.1.1.2" xref="A3.I2.i3.p1.5.m5.1.1.2.cmml">M</mi><mo id="A3.I2.i3.p1.5.m5.1.1.1" xref="A3.I2.i3.p1.5.m5.1.1.1.cmml">=</mo><mrow id="A3.I2.i3.p1.5.m5.1.1.3" xref="A3.I2.i3.p1.5.m5.1.1.3.cmml"><mi id="A3.I2.i3.p1.5.m5.1.1.3.2" xref="A3.I2.i3.p1.5.m5.1.1.3.2.cmml">R</mi><mo id="A3.I2.i3.p1.5.m5.1.1.3.1" xref="A3.I2.i3.p1.5.m5.1.1.3.1.cmml">+</mo><msup id="A3.I2.i3.p1.5.m5.1.1.3.3" xref="A3.I2.i3.p1.5.m5.1.1.3.3.cmml"><mi id="A3.I2.i3.p1.5.m5.1.1.3.3.2" xref="A3.I2.i3.p1.5.m5.1.1.3.3.2.cmml">R</mi><mi id="A3.I2.i3.p1.5.m5.1.1.3.3.3" xref="A3.I2.i3.p1.5.m5.1.1.3.3.3.cmml">T</mi></msup><mo id="A3.I2.i3.p1.5.m5.1.1.3.1a" xref="A3.I2.i3.p1.5.m5.1.1.3.1.cmml">+</mo><mrow id="A3.I2.i3.p1.5.m5.1.1.3.4" xref="A3.I2.i3.p1.5.m5.1.1.3.4.cmml"><mn id="A3.I2.i3.p1.5.m5.1.1.3.4.2" xref="A3.I2.i3.p1.5.m5.1.1.3.4.2.cmml">0.1</mn><mo lspace="0.222em" rspace="0.222em" id="A3.I2.i3.p1.5.m5.1.1.3.4.1" xref="A3.I2.i3.p1.5.m5.1.1.3.4.1.cmml">∗</mo><mi id="A3.I2.i3.p1.5.m5.1.1.3.4.3" xref="A3.I2.i3.p1.5.m5.1.1.3.4.3.cmml">I</mi></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A3.I2.i3.p1.5.m5.1b"><apply id="A3.I2.i3.p1.5.m5.1.1.cmml" xref="A3.I2.i3.p1.5.m5.1.1"><eq id="A3.I2.i3.p1.5.m5.1.1.1.cmml" xref="A3.I2.i3.p1.5.m5.1.1.1"></eq><ci id="A3.I2.i3.p1.5.m5.1.1.2.cmml" xref="A3.I2.i3.p1.5.m5.1.1.2">𝑀</ci><apply id="A3.I2.i3.p1.5.m5.1.1.3.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3"><plus id="A3.I2.i3.p1.5.m5.1.1.3.1.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.1"></plus><ci id="A3.I2.i3.p1.5.m5.1.1.3.2.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.2">𝑅</ci><apply id="A3.I2.i3.p1.5.m5.1.1.3.3.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.3"><csymbol cd="ambiguous" id="A3.I2.i3.p1.5.m5.1.1.3.3.1.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.3">superscript</csymbol><ci id="A3.I2.i3.p1.5.m5.1.1.3.3.2.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.3.2">𝑅</ci><ci id="A3.I2.i3.p1.5.m5.1.1.3.3.3.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.3.3">𝑇</ci></apply><apply id="A3.I2.i3.p1.5.m5.1.1.3.4.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.4"><times id="A3.I2.i3.p1.5.m5.1.1.3.4.1.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.4.1"></times><cn type="float" id="A3.I2.i3.p1.5.m5.1.1.3.4.2.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.4.2">0.1</cn><ci id="A3.I2.i3.p1.5.m5.1.1.3.4.3.cmml" xref="A3.I2.i3.p1.5.m5.1.1.3.4.3">𝐼</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A3.I2.i3.p1.5.m5.1c">M=R+R^{T}+0.1*I</annotation></semantics></math>.</p>
</div>
</li>
</ol>
</div>
</section>
<section id="A4" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix D </span>Model Architectures</h2>

<div id="A4.p1" class="ltx_para">
<p id="A4.p1.7" class="ltx_p"><span id="A4.p1.7.1" class="ltx_text ltx_font_bold">Graphical Algorithmic Computation</span>  For each iterative and feedforward method, we utilize the GINEConv layer from <cite class="ltx_cite ltx_citemacro_citep">(Hu et al., <a href="#bib.bib23" title="" class="ltx_ref">2019</a>)</cite>, where <math id="A4.p1.1.m1.1" class="ltx_math_unparsed" alttext="\text{GINEConv(}128,128)" display="inline"><semantics id="A4.p1.1.m1.1a"><mrow id="A4.p1.1.m1.1b"><mtext id="A4.p1.1.m1.1.2">GINEConv(</mtext><mn id="A4.p1.1.m1.1.3">128</mn><mo id="A4.p1.1.m1.1.4">,</mo><mn id="A4.p1.1.m1.1.1">128</mn><mo stretchy="false" id="A4.p1.1.m1.1.5">)</mo></mrow><annotation encoding="application/x-tex" id="A4.p1.1.m1.1c">\text{GINEConv(}128,128)</annotation></semantics></math> refers to a graph convolution operator with node features 128 and edge feature 128 . An input problem instance <math id="A4.p1.2.m2.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="A4.p1.2.m2.1a"><mi id="A4.p1.2.m2.1.1" xref="A4.p1.2.m2.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="A4.p1.2.m2.1b"><ci id="A4.p1.2.m2.1.1.cmml" xref="A4.p1.2.m2.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.2.m2.1c">{\bm{x}}</annotation></semantics></math> consists of a set of nodes features <math id="A4.p1.3.m3.1" class="ltx_Math" alttext="{\bm{v}}" display="inline"><semantics id="A4.p1.3.m3.1a"><mi id="A4.p1.3.m3.1.1" xref="A4.p1.3.m3.1.1.cmml">𝒗</mi><annotation-xml encoding="MathML-Content" id="A4.p1.3.m3.1b"><ci id="A4.p1.3.m3.1.1.cmml" xref="A4.p1.3.m3.1.1">𝒗</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.3.m3.1c">{\bm{v}}</annotation></semantics></math> and edge features <math id="A4.p1.4.m4.1" class="ltx_Math" alttext="{\bm{e}}" display="inline"><semantics id="A4.p1.4.m4.1a"><mi id="A4.p1.4.m4.1.1" xref="A4.p1.4.m4.1.1.cmml">𝒆</mi><annotation-xml encoding="MathML-Content" id="A4.p1.4.m4.1b"><ci id="A4.p1.4.m4.1.1.cmml" xref="A4.p1.4.m4.1.1">𝒆</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.4.m4.1c">{\bm{e}}</annotation></semantics></math>. To parameterize the EBM in IREM <math id="A4.p1.5.m5.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="A4.p1.5.m5.2a"><mrow id="A4.p1.5.m5.2.3" xref="A4.p1.5.m5.2.3.cmml"><msub id="A4.p1.5.m5.2.3.2" xref="A4.p1.5.m5.2.3.2.cmml"><mi id="A4.p1.5.m5.2.3.2.2" xref="A4.p1.5.m5.2.3.2.2.cmml">E</mi><mi id="A4.p1.5.m5.2.3.2.3" xref="A4.p1.5.m5.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="A4.p1.5.m5.2.3.1" xref="A4.p1.5.m5.2.3.1.cmml">​</mo><mrow id="A4.p1.5.m5.2.3.3.2" xref="A4.p1.5.m5.2.3.3.1.cmml"><mo stretchy="false" id="A4.p1.5.m5.2.3.3.2.1" xref="A4.p1.5.m5.2.3.3.1.cmml">(</mo><mi id="A4.p1.5.m5.1.1" xref="A4.p1.5.m5.1.1.cmml">𝒙</mi><mo id="A4.p1.5.m5.2.3.3.2.2" xref="A4.p1.5.m5.2.3.3.1.cmml">,</mo><mi id="A4.p1.5.m5.2.2" xref="A4.p1.5.m5.2.2.cmml">𝒚</mi><mo stretchy="false" id="A4.p1.5.m5.2.3.3.2.3" xref="A4.p1.5.m5.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.p1.5.m5.2b"><apply id="A4.p1.5.m5.2.3.cmml" xref="A4.p1.5.m5.2.3"><times id="A4.p1.5.m5.2.3.1.cmml" xref="A4.p1.5.m5.2.3.1"></times><apply id="A4.p1.5.m5.2.3.2.cmml" xref="A4.p1.5.m5.2.3.2"><csymbol cd="ambiguous" id="A4.p1.5.m5.2.3.2.1.cmml" xref="A4.p1.5.m5.2.3.2">subscript</csymbol><ci id="A4.p1.5.m5.2.3.2.2.cmml" xref="A4.p1.5.m5.2.3.2.2">𝐸</ci><ci id="A4.p1.5.m5.2.3.2.3.cmml" xref="A4.p1.5.m5.2.3.2.3">𝜃</ci></apply><interval closure="open" id="A4.p1.5.m5.2.3.3.1.cmml" xref="A4.p1.5.m5.2.3.3.2"><ci id="A4.p1.5.m5.1.1.cmml" xref="A4.p1.5.m5.1.1">𝒙</ci><ci id="A4.p1.5.m5.2.2.cmml" xref="A4.p1.5.m5.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.5.m5.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math> we concatenate the optimized prediction <math id="A4.p1.6.m6.1" class="ltx_Math" alttext="\hat{{\bm{y}}}" display="inline"><semantics id="A4.p1.6.m6.1a"><mover accent="true" id="A4.p1.6.m6.1.1" xref="A4.p1.6.m6.1.1.cmml"><mi id="A4.p1.6.m6.1.1.2" xref="A4.p1.6.m6.1.1.2.cmml">𝒚</mi><mo id="A4.p1.6.m6.1.1.1" xref="A4.p1.6.m6.1.1.1.cmml">^</mo></mover><annotation-xml encoding="MathML-Content" id="A4.p1.6.m6.1b"><apply id="A4.p1.6.m6.1.1.cmml" xref="A4.p1.6.m6.1.1"><ci id="A4.p1.6.m6.1.1.1.cmml" xref="A4.p1.6.m6.1.1.1">^</ci><ci id="A4.p1.6.m6.1.1.2.cmml" xref="A4.p1.6.m6.1.1.2">𝒚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.6.m6.1c">\hat{{\bm{y}}}</annotation></semantics></math>, with <math id="A4.p1.7.m7.1" class="ltx_Math" alttext="{\bm{e}}" display="inline"><semantics id="A4.p1.7.m7.1a"><mi id="A4.p1.7.m7.1.1" xref="A4.p1.7.m7.1.1.cmml">𝒆</mi><annotation-xml encoding="MathML-Content" id="A4.p1.7.m7.1b"><ci id="A4.p1.7.m7.1.1.cmml" xref="A4.p1.7.m7.1.1">𝒆</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.p1.7.m7.1c">{\bm{e}}</annotation></semantics></math>, which then utilized in the GINEConv layer. To obtain per edge predictions for baselines, we pairwise concatenate node features for the given edges, and apply an FC layer to obtain the corresponding prediction following <cite class="ltx_cite ltx_citemacro_citep">(Zhang &amp; Chen, <a href="#bib.bib42" title="" class="ltx_ref">2018</a>)</cite>. We specify the architecture for IREM in Table <a href="#A4.T6" title="Table 6 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>, the architecture for feedforward and iterative feedforward baselines in Table <a href="#A4.T7" title="Table 7 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>, the architecture for recurrent baselines in Table <a href="#A4.T8" title="Table 8 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>, and the architecture for programmatic execution baselines in Table <a href="#A4.T9" title="Table 9 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a>. All models have roughly the same number of underlying parameters.</p>
</div>
<figure id="A4.2" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.1.1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.1.1.1" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.1.1.1.2.1" class="ltx_tr">
<th id="A4.1.1.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.1.1.1.2.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.1.1.1.3.1" class="ltx_tr">
<td id="A4.1.1.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.1.1.1.3.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></td>
</tr>
<tr id="A4.1.1.1.4.2" class="ltx_tr">
<td id="A4.1.1.1.4.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.1.1.1.4.2.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></td>
</tr>
<tr id="A4.1.1.1.1" class="ltx_tr">
<td id="A4.1.1.1.1.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.1.1.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear 128 </span><math id="A4.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.1.1.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.1.1.1.1.1.m1.1.1" xref="A4.1.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.1.1.1.1.1.m1.1b"><ci id="A4.1.1.1.1.1.m1.1.1.cmml" xref="A4.1.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.1.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.1.1.1.1.1.2" class="ltx_text" style="font-size:90%;"> 1</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T6" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 6: </span>The model architecture for IREM on graphical tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.2.2" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.2.2.1" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.2.2.1.2.1" class="ltx_tr">
<th id="A4.2.2.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.2.2.1.2.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.2.2.1.3.1" class="ltx_tr">
<td id="A4.2.2.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.2.2.1.3.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></td>
</tr>
<tr id="A4.2.2.1.4.2" class="ltx_tr">
<td id="A4.2.2.1.4.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.2.2.1.4.2.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></td>
</tr>
<tr id="A4.2.2.1.1" class="ltx_tr">
<td id="A4.2.2.1.1.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.2.2.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear 256 </span><math id="A4.2.2.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.2.2.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.2.2.1.1.1.m1.1.1" xref="A4.2.2.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.2.2.1.1.1.m1.1b"><ci id="A4.2.2.1.1.1.m1.1.1.cmml" xref="A4.2.2.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.2.2.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.2.2.1.1.1.2" class="ltx_text" style="font-size:90%;"> Output Dim</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T7" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 7: </span>The model architecture for feedforward and iterative feedforward baselines on graphical tasks. </figcaption>
</figure>
</div>
</div>
</figure>
</div>
</div>
</figure>
<figure id="A4.5" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.3.1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.3.1.1" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.3.1.1.2.1" class="ltx_tr">
<th id="A4.3.1.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.3.1.1.2.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.3.1.1.3.1" class="ltx_tr">
<td id="A4.3.1.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.3.1.1.3.1.1.1" class="ltx_text" style="font-size:90%;">LSTM(128)</span></td>
</tr>
<tr id="A4.3.1.1.4.2" class="ltx_tr">
<td id="A4.3.1.1.4.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.3.1.1.4.2.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></td>
</tr>
<tr id="A4.3.1.1.1" class="ltx_tr">
<td id="A4.3.1.1.1.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.3.1.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear 256 </span><math id="A4.3.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.3.1.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.3.1.1.1.1.m1.1.1" xref="A4.3.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.3.1.1.1.1.m1.1b"><ci id="A4.3.1.1.1.1.m1.1.1.cmml" xref="A4.3.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.3.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.3.1.1.1.1.2" class="ltx_text" style="font-size:90%;"> Output Dim</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T8" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 8: </span>The model architecture for recurrent baseline on graphical tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.5.3" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.5.3.2" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.5.3.2.3.1" class="ltx_tr">
<th id="A4.5.3.2.3.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.5.3.2.3.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.5.3.2.4.1" class="ltx_tr">
<td id="A4.5.3.2.4.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.5.3.2.4.1.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128)</span></td>
</tr>
<tr id="A4.5.3.2.5.2" class="ltx_tr">
<td id="A4.5.3.2.5.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.5.3.2.5.2.1.1" class="ltx_text" style="font-size:90%;">GINEConv(128, 128)</span></td>
</tr>
<tr id="A4.4.2.1.1" class="ltx_tr">
<td id="A4.4.2.1.1.1" class="ltx_td ltx_align_center ltx_border_t">
<span id="A4.4.2.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear 256 </span><math id="A4.4.2.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.4.2.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.4.2.1.1.1.m1.1.1" xref="A4.4.2.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.4.2.1.1.1.m1.1b"><ci id="A4.4.2.1.1.1.m1.1.1.cmml" xref="A4.4.2.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.4.2.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.4.2.1.1.1.2" class="ltx_text" style="font-size:90%;"> Output Dim</span>
</td>
</tr>
<tr id="A4.5.3.2.2" class="ltx_tr">
<td id="A4.5.3.2.2.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.5.3.2.2.1.1" class="ltx_text" style="font-size:90%;">Linear </span><math id="A4.5.3.2.2.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.5.3.2.2.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.5.3.2.2.1.m1.1.1" xref="A4.5.3.2.2.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.5.3.2.2.1.m1.1b"><ci id="A4.5.3.2.2.1.m1.1.1.cmml" xref="A4.5.3.2.2.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.5.3.2.2.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.5.3.2.2.1.2" class="ltx_text" style="font-size:90%;"> 1</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T9" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 9: </span>The model architecture for PonderNet baseline on graphical tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
</div>
</figure>
<div id="A4.p2" class="ltx_para">
<p id="A4.p2.3" class="ltx_p"><span id="A4.p2.3.1" class="ltx_text ltx_font_bold">Continuous Algorithmic Computation</span>  For each iterative and feedforward method, we utilize a MLP to implement continuous algorithmic computation. To parameterize the EBM in IREM <math id="A4.p2.1.m1.2" class="ltx_Math" alttext="E_{\theta}({\bm{x}},{\bm{y}})" display="inline"><semantics id="A4.p2.1.m1.2a"><mrow id="A4.p2.1.m1.2.3" xref="A4.p2.1.m1.2.3.cmml"><msub id="A4.p2.1.m1.2.3.2" xref="A4.p2.1.m1.2.3.2.cmml"><mi id="A4.p2.1.m1.2.3.2.2" xref="A4.p2.1.m1.2.3.2.2.cmml">E</mi><mi id="A4.p2.1.m1.2.3.2.3" xref="A4.p2.1.m1.2.3.2.3.cmml">θ</mi></msub><mo lspace="0em" rspace="0em" id="A4.p2.1.m1.2.3.1" xref="A4.p2.1.m1.2.3.1.cmml">​</mo><mrow id="A4.p2.1.m1.2.3.3.2" xref="A4.p2.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="A4.p2.1.m1.2.3.3.2.1" xref="A4.p2.1.m1.2.3.3.1.cmml">(</mo><mi id="A4.p2.1.m1.1.1" xref="A4.p2.1.m1.1.1.cmml">𝒙</mi><mo id="A4.p2.1.m1.2.3.3.2.2" xref="A4.p2.1.m1.2.3.3.1.cmml">,</mo><mi id="A4.p2.1.m1.2.2" xref="A4.p2.1.m1.2.2.cmml">𝒚</mi><mo stretchy="false" id="A4.p2.1.m1.2.3.3.2.3" xref="A4.p2.1.m1.2.3.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A4.p2.1.m1.2b"><apply id="A4.p2.1.m1.2.3.cmml" xref="A4.p2.1.m1.2.3"><times id="A4.p2.1.m1.2.3.1.cmml" xref="A4.p2.1.m1.2.3.1"></times><apply id="A4.p2.1.m1.2.3.2.cmml" xref="A4.p2.1.m1.2.3.2"><csymbol cd="ambiguous" id="A4.p2.1.m1.2.3.2.1.cmml" xref="A4.p2.1.m1.2.3.2">subscript</csymbol><ci id="A4.p2.1.m1.2.3.2.2.cmml" xref="A4.p2.1.m1.2.3.2.2">𝐸</ci><ci id="A4.p2.1.m1.2.3.2.3.cmml" xref="A4.p2.1.m1.2.3.2.3">𝜃</ci></apply><interval closure="open" id="A4.p2.1.m1.2.3.3.1.cmml" xref="A4.p2.1.m1.2.3.3.2"><ci id="A4.p2.1.m1.1.1.cmml" xref="A4.p2.1.m1.1.1">𝒙</ci><ci id="A4.p2.1.m1.2.2.cmml" xref="A4.p2.1.m1.2.2">𝒚</ci></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A4.p2.1.m1.2c">E_{\theta}({\bm{x}},{\bm{y}})</annotation></semantics></math>, we concatenate <math id="A4.p2.2.m2.1" class="ltx_Math" alttext="{\bm{x}}" display="inline"><semantics id="A4.p2.2.m2.1a"><mi id="A4.p2.2.m2.1.1" xref="A4.p2.2.m2.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="A4.p2.2.m2.1b"><ci id="A4.p2.2.m2.1.1.cmml" xref="A4.p2.2.m2.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.p2.2.m2.1c">{\bm{x}}</annotation></semantics></math> and <math id="A4.p2.3.m3.1" class="ltx_Math" alttext="{\bm{y}}" display="inline"><semantics id="A4.p2.3.m3.1a"><mi id="A4.p2.3.m3.1.1" xref="A4.p2.3.m3.1.1.cmml">𝒚</mi><annotation-xml encoding="MathML-Content" id="A4.p2.3.m3.1b"><ci id="A4.p2.3.m3.1.1.cmml" xref="A4.p2.3.m3.1.1">𝒚</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.p2.3.m3.1c">{\bm{y}}</annotation></semantics></math> together as input into the network. We utilize the ReLU activation in all networks except IREM, where we utilize the Swish activation. We specify the architecture for IREM in Table <a href="#A4.T10" title="Table 10 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>, the architecture for feedforward and iterative feedforward baselines in Table <a href="#A4.T11" title="Table 11 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">11</span></a>, the architecture for recurrent baselines in Table <a href="#A4.T12" title="Table 12 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">12</span></a>, and the architecture for programmatic execution baselines in Table <a href="#A4.T13" title="Table 13 ‣ Appendix D Model Architectures ‣ Learning Iterative Reasoning through Energy Minimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">13</span></a>. All models have roughly the same number of underlying parameters.</p>
</div>
<figure id="A4.7" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.6.1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.6.1.1" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.6.1.1.2.1" class="ltx_tr">
<th id="A4.6.1.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.6.1.1.2.1.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.6.1.1.3.1" class="ltx_tr">
<td id="A4.6.1.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.6.1.1.3.1.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></td>
</tr>
<tr id="A4.6.1.1.4.2" class="ltx_tr">
<td id="A4.6.1.1.4.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.6.1.1.4.2.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></td>
</tr>
<tr id="A4.6.1.1.1" class="ltx_tr">
<td id="A4.6.1.1.1.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.6.1.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear </span><math id="A4.6.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.6.1.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.6.1.1.1.1.m1.1.1" xref="A4.6.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.6.1.1.1.1.m1.1b"><ci id="A4.6.1.1.1.1.m1.1.1.cmml" xref="A4.6.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.6.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.6.1.1.1.1.2" class="ltx_text" style="font-size:90%;"> 1</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T10" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 10: </span>The model architecture for IREM on continuous tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.7.2" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.7.2.1" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.7.2.1.2.1" class="ltx_tr">
<th id="A4.7.2.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.7.2.1.2.1.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.7.2.1.3.1" class="ltx_tr">
<td id="A4.7.2.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.7.2.1.3.1.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></td>
</tr>
<tr id="A4.7.2.1.4.2" class="ltx_tr">
<td id="A4.7.2.1.4.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.7.2.1.4.2.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></td>
</tr>
<tr id="A4.7.2.1.1" class="ltx_tr">
<td id="A4.7.2.1.1.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.7.2.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear </span><math id="A4.7.2.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.7.2.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.7.2.1.1.1.m1.1.1" xref="A4.7.2.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.7.2.1.1.1.m1.1b"><ci id="A4.7.2.1.1.1.m1.1.1.cmml" xref="A4.7.2.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.7.2.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.7.2.1.1.1.2" class="ltx_text" style="font-size:90%;"> Output Dim</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T11" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 11: </span>The model architecture for feedforward and iterative feedforward baselines on continuous tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
</div>
</figure>
<figure id="A4.10" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.8.1" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.8.1.1" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.8.1.1.2.1" class="ltx_tr">
<th id="A4.8.1.1.2.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.8.1.1.2.1.1.1" class="ltx_text" style="font-size:90%;">Linear 196</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.8.1.1.3.1" class="ltx_tr">
<td id="A4.8.1.1.3.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.8.1.1.3.1.1.1" class="ltx_text" style="font-size:90%;">LSTM 196</span></td>
</tr>
<tr id="A4.8.1.1.1" class="ltx_tr">
<td id="A4.8.1.1.1.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.8.1.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear </span><math id="A4.8.1.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.8.1.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.8.1.1.1.1.m1.1.1" xref="A4.8.1.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.8.1.1.1.1.m1.1b"><ci id="A4.8.1.1.1.1.m1.1.1.cmml" xref="A4.8.1.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.8.1.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.8.1.1.1.1.2" class="ltx_text" style="font-size:90%;"> Output Dim</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T12" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 12: </span>The model architecture for recurrent baseline on continuous tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.10.3" class="ltx_figure ltx_figure_panel ltx_minipage ltx_align_middle" style="width:195.1pt;">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<table id="A4.10.3.2" class="ltx_tabular ltx_figure_panel ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="A4.10.3.2.3.1" class="ltx_tr">
<th id="A4.10.3.2.3.1.1" class="ltx_td ltx_align_center ltx_th ltx_th_column ltx_border_tt"><span id="A4.10.3.2.3.1.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="A4.10.3.2.4.1" class="ltx_tr">
<td id="A4.10.3.2.4.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.10.3.2.4.1.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></td>
</tr>
<tr id="A4.10.3.2.5.2" class="ltx_tr">
<td id="A4.10.3.2.5.2.1" class="ltx_td ltx_align_center ltx_border_t"><span id="A4.10.3.2.5.2.1.1" class="ltx_text" style="font-size:90%;">Linear 512</span></td>
</tr>
<tr id="A4.9.2.1.1" class="ltx_tr">
<td id="A4.9.2.1.1.1" class="ltx_td ltx_align_center ltx_border_t">
<span id="A4.9.2.1.1.1.1" class="ltx_text" style="font-size:90%;">Linear </span><math id="A4.9.2.1.1.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.9.2.1.1.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.9.2.1.1.1.m1.1.1" xref="A4.9.2.1.1.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.9.2.1.1.1.m1.1b"><ci id="A4.9.2.1.1.1.m1.1.1.cmml" xref="A4.9.2.1.1.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.9.2.1.1.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.9.2.1.1.1.2" class="ltx_text" style="font-size:90%;"> Output Dim</span>
</td>
</tr>
<tr id="A4.10.3.2.2" class="ltx_tr">
<td id="A4.10.3.2.2.1" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">
<span id="A4.10.3.2.2.1.1" class="ltx_text" style="font-size:90%;">Linear </span><math id="A4.10.3.2.2.1.m1.1" class="ltx_Math" alttext="\rightarrow" display="inline"><semantics id="A4.10.3.2.2.1.m1.1a"><mo mathsize="90%" stretchy="false" id="A4.10.3.2.2.1.m1.1.1" xref="A4.10.3.2.2.1.m1.1.1.cmml">→</mo><annotation-xml encoding="MathML-Content" id="A4.10.3.2.2.1.m1.1b"><ci id="A4.10.3.2.2.1.m1.1.1.cmml" xref="A4.10.3.2.2.1.m1.1.1">→</ci></annotation-xml><annotation encoding="application/x-tex" id="A4.10.3.2.2.1.m1.1c">\rightarrow</annotation></semantics></math><span id="A4.10.3.2.2.1.2" class="ltx_text" style="font-size:90%;"> 1</span>
</td>
</tr>
</tbody>
</table>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="A4.T13" class="ltx_table ltx_figure_panel ltx_align_center">
<figcaption class="ltx_caption" style="font-size:90%;"><span class="ltx_tag ltx_tag_table">Table 13: </span>The model architecture for PonderNet baseline on continuous tasks.</figcaption>
</figure>
</div>
</div>
</figure>
</div>
</div>
</figure>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2206.15447" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2206.15448" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2206.15448">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2206.15448" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2206.15449" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Tue Feb 27 03:28:59 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2206.15448
Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition.
However, it struggles with tasks requiring nontrivial reasoning, such as algorithmic computation.
Humans are able to solve such tasks through iterative reasoning – spending more time thinking about harder tasks.
Most existing neural networks, however, exhibit a fixed computational budget controlled by the neural network architecture, preventing additional computational processing on harder tasks.
In this work, we present a new framework for iterative reasoning with neural networks.
We train a neural network to parameterize an energy landscape over all outputs, and implement each step of the iterative reasoning as an energy minimization step to find a minimal energy solution.
By formulating reasoning as an energy minimization problem, for harder problems that lead to more complex energy landscapes, we may then adjust our underlying computational budget by running a more complex optimization procedure.
We empirically illustrate that our iterative reasoning approach can solve more accurate and generalizable algorithmic reasoning tasks in both graph and continuous domains.
Finally, we illustrate that our approach can recursively solve algorithmic problems requiring nested reasoning. Code and additional information is available athttps://energy-based-model.github.io/iterative-reasoning-as-energy-minimization/.
abstract
Learning Iterative Reasoning through Energy Minimization
Abstract
2206.15448
1IntroductionHuman thinking is often characterized in terms of mechanisms for two distinct modes of cognitive processing:System 1mechanisms for fast, habitual, and associative processing, andSystem 2mechanisms for slower, more deliberate and controlled, symbolic reasoning. Neural networks have excelled at habitualSystem 1-style processing in familiar environments and task contexts, such as mapping images of familiar objects to their semantic classes, or familiar locations to the corresponding routes of movement. However, confronted with a novel environment or task demanding a more flexible response, humans can invoke controlledSystem 2processes, often in the form of iteratively reasoning about relationships between observed entities that builds on past experiences primarily through shared abstractions and algorithms rather than direct re-use of concrete, habitual responses. Such flexible processing of novel inputs is difficult for neural networks, with even large pretrained language modelsthat have proven effective in many zero-shot generalization contexts failing to extrapolate the simplest algorithmic operations, such as addition, to more complex inputs.Figure 1:Reasoning as Energy Minimization– IREM formulates an algorithmic reasoning problem with inputsand output, such as summation (left) , as an iterative energy minimization problem over a learned energy functionwhich parameterizes an energy landscape over all possible outputsconditioned on inputs(right) . We visualize the first two dimensionsand. Each iteration of energy minimization is shown by a star.Iterative reasoning, the ability to repeatedly apply an underlying computation to the outputs of previous reasoning steps, is a crucial component of scalableSystem 2-style processing. We may take abstractions learned from simpler variants of a problem, and iteratively apply them to solve harder variants of potentially unbounded complexity. As an example, having learned how to compute shortest paths on small graphs, we might generalize and apply our algorithmic intuition sequentially to substantially larger graph problems at test time through iterative application of learned computations. Such iterative processing enables humans to solve more challenging tasks, such as question answering, arithmetic calculation, or proof writing, even in novel or unfamiliar domains.We present a new neural approach towards iterative reasoning, which we formulate as an energy minimization process on a learned energy landscape (Figure1) . By representing individual steps of reasoning as an optimization process, we may iteratively reason for longer on harder problems by running additional steps of optimization on the induced energy landscapes. Simultaneously, by monitoring the geometric energy landscape surrounding an optimized solution, we may automatically determine the completion of the algorithmic computation (by checking the presence of a local energy minimum) . We refer to our underlying reasoning framework as Iterative Reasoning as Energy Minimization (IREM) .To evaluate and benchmark the effectiveness of iterative reasoning using IREM, we propose and construct a suite of different algorithmic reasoning tasks on both graph and continuous domains. Effective algorithmic reasoning requires repetitive application of underlying algorithmic computations, dependent on problem complexity, and thus serves as a natural benchmark for iterative reasoning. We compare IREM with past works on our algorithmic benchmark for iterative reasoning and find that IREM outperforms prior works in performance and generalization.Our contributions in this paper are threefold. First, we present IREM, a new framework for iterative reasoning and analyze why it is beneficial to use such a framework for reasoning. Second, we present a benchmark for iterative algorithmic reasoning, both on graphs and continuous vector inputs, and show that IREM significantly outperforms prior approaches in both performance and generalization. Finally, we show how our approach can recursively solve algorithmic computation requiring nested reasoning. Our results point to IREM as a promising new approach towards iterative reasoning.
introduction
Learning Iterative Reasoning through Energy Minimization
1Introduction
2206.15448
2Related WorkIterative ReasoningA variety of recent works have explored the integration of iterative reasoning into neural networks. One branch of works implements iterative reasoning by constructing neural programmatic operationswhich are repeatedly executed till halting. Another branch of work implements iterative reasoning through recurrent computation. A key challenge with both types of approaches lies in the halting time of computation. Existing approaches learn halting policies through reinforcement learning, heuristic policies, or variational inference. Such approaches are unstable in nature, and many of them require manual hyper-parameter specification. We present an orthogonal approach towards implementing iterative reasoning as an energy minimization procedure on a learned energy landscape. By determining when a local energy minimum has been found, our approach provides a natural mechanism for terminating computation.Algorithmic Reasoning with External MemorySeveral approaches towards iterative reasoning utilize an external memory scratchpad for algorithmic computation. Such a scratchpad enables models to store intermediate algorithmic computations, and thus boosts the underlying performance of the algorithm. In the Appendix B, we illustrate a manner through which we may utilize an external memory with IREM to improve underlying further improve reasoning performance.Optimization Based Computational BlocksPrior works have explored optimization as a computation block to solve different tasks. In, a dynamical system is constructed that can solve various algorithmic tasks. Optimization has since been used as an intermediate neural network computation block for quadratic programsand submodular programsfor flexible neural networks. Most similar to our work,utilizes equilibrium energy minimization as an intermediate computation block for memory-efficient neural networks. In contrast, we explore how direct optimization over a learned energy landscape can enable generalizable iterative reasoning. Concurrent to our work,utilizes energy minimization to simulate physical dynamics.Energy-Based ModelsOur work is related to works in Energy-Based Models (EBMs) . Most recent works using EBMs have focused on learning probabilistic models over data. Instead of using EBMs as a probabilistic model, we use EBMs to define an energy landscape for solving iterative reasoning problems.Learning OptimizersOur work utilizes backpropagation through intermediate optimization steps to train our energy function. Prior work has explored a similar idea of backpropagation through optimization to learn meta optimizerswhich enable more sample efficient and faster training of neural networks. In this work, we utilize this approach to learn an energy function so that a number of optimization steps on the energy function results in a solution to the algorithmic reasoning problem.
body
Learning Iterative Reasoning through Energy Minimization
2Related Work
2206.15448
3Learning Iterative Reasoning through Energy OptimizationLetbe a dataset of algorithmic reasoning problems consisting of inputsand corresponding solutions. Our goal is to learn a neural network operatorwhich can generalizeto a test input, wherecan be significantly larger and more challenging than the training data.
We present our approach, IREM, to adaptive algorithmic computation in Section3.1, and discuss how to learn such an energy landscape in Section3.2. We further provide analysis on why such a framework is favorable in Section3.3.3.1Iterative Reasoning as Energy MinimizationIterative reasoning is typically formulated as the repeated application of a neural network operator, to generate partial solutions(1) where the final predictionis output aftersuccessive applications of. The termination of iterative computation is often difficult to specify, with methods typically utilizing a halting policy that is difficult to train.In this work, we propose an alternative approach towards iterative computation with a natural termination criteria, IREM, which represents iterative reasoning as an optimization process over an Energy-Based Model (EBM) :(2) An individual reasoning step is then represented as(3) whereis the step size of each gradient descent step. We repeat the iterative computation in Equation3untilis a local minima of the energy landscape,, where additional steps of optimization do not change the energy value of. Finding a local minimum of the energy landscape thus provides us with a natural mechanism to halt iterative computation.3.2Learning Energy Landscapes for ReasoningWe next discuss how to learn IREM. Given an input problemwith a unique solution, the simplest method to learnis to directly supervise the minimal energy statewiththrough regression(4) However, in practice, during training time, it is computationally expensive to compute, as the underlying energy landscape may be complex.As a fast approximation to, we may approximate theoperation with respect toviasteps of gradient optimization. An approximate optimumis obtained by:(5) In Equation5, we initializeusing uniform noise, withdenoting the step size for each gradient step, and samplecorresponding to the result aftersteps of gradient descent. We then minimize the corresponding loss:(6) where we may directly differentiate through the underlying optimization procedure. To alleviate the computational burden of computing second-order gradients across optimization steps, we empirically found that simply truncating back-propagation to the last optimization step maintained good performance at a significantly faster training speed as illustrated in Table2.An underlying difficulty of utilizingas an approximation ofis that since only a small finite number of steps of gradient descent is applied,may be far from reaching. As a result, when running a larger number of iterative reasoning steps at test time to more precisely compute, our underlying solution may degrade.To remedy this issue, we maintain a replay buffer of previously optimized samples, and initializeeither from previously optimized valuesor uniform noise. By initializing gradient descent optimization with previously optimized samples, we ensure that
these samples are closer to. A similar application of replay buffers has been utilized to train EBMs for consistent probability landscapes.We provide pseudocode for training IREM in Algorithm1and executing algorithmic reasoning with IREM in Algorithm2, where an energy minimum is determined at test time once the energy value of a solution does not change for certain iterations. A fixed learning rate,, is used for training IREM. At test time,is empirically tuned so that energy values decrease smoothly across iterations of optimization (harder problems require smallerto optimize) . In AppendixB, we discuss how we may utilize optimization to further incorporate an external scratchpad into iterative reasoning using IREM.As IREM directly trains an energy landscape by optimizing samples to regress solutions, there is no guarantee that a smooth underlying energy landscapeis learned. Empirically, as seen in Figure1and Figure5, we find that our objective does lead to consistent energy landscapes similar to past work.Input:Data Dist, Replay Buffer, Step Size, Number of Steps, EBM, Uniform Distributionwhilenot convergeddoSample data and candidate solutions fromand replay buffer,Generate low energy solutions through optimization:forsample steptodoendforOptimize objectivewrt:Updatebased onusing Adam optimizerUpdate replay bufferendwhileAlgorithm 1IREM training algorithm3.3AnalysisIn this section, we provide complexity-theoretic motivation for representing iterative reasoning as an energy minimization is advantageous. We consider two approaches to represent reasoning:Feedforward Computation.We consider reasoning as a feedforward functionof the formwheremaps a inputto a predicted solution.Energy Minimization.We next consider reasoning as a energy minimization problem,, whereis a function from, mapping a inputand solutioninto an energy. This corresponds to the computation represented by IREM.We construct a class of algorithmic reasoning tasks that are easier to learn using an energy functionas opposed to the feedforward function. Our result is based on the intuition that learning the energy functioncorresponds to learning a solution verifier, which assigns minimal energy to the correct solution, and high energy to all the other solutions. In contrast, learning a functioncorresponds to explicitly generating a solution. We rely on the well-known theorem in complexity theory that constructing a solution verifier is easier than a solution generator to show that learning energy minimization is easier than feedforward computation.Input:Data Dist, Step Size, Number of Steps, EBM, Uniform DistributionSample input fromand initialize candidate solutionwhileNot at Energy MinimadoOptimize candidate solutionwith gradient descent:endwhileFinal predicted solution:Algorithm 2IREM prediction algorithmTaskMethodSame SizeLarger SizeTaskMethodSame Diff.Harder Diff.EdgeFeedforward0.30160.3124AddFeedforward0.04480.7029CopyRecurrent0.30150.3113Recurrent0.36102.6133Programmatic0.30530.4409Programmatic0.01110.3446Iterative Feedforward0.61630.6498Iterative Feedforward0.01440.1577IREM (Ours) 0.00190.0019IREM (Ours) 0.00030.0021ConnectedFeedforward0.17960.3460MatrixFeedforward0.02030.2720ComponentsRecurrent0.17940.2766CompletionRecurrent0.02660.3285Programmatic0.23383.1381Programmatic0.02030.2637Iterative Feedforward0.49081.2064Iterative Feedforward0.02530.2102IREM (Ours) 0.14240.2171IREM (Ours) 0.01830.2074ShortestFeedforward0.12331.4089MatrixFeedforward0.01120.2150PathRecurrent0.12590.1083InverseRecurrent0.01090.2123Programmatic0.13750.1290Programmatic0.01240.2209Iterative Feedforward0.45880.7688Iterative Feedforward0.02700.5250IREM (Ours) 0.02740.0464IREM (Ours) 0.01080.2083Table 1:Algorithmic Reasoning with IREM–IREM is general framework for iterative reasoning which can learn algorithmic computation on both graph (left)  and continuous (right)  inputs. IREM generalizes at test time to both larger (left)  and harder (right)  algorithmic problems through iterative computation. Error reported on each task using elementwise mean square error. Approaches on the left are trained on graphs with ten nodes and evaluated on larger graphs with fifteen nodes. IREM significantly outperforms comparisons.To analyze, we consider solving the 3-SATproblem. We construct a energy functionto verify the 3-SAT formula. Given a 3-SAT formulawithvariables andclauses, we construct an energy function to representas, whereencodes the clauses in a 3-SAT problem,corresponds to boolean assignments to each variable, andverifies whether the boolean assignments insatisfies theclause. To encode a set of K clauses using, we utilize an ordinal representation (e.g.to represent a particular clause) . The energy functionis then constructed by taking in clauseand outputting 0 if the corresponding entries ofsatisfy the encoded clause and 1 otherwise (polynomial time to compute) . We assume the Exponential Time Hypothesis (ETH) , which states that checking the satisfiability of a-SAT formula takes time exponential in the sum of the number of variables and the number of clauses.Remark 1.There exists a 3-SAT problem which may be encoded in an energy functionwhich can be evaluated at any input in time polynomial in the number of dimensions ofbut for which the computational complexity of encoding a feedforward solutionwhich may be evaluated at any input is (worse-case)  exponential in the number of dimensions of.Proof.We encode the 3-SAT energy functionas defined above, which is evaluated in time polynomial in the number of dimensions of. In contrast, ETH directly implies that constructing, which corresponds to solving the 3-SAT problem, is exponential in dimension of.
∎Our remark shows that it is computationally advantageous to represent an energy functionas opposed to a feedforward decoder. In particular, learning neural networksandto approximate eitheror, our remark implies that a larger network is necessary to represent the exponential computations of. Such a networkwould require either exponentially deeper or wider layers, if there is no iterative computation. With iterative computation, we may parameterizeandwith a similar number of parameters, but then it would be necessary forto be iteratively applied a exponential number of times. Direct recurrent backpropagation through such a number of iterative computations has been proven to be unstable to train. While in principle computing, would require a similar number of computations, we may trainwith a simple inexact energy minimization procedure and run a more extensive minimization procedure at test time. We next analyze the computational complexity of representingandas a function of problem size.Remark 2.As the underlying number of variables in 3-SAT problem increases, we may constructwhich can be evaluated at any input in time polynomial in the number of variables, but for which the computational complexity of encoding a feedforward solutionwhich may be evaluated at any input is (worse-case)  exponential in the number of underlying variables.Proof.Our constructed energy functionabove may be evaluated in time polynomial in the number of input variables. In contrast, ETH implies that constructing, which corresponds to solving the 3-SAT problem, is exponential in the number of underlying variables.
∎Similar to our previous remark, our result implies that representingrequires exponentially more computation as the underlying problem size of a 3-SAT problem increases. In particular, our remark has implications forgeneralization. If we learnandon smaller problem instances of, for our neural networks to generalize correctly to larger problems instances, the underlying computation executed by the neural network must increase polynomially forand exponentially for. Existing architectures, such as transformers and graph networks, adaptively increase computation polynomially for larger inputs. In contrast, few architectures can adaptively increase computation exponentially for larger problem instances. To realize, we thus require iterative approaches which can execute exponentially longer on larger inputs. As seen in Figure2, this is difficult for existing approaches.
body
Learning Iterative Reasoning through Energy Minimization
3Learning Iterative Reasoning through Energy Optimization
2206.15448
4Experiments4.1Experimental SetupWe compare IREM with feedforward and iterative baselines for reasoning. We discuss each approach in detail below.Feedforward Computation.First, we compare with (one-step)  feedforward computation, where we train a neural network that directly outputs the values of solutions.Recurrent Network Computation.Next, we compare our approach with methods utilizing a recurrent neural network to execute iterative computation. Recurrent architectures have been shown to successfully execute reasoning recently. We use a LSTM network to represent iterative computation.Learned Programmatic Computation.We compare our method with past works which construct iterative computation through building programmatic structures with neural networks. We compare our method with the recent architecture and training objective of PonderNet, which variationally learns both a halting probability and individual computation step networks.Iterative Feedforward Computation.We further compare our approach with direct iterative application of a feedforward computation. We train the iterative feedforward computation using an iterative denoising objective.We scale network sizes to ensure that each individual baseline has roughly the same number of parameters. The architectural details of each model are provided in AppendixDof the paper. We utilize MLP neural networks for continuous algorithmic reasoning tasks and graph neural networks for graph algorithmic reasoning tasks. All iterative methods are trained with 5 steps of reasoning. We provide comparisons with each baseline on graphical algorithmic reasoning in Section4.2and on continuous algorithmic reasoning in Section4.3. We provide additional benchmark comparisons of IREM on an existing iterative image denoising task in AppendixA.4.2Graphical Algorithmic ReasoningFigure 2:Computation Steps vs Problem Size– Error on shortest path computation as a factor of graph size and number of computation executed. Models are trained 5 steps of iterative computation on graph sizes of 10. IREM is able to generalize computation to larger graphs and a greater number of iterative computation steps, while a recurrent network fails to do so.Setup.We first evaluate our approach on graphical algorithmic reasoning. We train models on fully connected graphs with between 2 to 10 nodes, and evaluate performance on larger fully connected graphs with size 15 nodes. We report the underlying elementwise mean squared error between predictions from models and their associated ground truth outputs. We evaluate performance on the three different graphical algorithmic reasoning tasks, aiming to capture different aspects of reasoning, which we detail below, with additional details about each dataset in AppendixC.1.Edge Copy: We first test the ability of models to copy and output the values of all edges in a dense graph that is given as input. This task serves as a simple test for iterative reasoning, and requires a method to sequentially copy over input edge values to the output.2.Connected Components: Next, we evaluate the ability of models to infer the underlying connected components of a graph. We construct a sparse graph, where 5% of all fully connected edges exist, and ask models to predict a binary indicator on whether a node of a graph is connected to another for all pairs of nodes in a graph. This task tests structural discovery, an aspect of cognitive reasoning.3.Shortest Path: Finally, we evaluate the ability of models to compute the shortest path distances between all pairs of nodes in a graph. This task tests for planning, and the underlying calculation necessary to compute the shortest paths between nodes is analogous to that of planning.Quantitative Results.We present quantitative results on graphical algorithmic reasoning in the left column of Table1. During test time, we evaluate on problems with either similar or larger sizes than those seen during training. Across all three tasks, IREM outperforms all the compared baselines. This difference in performance is magnified when evaluating on larger graphs during testing. Approaches other than IREM fail even on the relatively simple edge copy task. This is due to the fact that the small size of graph networks (hidden size 128) , requires methods to iteratively copy different subsets of input edges to output predictions. While IREM successfully learns this iterative computation, the compared baselines were unable to do so.ReplayTruncateTestTrainingMemoryBufferGradientPerformanceSpeedUsageNoNo0.049187%295%YesNo0.028783%295%YesYes0.0274100%100%Table 2:Ablations –Ablations of proposed components of IREM on test performance on the shortest path algorithmic reasoning task. The use a replay buffer boosts the underlying performance of IREM and truncating gradient backpropogation to the last step of optimization reduces both training time and memory cost.Adaptive Computation.Next, we analyze the ability of IREM and baselines to adapt its underlying computational budget to different larger problem instances. In Figure2, we illustrate shortest path computation error on different input graphs sizes at test time. While all methods are only trained with 5 steps of iterative computation on size 10 graphs, we find that IREM can generalize iterative computation to size 30 graphs. In contrast, the learned iterative baselines, such as the recurrent network in Figure2, fail to do so. We find similar behavior across other graphical algorithmic reasoning tasks.Ablations.We run an ablation analysis on the impact of utilizing a replay buffer to train IREM, as well as the effect of truncating backpropagation to the last step of optimization. In Table2, we find that utilizing a replay buffer significantly improves the performance of IREM. We further find that truncating backpropagation to the last step of optimization has a limited impact on IREM and greatly reduces the overall memory cost of training, as well as slightly improving the training speed of IREM.4.3Continuous Algorithmic ReasoningFigure 3:Computation Steps vs Problem Difficulty– Illustration of MSE error of prediction as a factor of the test time difficulty (harder difficulty right)  of the task and computation steps applied. Each model is trained with 5 steps of iterative computation. Models missing in plots have errors greater than the range displayed in the plot. The error of IREM improves with the number of underlying algorithmic computation steps, with larger number of computation steps benefiting performance on harder algorithmic tasks.Data Setup.We next evaluate IREM and baselines on continuous algorithmic reasoning tasks. We apply algorithmic operations on input vectors of size 400 (resized tomatrices for matrix operations) . We report the underlying MSE error between the predictions and the associated ground truth outputs on test problem instances. We evaluate different methods on the following three tasks, aiming to capture different aspects of reasoning, with additional details in the AppendixC.1.Addition: We first evaluate the algorithmic computation of addition. We train networks to add entries in two separate input vectors (element-wise) . We construct harder variants of the addition problems at test time by feeding input vectors with larger magnitudes. This task aims to test simple arithmetic reasoning.2.Matrix Completion: Next, we evaluate the algorithmic computation of matrix completion. We mask outof the entries of a low-rank input matrix constructed from two separate low-rank matricesand, and train networks to reconstruct the original input matrix. We construct harder variants of the matrix completion problem at test time by increasing the complexity ofand. This task aims to test both structural and analogical reasoning, with both shown to be equivalent to matrix completion.3.Matrix Inverse: Finally, we evaluate the algorithmic computation of matrix inverse. We train networks to compute the matrix inverse of an input matrix. We construct harder matrix inverse problems by considering less well-conditioned input matrices. This task aims to test the numerical reasoning, with matrix inversion a crucial operation across various numerical algorithms.Figure 4:Qualitative Illustration of Addition– Illustration of per element error on the addition task as a function of underlying number iterative computation steps run with IREM. Individual inputs gradually approach ground truth values, with different elements approaching zero error at different rates.Figure 5:Energy Landscape– Plot of predicted energy values forand the corresponding MSE distance offrom the problem solution. The predicted energy ofcorrelates well with the distance offrom the ground truth. Plot for matrix inverse is similar to matrix completion.Figure 6:Addition Composition– Illustration of predictions from IREM (in blue)  when three separate addition executions are nested together. IREM is applied on vectors with 400 entries – we visualize the first 9 elements of inputs and predictions.Quantitative Results.We present quantitative results on each of our three continuous algorithmic reasoning problems on the right side of Table1on test problems with either the same or harder difficulty than training problems. Across all three tasks, we find that IREM outperforms baselines, with the underlying difference magnified on harder, out-of-distribution test problems. In particular, on the task of addition, we find that IREM is able to nearly perfectly solve the underlying task even on harder, out-of-distribution test problems. In contrast, all other evaluated iterative and feedforward baselines perform poorly effectively solve the underlying problem. A primary difficulty is that underlying input vectors, both of size 400, are large compared to the underlying hidden unit size of 512, thus requiring the networks to iteratively reason and execute the algorithmic computation on subsets of the input.Step SizeSame DifficultyHarder Difficulty100.00030.0021300.00030.00201000.00030.00213000.00040.002310000.00040.0025Table 3:Ablation Analysis of Step Size in IREM–Analysis of training step size of IREM performance on the continuous addition reasoning task.Qualitative Visualization.Next, we visualize the underlying iterative computation learned by IREM and baselines. In Figure3, we illustrate the prediction error of different methods as a function of the number of computation steps applied. As we increase the number of iterative computation steps, the underlying performance of IREM continues to improve. In particular, iterative computation helps more substantially on harder variants of the algorithmic problem (right column) , such as addition. In contrast, several iterative methods show significant degradation of performance with increased iterative computation.We further qualitatively visualize the underlying iterative computation learned by IREM. In Figure4, we visualize the element-wise mean square error of the predicted solution as a function of the number of iterative reasoning steps applied on the addition task. We find that energy minimization gradually refines a predicted solution to the ground truth additive answer, with different elements of the solution exhibiting different convergence rates.Energy Landscape.IREM parameterizes an energy landscape across all possible solutions for a given problem. Such an energy landscape enables us to assess the relative quality of solutions dependent on their associated energies, and further gives a natural objective to terminate iterative computation when an underlying local energy minimum is reached. In Figure5, we visualize the predicted energy of different candidate solutions and their corresponding MSE distances from the ground truth answer. We find that across different continuous algorithmic tasks, the underlying energy value assigned to a candidate solution is well correlated with its distance from the ground truth answer, with low energy solutions close to the ground truth.Sensitivity to Step Size.We assess the performance of IREM under different values of step sizeusing during energy optimization at training. We consider the continuous addition task in Table3. We find that the underlying performance is not sensitive to hyperparameter choice for step size, and utilize a fixed step size of 100 across our experiments.4.4Recursive Algorithmic ComputationSetup.We further evaluate the ability of algorithms represented by IREM to be recursively applied on inputs. Recursively nesting algorithms enable complex computations, but require learned networks to be robust to out-of-distribution outputs from prior algorithmic execution.Composed OperationsMethod2510Feedforward0.04450.27170.8898Recurrent2.13773.18614.8706Programmatic0.02030.10680.4587Iterative Feedforward0.08260.59303.6004IREM (Ours) 0.00140.00780.0422Table 4:Algorithmic Composition –Test performance when composing multiple instances of the addition operation. Error is reported using element-wise mean square error. IREM is able to generalize well when composing algorithmic computation.Figure 7:Error Composing Algorithmic Computation– Plot of error of predictions as factor of the number of composed algorithmic operations. IREM exhibits lower error when nesting a series of algorithmic operations together.We consider recursive applications of a learned algorithmic operatorrepresenting addition as introduced in Section4.3. We evaluate the element-wise mean square error of the predicted outputof recursively applying the learned algorithmic operatortimes(7) with the corresponding ground truth solution being, for different valuesof recursive application.Quantitative Results.We report the results of recursively applying each learned algorithmic operator between two to ten times in Table4. We find that IREM supports the most stable recursive application of algorithmic operators. IREM exhibits significantly lower error than all compared baseline, due to its ability to utilize iterative computation to deal with out-of-distribution inputs and to accurately compute intermediate algorithmic outputs.Qualitative Results.We illustrate error as a factor of the number of applied algorithmic operations in Figure7, and find that the error of predictions from IREM rises slowly in comparison to other baselines. We further visualize the nested algorithmic predictions from IREM. We illustrate the inferred array sum predicted by IREM when four separate inputs are summed in Figure6. As seen above, our approach enables us to closely approximate the addition of four input matrices (with the first nine entries shown) .
body
Learning Iterative Reasoning through Energy Minimization
4Experiments
2206.15448
5Conclusion and LimitationsIn this paper, we present IREM, a new approach towards iterative computation, by formulating it as an energy minimization process. We illustrate, on both continuous and graphical domains, how iterative computation utilizing IREM enables better algorithmic performance, as well as generalization to more complex instances of problems. We further illustrate how the underlying algorithmic computation learned by IREM may be nested to implement more complex algorithmic computations.Iterative reasoning with IREM has several limitations. First, while IREM substantially outperforms existing iterative methods on tasks where output solutions have high dimensionality, limited gains are obtained when IREM is executed on problems with lower dimensionality solutions (such as parity prediction) . Second, as training and inferring solutions with IREM relies on continuous gradient optimization, IREM struggles when output solutions have discrete values. An interesting line of future work would be to explore how discrete optimization could be integrated with training IREM to solve such discrete valued problems. Finally, since the training procedure of IREM requires backpropagation across gradient optimization steps, it is computationally expensive. An interesting line of future work could be exploring alternative ways to train an energy function for reasoning, such as utilizing gradient-free optimization.
body
Learning Iterative Reasoning through Energy Minimization
5Conclusion and Limitations
2206.15448
6AcknowledgementsWe thank Ben Poole for feedback and helpful comments on a early version of the manuscript. Yilun Du is supported by a fellowship from the National Science Foundation.
conclusion
Learning Iterative Reasoning through Energy Minimization
6Acknowledgements