<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2209.09675] Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization</title><meta property="og:description" content="Fast Function Extraction (FFX) is a deterministic algorithm for solving symbolic regression problems. We improve the accuracy of FFX by adding parameters to the arguments of nonlinear functions. Instead of only optimiz…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2209.09675">

<!--Generated on Thu Mar 14 00:39:47 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="keywords" lang="en" content="Symbolic Regression Machine Learning.">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document ltx_authors_1line"><span id="id1" class="ltx_note ltx_role_institutetext"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_note_type">institutetext: </span>
Josef Ressel Center for Symbolic Regression
<br class="ltx_break">Heuristic and Evolutionary Algorithms Laboratory
<br class="ltx_break">University of Applied Sciences Upper Austria, Hagenberg, Austria
<br class="ltx_break">
</span></span></span><span id="id2" class="ltx_note ltx_role_institutetext"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_note_type">institutetext: </span>Department of Computer Science
<br class="ltx_break">Johannes Kepler University, Linz, Austria
<br class="ltx_break"><span id="id2.1" class="ltx_note ltx_role_email"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_note_type">email: </span>lukas.kammerer@fh-hagenberg.at</span></span></span>
</span></span></span>
<h1 class="ltx_title ltx_title_document">
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">
Lukas Kammerer
</span><span class="ltx_author_notes">1122
<span class="ltx_contact ltx_role_orcid"><a target="_blank" href="https://orcid.org/0000-0001-8236-4294" title="ORCID identifier" class="ltx_ref">0000-0001-8236-4294</a></span>
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Gabriel Kronberger
</span><span class="ltx_author_notes">11
<span class="ltx_contact ltx_role_orcid"><a target="_blank" href="https://orcid.org/0000-0002-3012-3189" title="ORCID identifier" class="ltx_ref">0000-0002-3012-3189</a></span>
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Michael Kommenda
</span><span class="ltx_author_notes">11
<span class="ltx_contact ltx_role_orcid"><a target="_blank" href="https://orcid.org/0000-0003-2049-723X" title="ORCID identifier" class="ltx_ref">0000-0003-2049-723X</a></span>
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id1.id1" class="ltx_p">Fast Function Extraction (FFX) is a deterministic algorithm for solving symbolic regression problems. We improve the accuracy of FFX by adding parameters to the arguments of nonlinear functions. Instead of only optimizing linear parameters, we optimize these additional nonlinear parameters with separable nonlinear least squared optimization using a variable projection algorithm. Both FFX and our new algorithm is applied on the PennML benchmark suite. We show that the proposed extensions of FFX leads to higher accuracy while providing models of similar length and with only a small increase in runtime on the given data. Our results are compared to a large set of regression methods that were already published for the given benchmark suite.</p>
</div>
<div class="ltx_keywords">
<h6 class="ltx_title ltx_title_keywords">Keywords: </h6>Symbolic Regression Machine Learning.
</div>
<span id="footnotex1" class="ltx_note ltx_role_footnotetext"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotetext: </span>
Submitted manuscript to be published in <span id="footnotex1.1" class="ltx_text ltx_font_italic">Computer Aided Systems Theory - EUROCAST 2022: 18th International Conference, Las Palmas de Gran Canaria, Feb. 2022</span>.
</span></span></span>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Symbolic Regression and FFX</h2>

<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Symbolic regression is a machine learning task in which we try to identify mathematical formulas that cover linear and nonlinear relations within given data. The most common algorithm for solving symbolic regression is genetic programming (GP), which optimizes a population of mathematical models using crossover and mutation. GP is in theory capable of finding models of any syntactical structure and complexity. However, disadvantages of GP are its stochasticity, its long algorithm runtime for nontrivial problems and its complex hyperparameter settings. These characteristics led to the development of non-evolutionary algorithms which produce more restricted models but have advantages in determinism, runtime or complexity of hyperparameters <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib3" title="" class="ltx_ref">3</a>, <a href="#bib.bib6" title="" class="ltx_ref">6</a>, <a href="#bib.bib9" title="" class="ltx_ref">9</a>]</cite>.</p>
</div>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.10" class="ltx_p">One of the first algorithms that was developed to tackle the shortages of GP is Fast Function Extraction (FFX) <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib9" title="" class="ltx_ref">9</a>]</cite>. FFX generates a large set of <span id="S1.p2.10.1" class="ltx_text ltx_font_italic">base functions</span> first, then it learns a regularized linear model using these base functions as terms. The set of base functions is a combination of the original features with several nonlinear functions such as <math id="S1.p2.1.m1.2" class="ltx_Math" alttext="\exp(\dots)" display="inline"><semantics id="S1.p2.1.m1.2a"><mrow id="S1.p2.1.m1.2.3.2" xref="S1.p2.1.m1.2.3.1.cmml"><mi id="S1.p2.1.m1.1.1" xref="S1.p2.1.m1.1.1.cmml">exp</mi><mo id="S1.p2.1.m1.2.3.2a" xref="S1.p2.1.m1.2.3.1.cmml">⁡</mo><mrow id="S1.p2.1.m1.2.3.2.1" xref="S1.p2.1.m1.2.3.1.cmml"><mo stretchy="false" id="S1.p2.1.m1.2.3.2.1.1" xref="S1.p2.1.m1.2.3.1.cmml">(</mo><mi mathvariant="normal" id="S1.p2.1.m1.2.2" xref="S1.p2.1.m1.2.2.cmml">…</mi><mo stretchy="false" id="S1.p2.1.m1.2.3.2.1.2" xref="S1.p2.1.m1.2.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.p2.1.m1.2b"><apply id="S1.p2.1.m1.2.3.1.cmml" xref="S1.p2.1.m1.2.3.2"><exp id="S1.p2.1.m1.1.1.cmml" xref="S1.p2.1.m1.1.1"></exp><ci id="S1.p2.1.m1.2.2.cmml" xref="S1.p2.1.m1.2.2">…</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.1.m1.2c">\exp(\dots)</annotation></semantics></math> or <math id="S1.p2.2.m2.2" class="ltx_Math" alttext="\log(\dots)" display="inline"><semantics id="S1.p2.2.m2.2a"><mrow id="S1.p2.2.m2.2.3.2" xref="S1.p2.2.m2.2.3.1.cmml"><mi id="S1.p2.2.m2.1.1" xref="S1.p2.2.m2.1.1.cmml">log</mi><mo id="S1.p2.2.m2.2.3.2a" xref="S1.p2.2.m2.2.3.1.cmml">⁡</mo><mrow id="S1.p2.2.m2.2.3.2.1" xref="S1.p2.2.m2.2.3.1.cmml"><mo stretchy="false" id="S1.p2.2.m2.2.3.2.1.1" xref="S1.p2.2.m2.2.3.1.cmml">(</mo><mi mathvariant="normal" id="S1.p2.2.m2.2.2" xref="S1.p2.2.m2.2.2.cmml">…</mi><mo stretchy="false" id="S1.p2.2.m2.2.3.2.1.2" xref="S1.p2.2.m2.2.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.p2.2.m2.2b"><apply id="S1.p2.2.m2.2.3.1.cmml" xref="S1.p2.2.m2.2.3.2"><log id="S1.p2.2.m2.1.1.cmml" xref="S1.p2.2.m2.1.1"></log><ci id="S1.p2.2.m2.2.2.cmml" xref="S1.p2.2.m2.2.2">…</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.2.m2.2c">\log(\dots)</annotation></semantics></math> and a predefined set of real-valued exponent values. Examples of base functions for problems with features <math id="S1.p2.3.m3.2" class="ltx_Math" alttext="\{x_{1},x_{2}\}" display="inline"><semantics id="S1.p2.3.m3.2a"><mrow id="S1.p2.3.m3.2.2.2" xref="S1.p2.3.m3.2.2.3.cmml"><mo stretchy="false" id="S1.p2.3.m3.2.2.2.3" xref="S1.p2.3.m3.2.2.3.cmml">{</mo><msub id="S1.p2.3.m3.1.1.1.1" xref="S1.p2.3.m3.1.1.1.1.cmml"><mi id="S1.p2.3.m3.1.1.1.1.2" xref="S1.p2.3.m3.1.1.1.1.2.cmml">x</mi><mn id="S1.p2.3.m3.1.1.1.1.3" xref="S1.p2.3.m3.1.1.1.1.3.cmml">1</mn></msub><mo id="S1.p2.3.m3.2.2.2.4" xref="S1.p2.3.m3.2.2.3.cmml">,</mo><msub id="S1.p2.3.m3.2.2.2.2" xref="S1.p2.3.m3.2.2.2.2.cmml"><mi id="S1.p2.3.m3.2.2.2.2.2" xref="S1.p2.3.m3.2.2.2.2.2.cmml">x</mi><mn id="S1.p2.3.m3.2.2.2.2.3" xref="S1.p2.3.m3.2.2.2.2.3.cmml">2</mn></msub><mo stretchy="false" id="S1.p2.3.m3.2.2.2.5" xref="S1.p2.3.m3.2.2.3.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.p2.3.m3.2b"><set id="S1.p2.3.m3.2.2.3.cmml" xref="S1.p2.3.m3.2.2.2"><apply id="S1.p2.3.m3.1.1.1.1.cmml" xref="S1.p2.3.m3.1.1.1.1"><csymbol cd="ambiguous" id="S1.p2.3.m3.1.1.1.1.1.cmml" xref="S1.p2.3.m3.1.1.1.1">subscript</csymbol><ci id="S1.p2.3.m3.1.1.1.1.2.cmml" xref="S1.p2.3.m3.1.1.1.1.2">𝑥</ci><cn type="integer" id="S1.p2.3.m3.1.1.1.1.3.cmml" xref="S1.p2.3.m3.1.1.1.1.3">1</cn></apply><apply id="S1.p2.3.m3.2.2.2.2.cmml" xref="S1.p2.3.m3.2.2.2.2"><csymbol cd="ambiguous" id="S1.p2.3.m3.2.2.2.2.1.cmml" xref="S1.p2.3.m3.2.2.2.2">subscript</csymbol><ci id="S1.p2.3.m3.2.2.2.2.2.cmml" xref="S1.p2.3.m3.2.2.2.2.2">𝑥</ci><cn type="integer" id="S1.p2.3.m3.2.2.2.2.3.cmml" xref="S1.p2.3.m3.2.2.2.2.3">2</cn></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.3.m3.2c">\{x_{1},x_{2}\}</annotation></semantics></math> are <math id="S1.p2.4.m4.1" class="ltx_Math" alttext="x_{1}" display="inline"><semantics id="S1.p2.4.m4.1a"><msub id="S1.p2.4.m4.1.1" xref="S1.p2.4.m4.1.1.cmml"><mi id="S1.p2.4.m4.1.1.2" xref="S1.p2.4.m4.1.1.2.cmml">x</mi><mn id="S1.p2.4.m4.1.1.3" xref="S1.p2.4.m4.1.1.3.cmml">1</mn></msub><annotation-xml encoding="MathML-Content" id="S1.p2.4.m4.1b"><apply id="S1.p2.4.m4.1.1.cmml" xref="S1.p2.4.m4.1.1"><csymbol cd="ambiguous" id="S1.p2.4.m4.1.1.1.cmml" xref="S1.p2.4.m4.1.1">subscript</csymbol><ci id="S1.p2.4.m4.1.1.2.cmml" xref="S1.p2.4.m4.1.1.2">𝑥</ci><cn type="integer" id="S1.p2.4.m4.1.1.3.cmml" xref="S1.p2.4.m4.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.4.m4.1c">x_{1}</annotation></semantics></math>, <math id="S1.p2.5.m5.1" class="ltx_Math" alttext="x_{2}" display="inline"><semantics id="S1.p2.5.m5.1a"><msub id="S1.p2.5.m5.1.1" xref="S1.p2.5.m5.1.1.cmml"><mi id="S1.p2.5.m5.1.1.2" xref="S1.p2.5.m5.1.1.2.cmml">x</mi><mn id="S1.p2.5.m5.1.1.3" xref="S1.p2.5.m5.1.1.3.cmml">2</mn></msub><annotation-xml encoding="MathML-Content" id="S1.p2.5.m5.1b"><apply id="S1.p2.5.m5.1.1.cmml" xref="S1.p2.5.m5.1.1"><csymbol cd="ambiguous" id="S1.p2.5.m5.1.1.1.cmml" xref="S1.p2.5.m5.1.1">subscript</csymbol><ci id="S1.p2.5.m5.1.1.2.cmml" xref="S1.p2.5.m5.1.1.2">𝑥</ci><cn type="integer" id="S1.p2.5.m5.1.1.3.cmml" xref="S1.p2.5.m5.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.5.m5.1c">x_{2}</annotation></semantics></math>, <math id="S1.p2.6.m6.1" class="ltx_math_unparsed" alttext="x_{1}{}^{2}" display="inline"><semantics id="S1.p2.6.m6.1a"><mrow id="S1.p2.6.m6.1b"><msub id="S1.p2.6.m6.1.1"><mi id="S1.p2.6.m6.1.1.2">x</mi><mn id="S1.p2.6.m6.1.1.3">1</mn></msub><msup id="S1.p2.6.m6.1.2"><mi id="S1.p2.6.m6.1.2a"></mi><mn id="S1.p2.6.m6.1.2.1">2</mn></msup></mrow><annotation encoding="application/x-tex" id="S1.p2.6.m6.1c">x_{1}{}^{2}</annotation></semantics></math>, <math id="S1.p2.7.m7.2" class="ltx_Math" alttext="\exp(x_{1})" display="inline"><semantics id="S1.p2.7.m7.2a"><mrow id="S1.p2.7.m7.2.2.1" xref="S1.p2.7.m7.2.2.2.cmml"><mi id="S1.p2.7.m7.1.1" xref="S1.p2.7.m7.1.1.cmml">exp</mi><mo id="S1.p2.7.m7.2.2.1a" xref="S1.p2.7.m7.2.2.2.cmml">⁡</mo><mrow id="S1.p2.7.m7.2.2.1.1" xref="S1.p2.7.m7.2.2.2.cmml"><mo stretchy="false" id="S1.p2.7.m7.2.2.1.1.2" xref="S1.p2.7.m7.2.2.2.cmml">(</mo><msub id="S1.p2.7.m7.2.2.1.1.1" xref="S1.p2.7.m7.2.2.1.1.1.cmml"><mi id="S1.p2.7.m7.2.2.1.1.1.2" xref="S1.p2.7.m7.2.2.1.1.1.2.cmml">x</mi><mn id="S1.p2.7.m7.2.2.1.1.1.3" xref="S1.p2.7.m7.2.2.1.1.1.3.cmml">1</mn></msub><mo stretchy="false" id="S1.p2.7.m7.2.2.1.1.3" xref="S1.p2.7.m7.2.2.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.p2.7.m7.2b"><apply id="S1.p2.7.m7.2.2.2.cmml" xref="S1.p2.7.m7.2.2.1"><exp id="S1.p2.7.m7.1.1.cmml" xref="S1.p2.7.m7.1.1"></exp><apply id="S1.p2.7.m7.2.2.1.1.1.cmml" xref="S1.p2.7.m7.2.2.1.1.1"><csymbol cd="ambiguous" id="S1.p2.7.m7.2.2.1.1.1.1.cmml" xref="S1.p2.7.m7.2.2.1.1.1">subscript</csymbol><ci id="S1.p2.7.m7.2.2.1.1.1.2.cmml" xref="S1.p2.7.m7.2.2.1.1.1.2">𝑥</ci><cn type="integer" id="S1.p2.7.m7.2.2.1.1.1.3.cmml" xref="S1.p2.7.m7.2.2.1.1.1.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.7.m7.2c">\exp(x_{1})</annotation></semantics></math>, <math id="S1.p2.8.m8.1" class="ltx_math_unparsed" alttext="\exp(x_{1}{}^{2})" display="inline"><semantics id="S1.p2.8.m8.1a"><mrow id="S1.p2.8.m8.1b"><mi id="S1.p2.8.m8.1.1">exp</mi><mrow id="S1.p2.8.m8.1.2"><mo stretchy="false" id="S1.p2.8.m8.1.2.1">(</mo><msub id="S1.p2.8.m8.1.2.2"><mi id="S1.p2.8.m8.1.2.2.2">x</mi><mn id="S1.p2.8.m8.1.2.2.3">1</mn></msub><mmultiscripts id="S1.p2.8.m8.1.2.3"><mo stretchy="false" id="S1.p2.8.m8.1.2.3.2">)</mo><mprescripts id="S1.p2.8.m8.1.2.3a"></mprescripts><mrow id="S1.p2.8.m8.1.2.3b"></mrow><mn id="S1.p2.8.m8.1.2.3.3">2</mn></mmultiscripts></mrow></mrow><annotation encoding="application/x-tex" id="S1.p2.8.m8.1c">\exp(x_{1}{}^{2})</annotation></semantics></math>, <math id="S1.p2.9.m9.2" class="ltx_Math" alttext="\log(x_{1})" display="inline"><semantics id="S1.p2.9.m9.2a"><mrow id="S1.p2.9.m9.2.2.1" xref="S1.p2.9.m9.2.2.2.cmml"><mi id="S1.p2.9.m9.1.1" xref="S1.p2.9.m9.1.1.cmml">log</mi><mo id="S1.p2.9.m9.2.2.1a" xref="S1.p2.9.m9.2.2.2.cmml">⁡</mo><mrow id="S1.p2.9.m9.2.2.1.1" xref="S1.p2.9.m9.2.2.2.cmml"><mo stretchy="false" id="S1.p2.9.m9.2.2.1.1.2" xref="S1.p2.9.m9.2.2.2.cmml">(</mo><msub id="S1.p2.9.m9.2.2.1.1.1" xref="S1.p2.9.m9.2.2.1.1.1.cmml"><mi id="S1.p2.9.m9.2.2.1.1.1.2" xref="S1.p2.9.m9.2.2.1.1.1.2.cmml">x</mi><mn id="S1.p2.9.m9.2.2.1.1.1.3" xref="S1.p2.9.m9.2.2.1.1.1.3.cmml">1</mn></msub><mo stretchy="false" id="S1.p2.9.m9.2.2.1.1.3" xref="S1.p2.9.m9.2.2.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.p2.9.m9.2b"><apply id="S1.p2.9.m9.2.2.2.cmml" xref="S1.p2.9.m9.2.2.1"><log id="S1.p2.9.m9.1.1.cmml" xref="S1.p2.9.m9.1.1"></log><apply id="S1.p2.9.m9.2.2.1.1.1.cmml" xref="S1.p2.9.m9.2.2.1.1.1"><csymbol cd="ambiguous" id="S1.p2.9.m9.2.2.1.1.1.1.cmml" xref="S1.p2.9.m9.2.2.1.1.1">subscript</csymbol><ci id="S1.p2.9.m9.2.2.1.1.1.2.cmml" xref="S1.p2.9.m9.2.2.1.1.1.2">𝑥</ci><cn type="integer" id="S1.p2.9.m9.2.2.1.1.1.3.cmml" xref="S1.p2.9.m9.2.2.1.1.1.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p2.9.m9.2c">\log(x_{1})</annotation></semantics></math> or <math id="S1.p2.10.m10.1" class="ltx_math_unparsed" alttext="\log(x_{2}{}^{0.5})" display="inline"><semantics id="S1.p2.10.m10.1a"><mrow id="S1.p2.10.m10.1b"><mi id="S1.p2.10.m10.1.1">log</mi><mrow id="S1.p2.10.m10.1.2"><mo stretchy="false" id="S1.p2.10.m10.1.2.1">(</mo><msub id="S1.p2.10.m10.1.2.2"><mi id="S1.p2.10.m10.1.2.2.2">x</mi><mn id="S1.p2.10.m10.1.2.2.3">2</mn></msub><mmultiscripts id="S1.p2.10.m10.1.2.3"><mo stretchy="false" id="S1.p2.10.m10.1.2.3.2">)</mo><mprescripts id="S1.p2.10.m10.1.2.3a"></mprescripts><mrow id="S1.p2.10.m10.1.2.3b"></mrow><mn id="S1.p2.10.m10.1.2.3.3">0.5</mn></mmultiscripts></mrow></mrow><annotation encoding="application/x-tex" id="S1.p2.10.m10.1c">\log(x_{2}{}^{0.5})</annotation></semantics></math>.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.6" class="ltx_p">The structure of FFX models with <math id="S1.p3.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S1.p3.1.m1.1a"><mi id="S1.p3.1.m1.1.1" xref="S1.p3.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S1.p3.1.m1.1b"><ci id="S1.p3.1.m1.1.1.cmml" xref="S1.p3.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.1.m1.1c">n</annotation></semantics></math> features <math id="S1.p3.2.m2.1" class="ltx_Math" alttext="\{x_{1}...x_{n}\}" display="inline"><semantics id="S1.p3.2.m2.1a"><mrow id="S1.p3.2.m2.1.1.1" xref="S1.p3.2.m2.1.1.2.cmml"><mo stretchy="false" id="S1.p3.2.m2.1.1.1.2" xref="S1.p3.2.m2.1.1.2.cmml">{</mo><mrow id="S1.p3.2.m2.1.1.1.1" xref="S1.p3.2.m2.1.1.1.1.cmml"><msub id="S1.p3.2.m2.1.1.1.1.2" xref="S1.p3.2.m2.1.1.1.1.2.cmml"><mi id="S1.p3.2.m2.1.1.1.1.2.2" xref="S1.p3.2.m2.1.1.1.1.2.2.cmml">x</mi><mn id="S1.p3.2.m2.1.1.1.1.2.3" xref="S1.p3.2.m2.1.1.1.1.2.3.cmml">1</mn></msub><mo lspace="0em" rspace="0em" id="S1.p3.2.m2.1.1.1.1.1" xref="S1.p3.2.m2.1.1.1.1.1.cmml">​</mo><mi mathvariant="normal" id="S1.p3.2.m2.1.1.1.1.3" xref="S1.p3.2.m2.1.1.1.1.3.cmml">…</mi><mo lspace="0em" rspace="0em" id="S1.p3.2.m2.1.1.1.1.1a" xref="S1.p3.2.m2.1.1.1.1.1.cmml">​</mo><msub id="S1.p3.2.m2.1.1.1.1.4" xref="S1.p3.2.m2.1.1.1.1.4.cmml"><mi id="S1.p3.2.m2.1.1.1.1.4.2" xref="S1.p3.2.m2.1.1.1.1.4.2.cmml">x</mi><mi id="S1.p3.2.m2.1.1.1.1.4.3" xref="S1.p3.2.m2.1.1.1.1.4.3.cmml">n</mi></msub></mrow><mo stretchy="false" id="S1.p3.2.m2.1.1.1.3" xref="S1.p3.2.m2.1.1.2.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S1.p3.2.m2.1b"><set id="S1.p3.2.m2.1.1.2.cmml" xref="S1.p3.2.m2.1.1.1"><apply id="S1.p3.2.m2.1.1.1.1.cmml" xref="S1.p3.2.m2.1.1.1.1"><times id="S1.p3.2.m2.1.1.1.1.1.cmml" xref="S1.p3.2.m2.1.1.1.1.1"></times><apply id="S1.p3.2.m2.1.1.1.1.2.cmml" xref="S1.p3.2.m2.1.1.1.1.2"><csymbol cd="ambiguous" id="S1.p3.2.m2.1.1.1.1.2.1.cmml" xref="S1.p3.2.m2.1.1.1.1.2">subscript</csymbol><ci id="S1.p3.2.m2.1.1.1.1.2.2.cmml" xref="S1.p3.2.m2.1.1.1.1.2.2">𝑥</ci><cn type="integer" id="S1.p3.2.m2.1.1.1.1.2.3.cmml" xref="S1.p3.2.m2.1.1.1.1.2.3">1</cn></apply><ci id="S1.p3.2.m2.1.1.1.1.3.cmml" xref="S1.p3.2.m2.1.1.1.1.3">…</ci><apply id="S1.p3.2.m2.1.1.1.1.4.cmml" xref="S1.p3.2.m2.1.1.1.1.4"><csymbol cd="ambiguous" id="S1.p3.2.m2.1.1.1.1.4.1.cmml" xref="S1.p3.2.m2.1.1.1.1.4">subscript</csymbol><ci id="S1.p3.2.m2.1.1.1.1.4.2.cmml" xref="S1.p3.2.m2.1.1.1.1.4.2">𝑥</ci><ci id="S1.p3.2.m2.1.1.1.1.4.3.cmml" xref="S1.p3.2.m2.1.1.1.1.4.3">𝑛</ci></apply></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.2.m2.1c">\{x_{1}...x_{n}\}</annotation></semantics></math> is outlined in Equation <a href="#S1.E1" title="In 1 Symbolic Regression and FFX ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>. Parameters <math id="S1.p3.3.m3.1" class="ltx_math_unparsed" alttext="\{c_{0},c_{1}," display="inline"><semantics id="S1.p3.3.m3.1a"><mrow id="S1.p3.3.m3.1b"><mo stretchy="false" id="S1.p3.3.m3.1.1">{</mo><msub id="S1.p3.3.m3.1.2"><mi id="S1.p3.3.m3.1.2.2">c</mi><mn id="S1.p3.3.m3.1.2.3">0</mn></msub><mo id="S1.p3.3.m3.1.3">,</mo><msub id="S1.p3.3.m3.1.4"><mi id="S1.p3.3.m3.1.4.2">c</mi><mn id="S1.p3.3.m3.1.4.3">1</mn></msub><mo id="S1.p3.3.m3.1.5">,</mo></mrow><annotation encoding="application/x-tex" id="S1.p3.3.m3.1c">\{c_{0},c_{1},</annotation></semantics></math> <math id="S1.p3.4.m4.1" class="ltx_math_unparsed" alttext="\dots,c_{m}\}" display="inline"><semantics id="S1.p3.4.m4.1a"><mrow id="S1.p3.4.m4.1b"><mi mathvariant="normal" id="S1.p3.4.m4.1.1">…</mi><mo id="S1.p3.4.m4.1.2">,</mo><msub id="S1.p3.4.m4.1.3"><mi id="S1.p3.4.m4.1.3.2">c</mi><mi id="S1.p3.4.m4.1.3.3">m</mi></msub><mo stretchy="false" id="S1.p3.4.m4.1.4">}</mo></mrow><annotation encoding="application/x-tex" id="S1.p3.4.m4.1c">\dots,c_{m}\}</annotation></semantics></math> are linear parameters of a model with <math id="S1.p3.5.m5.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S1.p3.5.m5.1a"><mi id="S1.p3.5.m5.1.1" xref="S1.p3.5.m5.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S1.p3.5.m5.1b"><ci id="S1.p3.5.m5.1.1.cmml" xref="S1.p3.5.m5.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.5.m5.1c">m</annotation></semantics></math> base functions. They are learned with ElasticNet regression <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib4" title="" class="ltx_ref">4</a>]</cite>. ElasticNet regression identifies only the most relevant base functions due to its regularization by setting linear parameters <math id="S1.p3.6.m6.1" class="ltx_Math" alttext="c_{i}" display="inline"><semantics id="S1.p3.6.m6.1a"><msub id="S1.p3.6.m6.1.1" xref="S1.p3.6.m6.1.1.cmml"><mi id="S1.p3.6.m6.1.1.2" xref="S1.p3.6.m6.1.1.2.cmml">c</mi><mi id="S1.p3.6.m6.1.1.3" xref="S1.p3.6.m6.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S1.p3.6.m6.1b"><apply id="S1.p3.6.m6.1.1.cmml" xref="S1.p3.6.m6.1.1"><csymbol cd="ambiguous" id="S1.p3.6.m6.1.1.1.cmml" xref="S1.p3.6.m6.1.1">subscript</csymbol><ci id="S1.p3.6.m6.1.1.2.cmml" xref="S1.p3.6.m6.1.1.2">𝑐</ci><ci id="S1.p3.6.m6.1.1.3.cmml" xref="S1.p3.6.m6.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.6.m6.1c">c_{i}</annotation></semantics></math> of irrelevant base functions to zero (sparsification). Therefore, learned models contain only a subset of the original base functions.</p>
</div>
<div id="S1.p4" class="ltx_para">
<table id="S1.E1" class="ltx_equationgroup ltx_eqn_table">
<tbody>
<tr id="S1.E1X" class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S1.E1X.2.1.1.m1.1" class="ltx_Math" alttext="\displaystyle\hat{f}(\mathbf{x})=c_{0}+c_{1}" display="inline"><semantics id="S1.E1X.2.1.1.m1.1a"><mrow id="S1.E1X.2.1.1.m1.1.2" xref="S1.E1X.2.1.1.m1.1.2.cmml"><mrow id="S1.E1X.2.1.1.m1.1.2.2" xref="S1.E1X.2.1.1.m1.1.2.2.cmml"><mover accent="true" id="S1.E1X.2.1.1.m1.1.2.2.2" xref="S1.E1X.2.1.1.m1.1.2.2.2.cmml"><mi id="S1.E1X.2.1.1.m1.1.2.2.2.2" xref="S1.E1X.2.1.1.m1.1.2.2.2.2.cmml">f</mi><mo id="S1.E1X.2.1.1.m1.1.2.2.2.1" xref="S1.E1X.2.1.1.m1.1.2.2.2.1.cmml">^</mo></mover><mo lspace="0em" rspace="0em" id="S1.E1X.2.1.1.m1.1.2.2.1" xref="S1.E1X.2.1.1.m1.1.2.2.1.cmml">​</mo><mrow id="S1.E1X.2.1.1.m1.1.2.2.3.2" xref="S1.E1X.2.1.1.m1.1.2.2.cmml"><mo stretchy="false" id="S1.E1X.2.1.1.m1.1.2.2.3.2.1" xref="S1.E1X.2.1.1.m1.1.2.2.cmml">(</mo><mi id="S1.E1X.2.1.1.m1.1.1" xref="S1.E1X.2.1.1.m1.1.1.cmml">𝐱</mi><mo stretchy="false" id="S1.E1X.2.1.1.m1.1.2.2.3.2.2" xref="S1.E1X.2.1.1.m1.1.2.2.cmml">)</mo></mrow></mrow><mo id="S1.E1X.2.1.1.m1.1.2.1" xref="S1.E1X.2.1.1.m1.1.2.1.cmml">=</mo><mrow id="S1.E1X.2.1.1.m1.1.2.3" xref="S1.E1X.2.1.1.m1.1.2.3.cmml"><msub id="S1.E1X.2.1.1.m1.1.2.3.2" xref="S1.E1X.2.1.1.m1.1.2.3.2.cmml"><mi id="S1.E1X.2.1.1.m1.1.2.3.2.2" xref="S1.E1X.2.1.1.m1.1.2.3.2.2.cmml">c</mi><mn id="S1.E1X.2.1.1.m1.1.2.3.2.3" xref="S1.E1X.2.1.1.m1.1.2.3.2.3.cmml">0</mn></msub><mo id="S1.E1X.2.1.1.m1.1.2.3.1" xref="S1.E1X.2.1.1.m1.1.2.3.1.cmml">+</mo><msub id="S1.E1X.2.1.1.m1.1.2.3.3" xref="S1.E1X.2.1.1.m1.1.2.3.3.cmml"><mi id="S1.E1X.2.1.1.m1.1.2.3.3.2" xref="S1.E1X.2.1.1.m1.1.2.3.3.2.cmml">c</mi><mn id="S1.E1X.2.1.1.m1.1.2.3.3.3" xref="S1.E1X.2.1.1.m1.1.2.3.3.3.cmml">1</mn></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.E1X.2.1.1.m1.1b"><apply id="S1.E1X.2.1.1.m1.1.2.cmml" xref="S1.E1X.2.1.1.m1.1.2"><eq id="S1.E1X.2.1.1.m1.1.2.1.cmml" xref="S1.E1X.2.1.1.m1.1.2.1"></eq><apply id="S1.E1X.2.1.1.m1.1.2.2.cmml" xref="S1.E1X.2.1.1.m1.1.2.2"><times id="S1.E1X.2.1.1.m1.1.2.2.1.cmml" xref="S1.E1X.2.1.1.m1.1.2.2.1"></times><apply id="S1.E1X.2.1.1.m1.1.2.2.2.cmml" xref="S1.E1X.2.1.1.m1.1.2.2.2"><ci id="S1.E1X.2.1.1.m1.1.2.2.2.1.cmml" xref="S1.E1X.2.1.1.m1.1.2.2.2.1">^</ci><ci id="S1.E1X.2.1.1.m1.1.2.2.2.2.cmml" xref="S1.E1X.2.1.1.m1.1.2.2.2.2">𝑓</ci></apply><ci id="S1.E1X.2.1.1.m1.1.1.cmml" xref="S1.E1X.2.1.1.m1.1.1">𝐱</ci></apply><apply id="S1.E1X.2.1.1.m1.1.2.3.cmml" xref="S1.E1X.2.1.1.m1.1.2.3"><plus id="S1.E1X.2.1.1.m1.1.2.3.1.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.1"></plus><apply id="S1.E1X.2.1.1.m1.1.2.3.2.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.2"><csymbol cd="ambiguous" id="S1.E1X.2.1.1.m1.1.2.3.2.1.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.2">subscript</csymbol><ci id="S1.E1X.2.1.1.m1.1.2.3.2.2.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.2.2">𝑐</ci><cn type="integer" id="S1.E1X.2.1.1.m1.1.2.3.2.3.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.2.3">0</cn></apply><apply id="S1.E1X.2.1.1.m1.1.2.3.3.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.3"><csymbol cd="ambiguous" id="S1.E1X.2.1.1.m1.1.2.3.3.1.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.3">subscript</csymbol><ci id="S1.E1X.2.1.1.m1.1.2.3.3.2.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.3.2">𝑐</ci><cn type="integer" id="S1.E1X.2.1.1.m1.1.2.3.3.3.cmml" xref="S1.E1X.2.1.1.m1.1.2.3.3.3">1</cn></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.E1X.2.1.1.m1.1c">\displaystyle\hat{f}(\mathbf{x})=c_{0}+c_{1}</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="S1.E1X.3.2.2.m1.1" class="ltx_math_unparsed" alttext="\displaystyle\ \text{func}_{1}(x_{1}{}^{e_{1}})+\dots+c_{m}\ \text{func}_{m}(x_{n}{}^{e_{m}})" display="inline"><semantics id="S1.E1X.3.2.2.m1.1a"><mrow id="S1.E1X.3.2.2.m1.1b"><msub id="S1.E1X.3.2.2.m1.1.1"><mtext id="S1.E1X.3.2.2.m1.1.1.2">func</mtext><mn id="S1.E1X.3.2.2.m1.1.1.3">1</mn></msub><mrow id="S1.E1X.3.2.2.m1.1.2"><mo stretchy="false" id="S1.E1X.3.2.2.m1.1.2.1">(</mo><msub id="S1.E1X.3.2.2.m1.1.2.2"><mi id="S1.E1X.3.2.2.m1.1.2.2.2">x</mi><mn id="S1.E1X.3.2.2.m1.1.2.2.3">1</mn></msub><mmultiscripts id="S1.E1X.3.2.2.m1.1.2.3"><mo stretchy="false" id="S1.E1X.3.2.2.m1.1.2.3.2">)</mo><mprescripts id="S1.E1X.3.2.2.m1.1.2.3a"></mprescripts><mrow id="S1.E1X.3.2.2.m1.1.2.3b"></mrow><msub id="S1.E1X.3.2.2.m1.1.2.3.3"><mi id="S1.E1X.3.2.2.m1.1.2.3.3.2">e</mi><mn id="S1.E1X.3.2.2.m1.1.2.3.3.3">1</mn></msub></mmultiscripts></mrow><mo id="S1.E1X.3.2.2.m1.1.3">+</mo><mi mathvariant="normal" id="S1.E1X.3.2.2.m1.1.4">⋯</mi><mo id="S1.E1X.3.2.2.m1.1.5">+</mo><msub id="S1.E1X.3.2.2.m1.1.6"><mi id="S1.E1X.3.2.2.m1.1.6.2">c</mi><mi id="S1.E1X.3.2.2.m1.1.6.3">m</mi></msub><msub id="S1.E1X.3.2.2.m1.1.7"><mtext id="S1.E1X.3.2.2.m1.1.7.2">func</mtext><mi id="S1.E1X.3.2.2.m1.1.7.3">m</mi></msub><mrow id="S1.E1X.3.2.2.m1.1.8"><mo stretchy="false" id="S1.E1X.3.2.2.m1.1.8.1">(</mo><msub id="S1.E1X.3.2.2.m1.1.8.2"><mi id="S1.E1X.3.2.2.m1.1.8.2.2">x</mi><mi id="S1.E1X.3.2.2.m1.1.8.2.3">n</mi></msub><mmultiscripts id="S1.E1X.3.2.2.m1.1.8.3"><mo stretchy="false" id="S1.E1X.3.2.2.m1.1.8.3.2">)</mo><mprescripts id="S1.E1X.3.2.2.m1.1.8.3a"></mprescripts><mrow id="S1.E1X.3.2.2.m1.1.8.3b"></mrow><msub id="S1.E1X.3.2.2.m1.1.8.3.3"><mi id="S1.E1X.3.2.2.m1.1.8.3.3.2">e</mi><mi id="S1.E1X.3.2.2.m1.1.8.3.3.3">m</mi></msub></mmultiscripts></mrow></mrow><annotation encoding="application/x-tex" id="S1.E1X.3.2.2.m1.1c">\displaystyle\ \text{func}_{1}(x_{1}{}^{e_{1}})+\dots+c_{m}\ \text{func}_{m}(x_{n}{}^{e_{m}})</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="2" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equationgroup ltx_align_right">(1)</span></td>
</tr>
<tr id="S1.E1Xa" class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S1.E1Xa.2.1.1.m1.3" class="ltx_Math" alttext="\displaystyle\text{with}\ \text{func}_{1},\dots,\text{func}_{m}\in" display="inline"><semantics id="S1.E1Xa.2.1.1.m1.3a"><mrow id="S1.E1Xa.2.1.1.m1.3.3" xref="S1.E1Xa.2.1.1.m1.3.3.cmml"><mrow id="S1.E1Xa.2.1.1.m1.3.3.2.2" xref="S1.E1Xa.2.1.1.m1.3.3.2.3.cmml"><mrow id="S1.E1Xa.2.1.1.m1.2.2.1.1.1" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.cmml"><mtext id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.2" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.2a.cmml">with</mtext><mo lspace="0.500em" rspace="0em" id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.1" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.1.cmml">​</mo><msub id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.cmml"><mtext id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.2" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.2a.cmml">func</mtext><mn id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.3" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.3.cmml">1</mn></msub></mrow><mo id="S1.E1Xa.2.1.1.m1.3.3.2.2.3" xref="S1.E1Xa.2.1.1.m1.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="S1.E1Xa.2.1.1.m1.1.1" xref="S1.E1Xa.2.1.1.m1.1.1.cmml">…</mi><mo id="S1.E1Xa.2.1.1.m1.3.3.2.2.4" xref="S1.E1Xa.2.1.1.m1.3.3.2.3.cmml">,</mo><msub id="S1.E1Xa.2.1.1.m1.3.3.2.2.2" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2.cmml"><mtext id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.2" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2.2a.cmml">func</mtext><mi id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.3" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2.3.cmml">m</mi></msub></mrow><mo id="S1.E1Xa.2.1.1.m1.3.3.3" xref="S1.E1Xa.2.1.1.m1.3.3.3.cmml">∈</mo><mi id="S1.E1Xa.2.1.1.m1.3.3.4" xref="S1.E1Xa.2.1.1.m1.3.3.4.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="S1.E1Xa.2.1.1.m1.3b"><apply id="S1.E1Xa.2.1.1.m1.3.3.cmml" xref="S1.E1Xa.2.1.1.m1.3.3"><in id="S1.E1Xa.2.1.1.m1.3.3.3.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.3"></in><list id="S1.E1Xa.2.1.1.m1.3.3.2.3.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.2.2"><apply id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1"><times id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.1.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.1"></times><ci id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.2a.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.2"><mtext id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.2.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.2">with</mtext></ci><apply id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3"><csymbol cd="ambiguous" id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.1.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3">subscript</csymbol><ci id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.2a.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.2"><mtext id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.2.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.2">func</mtext></ci><cn type="integer" id="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.3.cmml" xref="S1.E1Xa.2.1.1.m1.2.2.1.1.1.3.3">1</cn></apply></apply><ci id="S1.E1Xa.2.1.1.m1.1.1.cmml" xref="S1.E1Xa.2.1.1.m1.1.1">…</ci><apply id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.1.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2">subscript</csymbol><ci id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.2a.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2.2"><mtext id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.2.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2.2">func</mtext></ci><ci id="S1.E1Xa.2.1.1.m1.3.3.2.2.2.3.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.2.2.2.3">𝑚</ci></apply></list><csymbol cd="latexml" id="S1.E1Xa.2.1.1.m1.3.3.4.cmml" xref="S1.E1Xa.2.1.1.m1.3.3.4">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.E1Xa.2.1.1.m1.3c">\displaystyle\text{with}\ \text{func}_{1},\dots,\text{func}_{m}\in</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="S1.E1Xa.3.2.2.m1.7" class="ltx_Math" alttext="\displaystyle\{\text{abs}(),\log(),\dots\}\ \text{and}\ e_{1},\dots,e_{m}\in\{0.5,1,2\}" display="inline"><semantics id="S1.E1Xa.3.2.2.m1.7a"><mrow id="S1.E1Xa.3.2.2.m1.7.7" xref="S1.E1Xa.3.2.2.m1.7.7.cmml"><mrow id="S1.E1Xa.3.2.2.m1.7.7.2.2" xref="S1.E1Xa.3.2.2.m1.7.7.2.3.cmml"><mrow id="S1.E1Xa.3.2.2.m1.6.6.1.1.1" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.cmml"><mrow id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.3.cmml"><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.3" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.3.cmml">{</mo><mrow id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.cmml"><mtext id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.2a.cmml">abs</mtext><mo lspace="0em" rspace="0em" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.1" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.1.cmml">​</mo><mrow id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.2.1" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.1.cmml">(</mo><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.2.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.1.cmml">)</mo></mrow></mrow><mo id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.4" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.3.cmml">,</mo><mrow id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.cmml"><mi id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.2.cmml">log</mi><mo lspace="0em" rspace="0em" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.1" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.1.cmml">​</mo><mrow id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.cmml"><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.2.1" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.1.cmml">(</mo><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.2.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.1.cmml">)</mo></mrow></mrow><mo id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.5" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.3.cmml">,</mo><mi mathvariant="normal" id="S1.E1Xa.3.2.2.m1.1.1" xref="S1.E1Xa.3.2.2.m1.1.1.cmml">…</mi><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.6" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.3.cmml">}</mo></mrow><mo lspace="0.500em" rspace="0em" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.3" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.3.cmml">​</mo><mtext id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.4" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.4a.cmml">and</mtext><mo lspace="0.500em" rspace="0em" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.3a" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.3.cmml">​</mo><msub id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.cmml"><mi id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.2" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.2.cmml">e</mi><mn id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.3" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.3.cmml">1</mn></msub></mrow><mo id="S1.E1Xa.3.2.2.m1.7.7.2.2.3" xref="S1.E1Xa.3.2.2.m1.7.7.2.3.cmml">,</mo><mi mathvariant="normal" id="S1.E1Xa.3.2.2.m1.5.5" xref="S1.E1Xa.3.2.2.m1.5.5.cmml">…</mi><mo id="S1.E1Xa.3.2.2.m1.7.7.2.2.4" xref="S1.E1Xa.3.2.2.m1.7.7.2.3.cmml">,</mo><msub id="S1.E1Xa.3.2.2.m1.7.7.2.2.2" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2.cmml"><mi id="S1.E1Xa.3.2.2.m1.7.7.2.2.2.2" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2.2.cmml">e</mi><mi id="S1.E1Xa.3.2.2.m1.7.7.2.2.2.3" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2.3.cmml">m</mi></msub></mrow><mo id="S1.E1Xa.3.2.2.m1.7.7.3" xref="S1.E1Xa.3.2.2.m1.7.7.3.cmml">∈</mo><mrow id="S1.E1Xa.3.2.2.m1.7.7.4.2" xref="S1.E1Xa.3.2.2.m1.7.7.4.1.cmml"><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.7.7.4.2.1" xref="S1.E1Xa.3.2.2.m1.7.7.4.1.cmml">{</mo><mn id="S1.E1Xa.3.2.2.m1.2.2" xref="S1.E1Xa.3.2.2.m1.2.2.cmml">0.5</mn><mo id="S1.E1Xa.3.2.2.m1.7.7.4.2.2" xref="S1.E1Xa.3.2.2.m1.7.7.4.1.cmml">,</mo><mn id="S1.E1Xa.3.2.2.m1.3.3" xref="S1.E1Xa.3.2.2.m1.3.3.cmml">1</mn><mo id="S1.E1Xa.3.2.2.m1.7.7.4.2.3" xref="S1.E1Xa.3.2.2.m1.7.7.4.1.cmml">,</mo><mn id="S1.E1Xa.3.2.2.m1.4.4" xref="S1.E1Xa.3.2.2.m1.4.4.cmml">2</mn><mo stretchy="false" id="S1.E1Xa.3.2.2.m1.7.7.4.2.4" xref="S1.E1Xa.3.2.2.m1.7.7.4.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.E1Xa.3.2.2.m1.7b"><apply id="S1.E1Xa.3.2.2.m1.7.7.cmml" xref="S1.E1Xa.3.2.2.m1.7.7"><in id="S1.E1Xa.3.2.2.m1.7.7.3.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.3"></in><list id="S1.E1Xa.3.2.2.m1.7.7.2.3.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.2.2"><apply id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1"><times id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.3.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.3"></times><set id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.3.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2"><apply id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1"><times id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.1"></times><ci id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.2a.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.2"><mtext id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.2.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.2">abs</mtext></ci><list id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.1.1.1.3.2.1"></list></apply><apply id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2"><times id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.1"></times><log id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.2.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.2"></log><list id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.2.2.2.3.2.1"></list></apply><ci id="S1.E1Xa.3.2.2.m1.1.1.cmml" xref="S1.E1Xa.3.2.2.m1.1.1">…</ci></set><ci id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.4a.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.4"><mtext id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.4.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.4">and</mtext></ci><apply id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5"><csymbol cd="ambiguous" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.1.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5">subscript</csymbol><ci id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.2.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.2">𝑒</ci><cn type="integer" id="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.3.cmml" xref="S1.E1Xa.3.2.2.m1.6.6.1.1.1.5.3">1</cn></apply></apply><ci id="S1.E1Xa.3.2.2.m1.5.5.cmml" xref="S1.E1Xa.3.2.2.m1.5.5">…</ci><apply id="S1.E1Xa.3.2.2.m1.7.7.2.2.2.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2"><csymbol cd="ambiguous" id="S1.E1Xa.3.2.2.m1.7.7.2.2.2.1.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2">subscript</csymbol><ci id="S1.E1Xa.3.2.2.m1.7.7.2.2.2.2.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2.2">𝑒</ci><ci id="S1.E1Xa.3.2.2.m1.7.7.2.2.2.3.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.2.2.2.3">𝑚</ci></apply></list><set id="S1.E1Xa.3.2.2.m1.7.7.4.1.cmml" xref="S1.E1Xa.3.2.2.m1.7.7.4.2"><cn type="float" id="S1.E1Xa.3.2.2.m1.2.2.cmml" xref="S1.E1Xa.3.2.2.m1.2.2">0.5</cn><cn type="integer" id="S1.E1Xa.3.2.2.m1.3.3.cmml" xref="S1.E1Xa.3.2.2.m1.3.3">1</cn><cn type="integer" id="S1.E1Xa.3.2.2.m1.4.4.cmml" xref="S1.E1Xa.3.2.2.m1.4.4">2</cn></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.E1Xa.3.2.2.m1.7c">\displaystyle\{\text{abs}(),\log(),\dots\}\ \text{and}\ e_{1},\dots,e_{m}\in\{0.5,1,2\}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</tbody>
</table>
</div>
<section id="S1.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">1.1 </span>Motivation and Objectives</h3>

<div id="S1.SS1.p1" class="ltx_para">
<p id="S1.SS1.p1.5" class="ltx_p">A disadvantage of FFX is that only linear parameters are optimized. Parameters within nonlinear functions are not present. For example functions with feature <math id="S1.SS1.p1.1.m1.1" class="ltx_Math" alttext="x" display="inline"><semantics id="S1.SS1.p1.1.m1.1a"><mi id="S1.SS1.p1.1.m1.1.1" xref="S1.SS1.p1.1.m1.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="S1.SS1.p1.1.m1.1b"><ci id="S1.SS1.p1.1.m1.1.1.cmml" xref="S1.SS1.p1.1.m1.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.SS1.p1.1.m1.1c">x</annotation></semantics></math> and nonlinear parameters <math id="S1.SS1.p1.2.m2.1" class="ltx_Math" alttext="k_{i}" display="inline"><semantics id="S1.SS1.p1.2.m2.1a"><msub id="S1.SS1.p1.2.m2.1.1" xref="S1.SS1.p1.2.m2.1.1.cmml"><mi id="S1.SS1.p1.2.m2.1.1.2" xref="S1.SS1.p1.2.m2.1.1.2.cmml">k</mi><mi id="S1.SS1.p1.2.m2.1.1.3" xref="S1.SS1.p1.2.m2.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S1.SS1.p1.2.m2.1b"><apply id="S1.SS1.p1.2.m2.1.1.cmml" xref="S1.SS1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S1.SS1.p1.2.m2.1.1.1.cmml" xref="S1.SS1.p1.2.m2.1.1">subscript</csymbol><ci id="S1.SS1.p1.2.m2.1.1.2.cmml" xref="S1.SS1.p1.2.m2.1.1.2">𝑘</ci><ci id="S1.SS1.p1.2.m2.1.1.3.cmml" xref="S1.SS1.p1.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.SS1.p1.2.m2.1c">k_{i}</annotation></semantics></math> such as <math id="S1.SS1.p1.3.m3.2" class="ltx_Math" alttext="\log(x+k_{i})" display="inline"><semantics id="S1.SS1.p1.3.m3.2a"><mrow id="S1.SS1.p1.3.m3.2.2.1" xref="S1.SS1.p1.3.m3.2.2.2.cmml"><mi id="S1.SS1.p1.3.m3.1.1" xref="S1.SS1.p1.3.m3.1.1.cmml">log</mi><mo id="S1.SS1.p1.3.m3.2.2.1a" xref="S1.SS1.p1.3.m3.2.2.2.cmml">⁡</mo><mrow id="S1.SS1.p1.3.m3.2.2.1.1" xref="S1.SS1.p1.3.m3.2.2.2.cmml"><mo stretchy="false" id="S1.SS1.p1.3.m3.2.2.1.1.2" xref="S1.SS1.p1.3.m3.2.2.2.cmml">(</mo><mrow id="S1.SS1.p1.3.m3.2.2.1.1.1" xref="S1.SS1.p1.3.m3.2.2.1.1.1.cmml"><mi id="S1.SS1.p1.3.m3.2.2.1.1.1.2" xref="S1.SS1.p1.3.m3.2.2.1.1.1.2.cmml">x</mi><mo id="S1.SS1.p1.3.m3.2.2.1.1.1.1" xref="S1.SS1.p1.3.m3.2.2.1.1.1.1.cmml">+</mo><msub id="S1.SS1.p1.3.m3.2.2.1.1.1.3" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3.cmml"><mi id="S1.SS1.p1.3.m3.2.2.1.1.1.3.2" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3.2.cmml">k</mi><mi id="S1.SS1.p1.3.m3.2.2.1.1.1.3.3" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3.3.cmml">i</mi></msub></mrow><mo stretchy="false" id="S1.SS1.p1.3.m3.2.2.1.1.3" xref="S1.SS1.p1.3.m3.2.2.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.SS1.p1.3.m3.2b"><apply id="S1.SS1.p1.3.m3.2.2.2.cmml" xref="S1.SS1.p1.3.m3.2.2.1"><log id="S1.SS1.p1.3.m3.1.1.cmml" xref="S1.SS1.p1.3.m3.1.1"></log><apply id="S1.SS1.p1.3.m3.2.2.1.1.1.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1"><plus id="S1.SS1.p1.3.m3.2.2.1.1.1.1.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1.1"></plus><ci id="S1.SS1.p1.3.m3.2.2.1.1.1.2.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1.2">𝑥</ci><apply id="S1.SS1.p1.3.m3.2.2.1.1.1.3.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3"><csymbol cd="ambiguous" id="S1.SS1.p1.3.m3.2.2.1.1.1.3.1.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3">subscript</csymbol><ci id="S1.SS1.p1.3.m3.2.2.1.1.1.3.2.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3.2">𝑘</ci><ci id="S1.SS1.p1.3.m3.2.2.1.1.1.3.3.cmml" xref="S1.SS1.p1.3.m3.2.2.1.1.1.3.3">𝑖</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.SS1.p1.3.m3.2c">\log(x+k_{i})</annotation></semantics></math> or <math id="S1.SS1.p1.4.m4.2" class="ltx_Math" alttext="\exp(k_{i}x)" display="inline"><semantics id="S1.SS1.p1.4.m4.2a"><mrow id="S1.SS1.p1.4.m4.2.2.1" xref="S1.SS1.p1.4.m4.2.2.2.cmml"><mi id="S1.SS1.p1.4.m4.1.1" xref="S1.SS1.p1.4.m4.1.1.cmml">exp</mi><mo id="S1.SS1.p1.4.m4.2.2.1a" xref="S1.SS1.p1.4.m4.2.2.2.cmml">⁡</mo><mrow id="S1.SS1.p1.4.m4.2.2.1.1" xref="S1.SS1.p1.4.m4.2.2.2.cmml"><mo stretchy="false" id="S1.SS1.p1.4.m4.2.2.1.1.2" xref="S1.SS1.p1.4.m4.2.2.2.cmml">(</mo><mrow id="S1.SS1.p1.4.m4.2.2.1.1.1" xref="S1.SS1.p1.4.m4.2.2.1.1.1.cmml"><msub id="S1.SS1.p1.4.m4.2.2.1.1.1.2" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2.cmml"><mi id="S1.SS1.p1.4.m4.2.2.1.1.1.2.2" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2.2.cmml">k</mi><mi id="S1.SS1.p1.4.m4.2.2.1.1.1.2.3" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S1.SS1.p1.4.m4.2.2.1.1.1.1" xref="S1.SS1.p1.4.m4.2.2.1.1.1.1.cmml">​</mo><mi id="S1.SS1.p1.4.m4.2.2.1.1.1.3" xref="S1.SS1.p1.4.m4.2.2.1.1.1.3.cmml">x</mi></mrow><mo stretchy="false" id="S1.SS1.p1.4.m4.2.2.1.1.3" xref="S1.SS1.p1.4.m4.2.2.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S1.SS1.p1.4.m4.2b"><apply id="S1.SS1.p1.4.m4.2.2.2.cmml" xref="S1.SS1.p1.4.m4.2.2.1"><exp id="S1.SS1.p1.4.m4.1.1.cmml" xref="S1.SS1.p1.4.m4.1.1"></exp><apply id="S1.SS1.p1.4.m4.2.2.1.1.1.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1"><times id="S1.SS1.p1.4.m4.2.2.1.1.1.1.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1.1"></times><apply id="S1.SS1.p1.4.m4.2.2.1.1.1.2.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2"><csymbol cd="ambiguous" id="S1.SS1.p1.4.m4.2.2.1.1.1.2.1.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2">subscript</csymbol><ci id="S1.SS1.p1.4.m4.2.2.1.1.1.2.2.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2.2">𝑘</ci><ci id="S1.SS1.p1.4.m4.2.2.1.1.1.2.3.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1.2.3">𝑖</ci></apply><ci id="S1.SS1.p1.4.m4.2.2.1.1.1.3.cmml" xref="S1.SS1.p1.4.m4.2.2.1.1.1.3">𝑥</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.SS1.p1.4.m4.2c">\exp(k_{i}x)</annotation></semantics></math> have to be approximated by FFX with a linear model of several base functions. In this work, we extend the capabilities of FFX by adding such a real-valued parameter <math id="S1.SS1.p1.5.m5.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S1.SS1.p1.5.m5.1a"><mi id="S1.SS1.p1.5.m5.1.1" xref="S1.SS1.p1.5.m5.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S1.SS1.p1.5.m5.1b"><ci id="S1.SS1.p1.5.m5.1.1.cmml" xref="S1.SS1.p1.5.m5.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.SS1.p1.5.m5.1c">k</annotation></semantics></math> to each generated base functions. Adding several nonlinear parameters to the possible model structure should allow to fit additive models with fewer base functions, as we have more degrees of freedom per base function.</p>
</div>
<div id="S1.SS1.p2" class="ltx_para">
<p id="S1.SS1.p2.1" class="ltx_p">The introduced nonlinear parameters are optimized in combination with the linear parameters by separable nonlinear least squares optmization (NLS) using a variable projection algorithm <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib2" title="" class="ltx_ref">2</a>]</cite>. We call this new algorithm <span id="S1.SS1.p2.1.1" class="ltx_text ltx_font_italic">FFX with Nonlinear Least Squares Optimization</span> (FFX NLS). We test whether FFX NLS leads to higher accuracy than the original implementation on the <span id="S1.SS1.p2.1.2" class="ltx_text ltx_font_italic">PennML</span> benchmark suite <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib10" title="" class="ltx_ref">10</a>]</cite> and compare the complexity of the generated models. Given, that we have more degrees of freedom to fit data with a single base function, we expect to find that it produces models with a lower number of base functions and therefore simpler models than the original FFX algorithm.</p>
</div>
</section>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Algorithm Description</h2>

<div id="S2.p1" class="ltx_para">
<p id="S2.p1.1" class="ltx_p">Similar to FFX, FFX NLS runs in several steps. First, base functions are generated. Then the most relevant base functions and the model’s parameters are determined. In difference to FFX, the selection of relevant base function and the optimization of parameters are separate steps. FFX NLS performs the following four steps that are described in detail in the next sections:</p>
</div>
<div id="S2.p2" class="ltx_para">
<ol id="S2.I1" class="ltx_enumerate">
<li id="S2.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S2.I1.i1.p1" class="ltx_para">
<p id="S2.I1.i1.p1.1" class="ltx_p">Generate a list of all univariate base functions <math id="S2.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="\bm{f}" display="inline"><semantics id="S2.I1.i1.p1.1.m1.1a"><mi id="S2.I1.i1.p1.1.m1.1.1" xref="S2.I1.i1.p1.1.m1.1.1.cmml">𝒇</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i1.p1.1.m1.1b"><ci id="S2.I1.i1.p1.1.m1.1.1.cmml" xref="S2.I1.i1.p1.1.m1.1.1">𝒇</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i1.p1.1.m1.1c">\bm{f}</annotation></semantics></math> (cf. Section <a href="#S2.SS1" title="2.1 Base Functions ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.1</span></a>).</p>
</div>
</li>
<li id="S2.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S2.I1.i2.p1" class="ltx_para">
<p id="S2.I1.i2.p1.6" class="ltx_p">Optimize all parameters <math id="S2.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.I1.i2.p1.1.m1.1a"><mi id="S2.I1.i2.p1.1.m1.1.1" xref="S2.I1.i2.p1.1.m1.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.1.m1.1b"><ci id="S2.I1.i2.p1.1.m1.1.1.cmml" xref="S2.I1.i2.p1.1.m1.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.1.m1.1c">\bm{k}</annotation></semantics></math> and <math id="S2.I1.i2.p1.2.m2.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.I1.i2.p1.2.m2.1a"><mi id="S2.I1.i2.p1.2.m2.1.1" xref="S2.I1.i2.p1.2.m2.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.2.m2.1b"><ci id="S2.I1.i2.p1.2.m2.1.1.cmml" xref="S2.I1.i2.p1.2.m2.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.2.m2.1c">\bm{l}</annotation></semantics></math> of a nonlinear model <math id="S2.I1.i2.p1.3.m3.2" class="ltx_Math" alttext="l_{0}+\sum_{i}{l_{i}f_{i}(\bm{k},\bm{x})}" display="inline"><semantics id="S2.I1.i2.p1.3.m3.2a"><mrow id="S2.I1.i2.p1.3.m3.2.3" xref="S2.I1.i2.p1.3.m3.2.3.cmml"><msub id="S2.I1.i2.p1.3.m3.2.3.2" xref="S2.I1.i2.p1.3.m3.2.3.2.cmml"><mi id="S2.I1.i2.p1.3.m3.2.3.2.2" xref="S2.I1.i2.p1.3.m3.2.3.2.2.cmml">l</mi><mn id="S2.I1.i2.p1.3.m3.2.3.2.3" xref="S2.I1.i2.p1.3.m3.2.3.2.3.cmml">0</mn></msub><mo rspace="0.055em" id="S2.I1.i2.p1.3.m3.2.3.1" xref="S2.I1.i2.p1.3.m3.2.3.1.cmml">+</mo><mrow id="S2.I1.i2.p1.3.m3.2.3.3" xref="S2.I1.i2.p1.3.m3.2.3.3.cmml"><msub id="S2.I1.i2.p1.3.m3.2.3.3.1" xref="S2.I1.i2.p1.3.m3.2.3.3.1.cmml"><mo id="S2.I1.i2.p1.3.m3.2.3.3.1.2" xref="S2.I1.i2.p1.3.m3.2.3.3.1.2.cmml">∑</mo><mi id="S2.I1.i2.p1.3.m3.2.3.3.1.3" xref="S2.I1.i2.p1.3.m3.2.3.3.1.3.cmml">i</mi></msub><mrow id="S2.I1.i2.p1.3.m3.2.3.3.2" xref="S2.I1.i2.p1.3.m3.2.3.3.2.cmml"><msub id="S2.I1.i2.p1.3.m3.2.3.3.2.2" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2.cmml"><mi id="S2.I1.i2.p1.3.m3.2.3.3.2.2.2" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2.2.cmml">l</mi><mi id="S2.I1.i2.p1.3.m3.2.3.3.2.2.3" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.I1.i2.p1.3.m3.2.3.3.2.1" xref="S2.I1.i2.p1.3.m3.2.3.3.2.1.cmml">​</mo><msub id="S2.I1.i2.p1.3.m3.2.3.3.2.3" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3.cmml"><mi id="S2.I1.i2.p1.3.m3.2.3.3.2.3.2" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3.2.cmml">f</mi><mi id="S2.I1.i2.p1.3.m3.2.3.3.2.3.3" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.I1.i2.p1.3.m3.2.3.3.2.1a" xref="S2.I1.i2.p1.3.m3.2.3.3.2.1.cmml">​</mo><mrow id="S2.I1.i2.p1.3.m3.2.3.3.2.4.2" xref="S2.I1.i2.p1.3.m3.2.3.3.2.4.1.cmml"><mo stretchy="false" id="S2.I1.i2.p1.3.m3.2.3.3.2.4.2.1" xref="S2.I1.i2.p1.3.m3.2.3.3.2.4.1.cmml">(</mo><mi id="S2.I1.i2.p1.3.m3.1.1" xref="S2.I1.i2.p1.3.m3.1.1.cmml">𝒌</mi><mo id="S2.I1.i2.p1.3.m3.2.3.3.2.4.2.2" xref="S2.I1.i2.p1.3.m3.2.3.3.2.4.1.cmml">,</mo><mi id="S2.I1.i2.p1.3.m3.2.2" xref="S2.I1.i2.p1.3.m3.2.2.cmml">𝒙</mi><mo stretchy="false" id="S2.I1.i2.p1.3.m3.2.3.3.2.4.2.3" xref="S2.I1.i2.p1.3.m3.2.3.3.2.4.1.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.3.m3.2b"><apply id="S2.I1.i2.p1.3.m3.2.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3"><plus id="S2.I1.i2.p1.3.m3.2.3.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.1"></plus><apply id="S2.I1.i2.p1.3.m3.2.3.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.2"><csymbol cd="ambiguous" id="S2.I1.i2.p1.3.m3.2.3.2.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.2">subscript</csymbol><ci id="S2.I1.i2.p1.3.m3.2.3.2.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.2.2">𝑙</ci><cn type="integer" id="S2.I1.i2.p1.3.m3.2.3.2.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3.2.3">0</cn></apply><apply id="S2.I1.i2.p1.3.m3.2.3.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3"><apply id="S2.I1.i2.p1.3.m3.2.3.3.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.1"><csymbol cd="ambiguous" id="S2.I1.i2.p1.3.m3.2.3.3.1.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.1">subscript</csymbol><sum id="S2.I1.i2.p1.3.m3.2.3.3.1.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.1.2"></sum><ci id="S2.I1.i2.p1.3.m3.2.3.3.1.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.1.3">𝑖</ci></apply><apply id="S2.I1.i2.p1.3.m3.2.3.3.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2"><times id="S2.I1.i2.p1.3.m3.2.3.3.2.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.1"></times><apply id="S2.I1.i2.p1.3.m3.2.3.3.2.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2"><csymbol cd="ambiguous" id="S2.I1.i2.p1.3.m3.2.3.3.2.2.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2">subscript</csymbol><ci id="S2.I1.i2.p1.3.m3.2.3.3.2.2.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2.2">𝑙</ci><ci id="S2.I1.i2.p1.3.m3.2.3.3.2.2.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.2.3">𝑖</ci></apply><apply id="S2.I1.i2.p1.3.m3.2.3.3.2.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3"><csymbol cd="ambiguous" id="S2.I1.i2.p1.3.m3.2.3.3.2.3.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3">subscript</csymbol><ci id="S2.I1.i2.p1.3.m3.2.3.3.2.3.2.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3.2">𝑓</ci><ci id="S2.I1.i2.p1.3.m3.2.3.3.2.3.3.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.3.3">𝑖</ci></apply><interval closure="open" id="S2.I1.i2.p1.3.m3.2.3.3.2.4.1.cmml" xref="S2.I1.i2.p1.3.m3.2.3.3.2.4.2"><ci id="S2.I1.i2.p1.3.m3.1.1.cmml" xref="S2.I1.i2.p1.3.m3.1.1">𝒌</ci><ci id="S2.I1.i2.p1.3.m3.2.2.cmml" xref="S2.I1.i2.p1.3.m3.2.2">𝒙</ci></interval></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.3.m3.2c">l_{0}+\sum_{i}{l_{i}f_{i}(\bm{k},\bm{x})}</annotation></semantics></math> that consists of the base functions as terms. <math id="S2.I1.i2.p1.4.m4.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.I1.i2.p1.4.m4.1a"><mi id="S2.I1.i2.p1.4.m4.1.1" xref="S2.I1.i2.p1.4.m4.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.4.m4.1b"><ci id="S2.I1.i2.p1.4.m4.1.1.cmml" xref="S2.I1.i2.p1.4.m4.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.4.m4.1c">\bm{k}</annotation></semantics></math> is a vector of all nonlinear parameters. <math id="S2.I1.i2.p1.5.m5.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.I1.i2.p1.5.m5.1a"><mi id="S2.I1.i2.p1.5.m5.1.1" xref="S2.I1.i2.p1.5.m5.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.5.m5.1b"><ci id="S2.I1.i2.p1.5.m5.1.1.cmml" xref="S2.I1.i2.p1.5.m5.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.5.m5.1c">\bm{l}</annotation></semantics></math> are linear parameters, <math id="S2.I1.i2.p1.6.m6.1" class="ltx_Math" alttext="\bm{x}" display="inline"><semantics id="S2.I1.i2.p1.6.m6.1a"><mi id="S2.I1.i2.p1.6.m6.1.1" xref="S2.I1.i2.p1.6.m6.1.1.cmml">𝒙</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.6.m6.1b"><ci id="S2.I1.i2.p1.6.m6.1.1.cmml" xref="S2.I1.i2.p1.6.m6.1.1">𝒙</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.6.m6.1c">\bm{x}</annotation></semantics></math> are features (cf. Section <a href="#S2.SS2" title="2.2 Parameter Optimization with Variable Projection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a>).</p>
</div>
</li>
<li id="S2.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="S2.I1.i3.p1" class="ltx_para">
<p id="S2.I1.i3.p1.2" class="ltx_p">Select most important base functions of <math id="S2.I1.i3.p1.1.m1.1" class="ltx_Math" alttext="\bm{f}" display="inline"><semantics id="S2.I1.i3.p1.1.m1.1a"><mi id="S2.I1.i3.p1.1.m1.1.1" xref="S2.I1.i3.p1.1.m1.1.1.cmml">𝒇</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i3.p1.1.m1.1b"><ci id="S2.I1.i3.p1.1.m1.1.1.cmml" xref="S2.I1.i3.p1.1.m1.1.1">𝒇</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i3.p1.1.m1.1c">\bm{f}</annotation></semantics></math> with a regularized linear model and the nonlinear parameters <math id="S2.I1.i3.p1.2.m2.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.I1.i3.p1.2.m2.1a"><mi id="S2.I1.i3.p1.2.m2.1.1" xref="S2.I1.i3.p1.2.m2.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i3.p1.2.m2.1b"><ci id="S2.I1.i3.p1.2.m2.1.1.cmml" xref="S2.I1.i3.p1.2.m2.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i3.p1.2.m2.1c">\bm{k}</annotation></semantics></math> from the previous step (cf. Section <a href="#S2.SS3" title="2.3 Base Function Selection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.3</span></a>).</p>
</div>
</li>
<li id="S2.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">4.</span> 
<div id="S2.I1.i4.p1" class="ltx_para">
<p id="S2.I1.i4.p1.1" class="ltx_p">The final model is created by optimizing all parameters again with nonlinear least squares optimization but only using the most important base function (cf. Section <a href="#S2.SS4" title="2.4 Training of Final Model and Bivariate Base Functions ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.4</span></a>).</p>
</div>
</li>
</ol>
</div>
<section id="S2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.1 </span>Base Functions</h3>

<div id="S2.SS1.p1" class="ltx_para">
<p id="S2.SS1.p1.10" class="ltx_p">In step 1, we generate all univariate base functions <math id="S2.SS1.p1.1.m1.1" class="ltx_Math" alttext="\bm{f}" display="inline"><semantics id="S2.SS1.p1.1.m1.1a"><mi id="S2.SS1.p1.1.m1.1.1" xref="S2.SS1.p1.1.m1.1.1.cmml">𝒇</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.1.m1.1b"><ci id="S2.SS1.p1.1.m1.1.1.cmml" xref="S2.SS1.p1.1.m1.1.1">𝒇</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.1.m1.1c">\bm{f}</annotation></semantics></math> with placeholders for nonlinear parameters. For each feature <math id="S2.SS1.p1.2.m2.1" class="ltx_Math" alttext="x_{i}" display="inline"><semantics id="S2.SS1.p1.2.m2.1a"><msub id="S2.SS1.p1.2.m2.1.1" xref="S2.SS1.p1.2.m2.1.1.cmml"><mi id="S2.SS1.p1.2.m2.1.1.2" xref="S2.SS1.p1.2.m2.1.1.2.cmml">x</mi><mi id="S2.SS1.p1.2.m2.1.1.3" xref="S2.SS1.p1.2.m2.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.2.m2.1b"><apply id="S2.SS1.p1.2.m2.1.1.cmml" xref="S2.SS1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.2.m2.1.1.1.cmml" xref="S2.SS1.p1.2.m2.1.1">subscript</csymbol><ci id="S2.SS1.p1.2.m2.1.1.2.cmml" xref="S2.SS1.p1.2.m2.1.1.2">𝑥</ci><ci id="S2.SS1.p1.2.m2.1.1.3.cmml" xref="S2.SS1.p1.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.2.m2.1c">x_{i}</annotation></semantics></math> we create base functions of structure <math id="S2.SS1.p1.3.m3.1" class="ltx_Math" alttext="\mathit{func}(ax_{i}^{p}+b)" display="inline"><semantics id="S2.SS1.p1.3.m3.1a"><mrow id="S2.SS1.p1.3.m3.1.1" xref="S2.SS1.p1.3.m3.1.1.cmml"><mi id="S2.SS1.p1.3.m3.1.1.3" xref="S2.SS1.p1.3.m3.1.1.3.cmml">𝑓𝑢𝑛𝑐</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p1.3.m3.1.1.2" xref="S2.SS1.p1.3.m3.1.1.2.cmml">​</mo><mrow id="S2.SS1.p1.3.m3.1.1.1.1" xref="S2.SS1.p1.3.m3.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.p1.3.m3.1.1.1.1.2" xref="S2.SS1.p1.3.m3.1.1.1.1.1.cmml">(</mo><mrow id="S2.SS1.p1.3.m3.1.1.1.1.1" xref="S2.SS1.p1.3.m3.1.1.1.1.1.cmml"><mrow id="S2.SS1.p1.3.m3.1.1.1.1.1.2" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.cmml"><mi id="S2.SS1.p1.3.m3.1.1.1.1.1.2.2" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.2.cmml">a</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p1.3.m3.1.1.1.1.1.2.1" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.1.cmml">​</mo><msubsup id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.cmml"><mi id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.2" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.2.cmml">x</mi><mi id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.3" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.3.cmml">i</mi><mi id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.3" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.3.cmml">p</mi></msubsup></mrow><mo id="S2.SS1.p1.3.m3.1.1.1.1.1.1" xref="S2.SS1.p1.3.m3.1.1.1.1.1.1.cmml">+</mo><mi id="S2.SS1.p1.3.m3.1.1.1.1.1.3" xref="S2.SS1.p1.3.m3.1.1.1.1.1.3.cmml">b</mi></mrow><mo stretchy="false" id="S2.SS1.p1.3.m3.1.1.1.1.3" xref="S2.SS1.p1.3.m3.1.1.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.3.m3.1b"><apply id="S2.SS1.p1.3.m3.1.1.cmml" xref="S2.SS1.p1.3.m3.1.1"><times id="S2.SS1.p1.3.m3.1.1.2.cmml" xref="S2.SS1.p1.3.m3.1.1.2"></times><ci id="S2.SS1.p1.3.m3.1.1.3.cmml" xref="S2.SS1.p1.3.m3.1.1.3">𝑓𝑢𝑛𝑐</ci><apply id="S2.SS1.p1.3.m3.1.1.1.1.1.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1"><plus id="S2.SS1.p1.3.m3.1.1.1.1.1.1.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.1"></plus><apply id="S2.SS1.p1.3.m3.1.1.1.1.1.2.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2"><times id="S2.SS1.p1.3.m3.1.1.1.1.1.2.1.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.1"></times><ci id="S2.SS1.p1.3.m3.1.1.1.1.1.2.2.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.2">𝑎</ci><apply id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.1.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3">superscript</csymbol><apply id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3"><csymbol cd="ambiguous" id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.1.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3">subscript</csymbol><ci id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.2.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.2">𝑥</ci><ci id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.3.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.2.3">𝑖</ci></apply><ci id="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.3.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.2.3.3">𝑝</ci></apply></apply><ci id="S2.SS1.p1.3.m3.1.1.1.1.1.3.cmml" xref="S2.SS1.p1.3.m3.1.1.1.1.1.3">𝑏</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.3.m3.1c">\mathit{func}(ax_{i}^{p}+b)</annotation></semantics></math> with <math id="S2.SS1.p1.4.m4.4" class="ltx_Math" alttext="\mathit{func}\in\{\text{id},\log,\exp,\text{sqrt}\}" display="inline"><semantics id="S2.SS1.p1.4.m4.4a"><mrow id="S2.SS1.p1.4.m4.4.5" xref="S2.SS1.p1.4.m4.4.5.cmml"><mi id="S2.SS1.p1.4.m4.4.5.2" xref="S2.SS1.p1.4.m4.4.5.2.cmml">𝑓𝑢𝑛𝑐</mi><mo id="S2.SS1.p1.4.m4.4.5.1" xref="S2.SS1.p1.4.m4.4.5.1.cmml">∈</mo><mrow id="S2.SS1.p1.4.m4.4.5.3.2" xref="S2.SS1.p1.4.m4.4.5.3.1.cmml"><mo stretchy="false" id="S2.SS1.p1.4.m4.4.5.3.2.1" xref="S2.SS1.p1.4.m4.4.5.3.1.cmml">{</mo><mtext id="S2.SS1.p1.4.m4.1.1" xref="S2.SS1.p1.4.m4.1.1a.cmml">id</mtext><mo id="S2.SS1.p1.4.m4.4.5.3.2.2" xref="S2.SS1.p1.4.m4.4.5.3.1.cmml">,</mo><mi id="S2.SS1.p1.4.m4.2.2" xref="S2.SS1.p1.4.m4.2.2.cmml">log</mi><mo id="S2.SS1.p1.4.m4.4.5.3.2.3" xref="S2.SS1.p1.4.m4.4.5.3.1.cmml">,</mo><mi id="S2.SS1.p1.4.m4.3.3" xref="S2.SS1.p1.4.m4.3.3.cmml">exp</mi><mo id="S2.SS1.p1.4.m4.4.5.3.2.4" xref="S2.SS1.p1.4.m4.4.5.3.1.cmml">,</mo><mtext id="S2.SS1.p1.4.m4.4.4" xref="S2.SS1.p1.4.m4.4.4a.cmml">sqrt</mtext><mo stretchy="false" id="S2.SS1.p1.4.m4.4.5.3.2.5" xref="S2.SS1.p1.4.m4.4.5.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.4.m4.4b"><apply id="S2.SS1.p1.4.m4.4.5.cmml" xref="S2.SS1.p1.4.m4.4.5"><in id="S2.SS1.p1.4.m4.4.5.1.cmml" xref="S2.SS1.p1.4.m4.4.5.1"></in><ci id="S2.SS1.p1.4.m4.4.5.2.cmml" xref="S2.SS1.p1.4.m4.4.5.2">𝑓𝑢𝑛𝑐</ci><set id="S2.SS1.p1.4.m4.4.5.3.1.cmml" xref="S2.SS1.p1.4.m4.4.5.3.2"><ci id="S2.SS1.p1.4.m4.1.1a.cmml" xref="S2.SS1.p1.4.m4.1.1"><mtext id="S2.SS1.p1.4.m4.1.1.cmml" xref="S2.SS1.p1.4.m4.1.1">id</mtext></ci><log id="S2.SS1.p1.4.m4.2.2.cmml" xref="S2.SS1.p1.4.m4.2.2"></log><exp id="S2.SS1.p1.4.m4.3.3.cmml" xref="S2.SS1.p1.4.m4.3.3"></exp><ci id="S2.SS1.p1.4.m4.4.4a.cmml" xref="S2.SS1.p1.4.m4.4.4"><mtext id="S2.SS1.p1.4.m4.4.4.cmml" xref="S2.SS1.p1.4.m4.4.4">sqrt</mtext></ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.4.m4.4c">\mathit{func}\in\{\text{id},\log,\exp,\text{sqrt}\}</annotation></semantics></math>, <math id="S2.SS1.p1.5.m5.2" class="ltx_Math" alttext="p\in\{1,2\}" display="inline"><semantics id="S2.SS1.p1.5.m5.2a"><mrow id="S2.SS1.p1.5.m5.2.3" xref="S2.SS1.p1.5.m5.2.3.cmml"><mi id="S2.SS1.p1.5.m5.2.3.2" xref="S2.SS1.p1.5.m5.2.3.2.cmml">p</mi><mo id="S2.SS1.p1.5.m5.2.3.1" xref="S2.SS1.p1.5.m5.2.3.1.cmml">∈</mo><mrow id="S2.SS1.p1.5.m5.2.3.3.2" xref="S2.SS1.p1.5.m5.2.3.3.1.cmml"><mo stretchy="false" id="S2.SS1.p1.5.m5.2.3.3.2.1" xref="S2.SS1.p1.5.m5.2.3.3.1.cmml">{</mo><mn id="S2.SS1.p1.5.m5.1.1" xref="S2.SS1.p1.5.m5.1.1.cmml">1</mn><mo id="S2.SS1.p1.5.m5.2.3.3.2.2" xref="S2.SS1.p1.5.m5.2.3.3.1.cmml">,</mo><mn id="S2.SS1.p1.5.m5.2.2" xref="S2.SS1.p1.5.m5.2.2.cmml">2</mn><mo stretchy="false" id="S2.SS1.p1.5.m5.2.3.3.2.3" xref="S2.SS1.p1.5.m5.2.3.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.5.m5.2b"><apply id="S2.SS1.p1.5.m5.2.3.cmml" xref="S2.SS1.p1.5.m5.2.3"><in id="S2.SS1.p1.5.m5.2.3.1.cmml" xref="S2.SS1.p1.5.m5.2.3.1"></in><ci id="S2.SS1.p1.5.m5.2.3.2.cmml" xref="S2.SS1.p1.5.m5.2.3.2">𝑝</ci><set id="S2.SS1.p1.5.m5.2.3.3.1.cmml" xref="S2.SS1.p1.5.m5.2.3.3.2"><cn type="integer" id="S2.SS1.p1.5.m5.1.1.cmml" xref="S2.SS1.p1.5.m5.1.1">1</cn><cn type="integer" id="S2.SS1.p1.5.m5.2.2.cmml" xref="S2.SS1.p1.5.m5.2.2">2</cn></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.5.m5.2c">p\in\{1,2\}</annotation></semantics></math> and two scaling values <math id="S2.SS1.p1.6.m6.1" class="ltx_Math" alttext="a" display="inline"><semantics id="S2.SS1.p1.6.m6.1a"><mi id="S2.SS1.p1.6.m6.1.1" xref="S2.SS1.p1.6.m6.1.1.cmml">a</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.6.m6.1b"><ci id="S2.SS1.p1.6.m6.1.1.cmml" xref="S2.SS1.p1.6.m6.1.1">𝑎</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.6.m6.1c">a</annotation></semantics></math> and <math id="S2.SS1.p1.7.m7.1" class="ltx_Math" alttext="b" display="inline"><semantics id="S2.SS1.p1.7.m7.1a"><mi id="S2.SS1.p1.7.m7.1.1" xref="S2.SS1.p1.7.m7.1.1.cmml">b</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.7.m7.1b"><ci id="S2.SS1.p1.7.m7.1.1.cmml" xref="S2.SS1.p1.7.m7.1.1">𝑏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.7.m7.1c">b</annotation></semantics></math> with <math id="S2.SS1.p1.8.m8.1" class="ltx_Math" alttext="\text{id}(x)=x" display="inline"><semantics id="S2.SS1.p1.8.m8.1a"><mrow id="S2.SS1.p1.8.m8.1.2" xref="S2.SS1.p1.8.m8.1.2.cmml"><mrow id="S2.SS1.p1.8.m8.1.2.2" xref="S2.SS1.p1.8.m8.1.2.2.cmml"><mtext id="S2.SS1.p1.8.m8.1.2.2.2" xref="S2.SS1.p1.8.m8.1.2.2.2a.cmml">id</mtext><mo lspace="0em" rspace="0em" id="S2.SS1.p1.8.m8.1.2.2.1" xref="S2.SS1.p1.8.m8.1.2.2.1.cmml">​</mo><mrow id="S2.SS1.p1.8.m8.1.2.2.3.2" xref="S2.SS1.p1.8.m8.1.2.2.cmml"><mo stretchy="false" id="S2.SS1.p1.8.m8.1.2.2.3.2.1" xref="S2.SS1.p1.8.m8.1.2.2.cmml">(</mo><mi id="S2.SS1.p1.8.m8.1.1" xref="S2.SS1.p1.8.m8.1.1.cmml">x</mi><mo stretchy="false" id="S2.SS1.p1.8.m8.1.2.2.3.2.2" xref="S2.SS1.p1.8.m8.1.2.2.cmml">)</mo></mrow></mrow><mo id="S2.SS1.p1.8.m8.1.2.1" xref="S2.SS1.p1.8.m8.1.2.1.cmml">=</mo><mi id="S2.SS1.p1.8.m8.1.2.3" xref="S2.SS1.p1.8.m8.1.2.3.cmml">x</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.8.m8.1b"><apply id="S2.SS1.p1.8.m8.1.2.cmml" xref="S2.SS1.p1.8.m8.1.2"><eq id="S2.SS1.p1.8.m8.1.2.1.cmml" xref="S2.SS1.p1.8.m8.1.2.1"></eq><apply id="S2.SS1.p1.8.m8.1.2.2.cmml" xref="S2.SS1.p1.8.m8.1.2.2"><times id="S2.SS1.p1.8.m8.1.2.2.1.cmml" xref="S2.SS1.p1.8.m8.1.2.2.1"></times><ci id="S2.SS1.p1.8.m8.1.2.2.2a.cmml" xref="S2.SS1.p1.8.m8.1.2.2.2"><mtext id="S2.SS1.p1.8.m8.1.2.2.2.cmml" xref="S2.SS1.p1.8.m8.1.2.2.2">id</mtext></ci><ci id="S2.SS1.p1.8.m8.1.1.cmml" xref="S2.SS1.p1.8.m8.1.1">𝑥</ci></apply><ci id="S2.SS1.p1.8.m8.1.2.3.cmml" xref="S2.SS1.p1.8.m8.1.2.3">𝑥</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.8.m8.1c">\text{id}(x)=x</annotation></semantics></math>. The scaling values <math id="S2.SS1.p1.9.m9.1" class="ltx_Math" alttext="a" display="inline"><semantics id="S2.SS1.p1.9.m9.1a"><mi id="S2.SS1.p1.9.m9.1.1" xref="S2.SS1.p1.9.m9.1.1.cmml">a</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.9.m9.1b"><ci id="S2.SS1.p1.9.m9.1.1.cmml" xref="S2.SS1.p1.9.m9.1.1">𝑎</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.9.m9.1c">a</annotation></semantics></math> and <math id="S2.SS1.p1.10.m10.1" class="ltx_Math" alttext="b" display="inline"><semantics id="S2.SS1.p1.10.m10.1a"><mi id="S2.SS1.p1.10.m10.1.1" xref="S2.SS1.p1.10.m10.1.1.cmml">b</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.10.m10.1b"><ci id="S2.SS1.p1.10.m10.1.1.cmml" xref="S2.SS1.p1.10.m10.1.1">𝑏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.10.m10.1c">b</annotation></semantics></math> are placeholder for nonlinear parameters that will be optimized later on. We also include bivariate base functions, which are described in Section <a href="#S2.SS4" title="2.4 Training of Final Model and Bivariate Base Functions ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.4</span></a>.</p>
</div>
<div id="S2.SS1.p2" class="ltx_para">
<p id="S2.SS1.p2.8" class="ltx_p">We utilize the linear structure in the final model as well as mathematical identities of the used nonlinear functions to reduce the number of nonlinear parameters. Due to mathematical identities such as <math id="S2.SS1.p2.1.m1.6" class="ltx_Math" alttext="\log(ax_{i})=\log(a)+\log(x_{i})" display="inline"><semantics id="S2.SS1.p2.1.m1.6a"><mrow id="S2.SS1.p2.1.m1.6.6" xref="S2.SS1.p2.1.m1.6.6.cmml"><mrow id="S2.SS1.p2.1.m1.5.5.1.1" xref="S2.SS1.p2.1.m1.5.5.1.2.cmml"><mi id="S2.SS1.p2.1.m1.1.1" xref="S2.SS1.p2.1.m1.1.1.cmml">log</mi><mo id="S2.SS1.p2.1.m1.5.5.1.1a" xref="S2.SS1.p2.1.m1.5.5.1.2.cmml">⁡</mo><mrow id="S2.SS1.p2.1.m1.5.5.1.1.1" xref="S2.SS1.p2.1.m1.5.5.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.1.m1.5.5.1.1.1.2" xref="S2.SS1.p2.1.m1.5.5.1.2.cmml">(</mo><mrow id="S2.SS1.p2.1.m1.5.5.1.1.1.1" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.cmml"><mi id="S2.SS1.p2.1.m1.5.5.1.1.1.1.2" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.2.cmml">a</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.1.m1.5.5.1.1.1.1.1" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.1.cmml">​</mo><msub id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.cmml"><mi id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.2" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.2.cmml">x</mi><mi id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.3" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.3.cmml">i</mi></msub></mrow><mo stretchy="false" id="S2.SS1.p2.1.m1.5.5.1.1.1.3" xref="S2.SS1.p2.1.m1.5.5.1.2.cmml">)</mo></mrow></mrow><mo id="S2.SS1.p2.1.m1.6.6.3" xref="S2.SS1.p2.1.m1.6.6.3.cmml">=</mo><mrow id="S2.SS1.p2.1.m1.6.6.2" xref="S2.SS1.p2.1.m1.6.6.2.cmml"><mrow id="S2.SS1.p2.1.m1.6.6.2.3.2" xref="S2.SS1.p2.1.m1.6.6.2.3.1.cmml"><mi id="S2.SS1.p2.1.m1.2.2" xref="S2.SS1.p2.1.m1.2.2.cmml">log</mi><mo id="S2.SS1.p2.1.m1.6.6.2.3.2a" xref="S2.SS1.p2.1.m1.6.6.2.3.1.cmml">⁡</mo><mrow id="S2.SS1.p2.1.m1.6.6.2.3.2.1" xref="S2.SS1.p2.1.m1.6.6.2.3.1.cmml"><mo stretchy="false" id="S2.SS1.p2.1.m1.6.6.2.3.2.1.1" xref="S2.SS1.p2.1.m1.6.6.2.3.1.cmml">(</mo><mi id="S2.SS1.p2.1.m1.3.3" xref="S2.SS1.p2.1.m1.3.3.cmml">a</mi><mo stretchy="false" id="S2.SS1.p2.1.m1.6.6.2.3.2.1.2" xref="S2.SS1.p2.1.m1.6.6.2.3.1.cmml">)</mo></mrow></mrow><mo id="S2.SS1.p2.1.m1.6.6.2.2" xref="S2.SS1.p2.1.m1.6.6.2.2.cmml">+</mo><mrow id="S2.SS1.p2.1.m1.6.6.2.1.1" xref="S2.SS1.p2.1.m1.6.6.2.1.2.cmml"><mi id="S2.SS1.p2.1.m1.4.4" xref="S2.SS1.p2.1.m1.4.4.cmml">log</mi><mo id="S2.SS1.p2.1.m1.6.6.2.1.1a" xref="S2.SS1.p2.1.m1.6.6.2.1.2.cmml">⁡</mo><mrow id="S2.SS1.p2.1.m1.6.6.2.1.1.1" xref="S2.SS1.p2.1.m1.6.6.2.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.1.m1.6.6.2.1.1.1.2" xref="S2.SS1.p2.1.m1.6.6.2.1.2.cmml">(</mo><msub id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.cmml"><mi id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.2" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.2.cmml">x</mi><mi id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.3" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="S2.SS1.p2.1.m1.6.6.2.1.1.1.3" xref="S2.SS1.p2.1.m1.6.6.2.1.2.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.1.m1.6b"><apply id="S2.SS1.p2.1.m1.6.6.cmml" xref="S2.SS1.p2.1.m1.6.6"><eq id="S2.SS1.p2.1.m1.6.6.3.cmml" xref="S2.SS1.p2.1.m1.6.6.3"></eq><apply id="S2.SS1.p2.1.m1.5.5.1.2.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1"><log id="S2.SS1.p2.1.m1.1.1.cmml" xref="S2.SS1.p2.1.m1.1.1"></log><apply id="S2.SS1.p2.1.m1.5.5.1.1.1.1.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1"><times id="S2.SS1.p2.1.m1.5.5.1.1.1.1.1.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.1"></times><ci id="S2.SS1.p2.1.m1.5.5.1.1.1.1.2.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.2">𝑎</ci><apply id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3"><csymbol cd="ambiguous" id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.1.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3">subscript</csymbol><ci id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.2.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.2">𝑥</ci><ci id="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.3.cmml" xref="S2.SS1.p2.1.m1.5.5.1.1.1.1.3.3">𝑖</ci></apply></apply></apply><apply id="S2.SS1.p2.1.m1.6.6.2.cmml" xref="S2.SS1.p2.1.m1.6.6.2"><plus id="S2.SS1.p2.1.m1.6.6.2.2.cmml" xref="S2.SS1.p2.1.m1.6.6.2.2"></plus><apply id="S2.SS1.p2.1.m1.6.6.2.3.1.cmml" xref="S2.SS1.p2.1.m1.6.6.2.3.2"><log id="S2.SS1.p2.1.m1.2.2.cmml" xref="S2.SS1.p2.1.m1.2.2"></log><ci id="S2.SS1.p2.1.m1.3.3.cmml" xref="S2.SS1.p2.1.m1.3.3">𝑎</ci></apply><apply id="S2.SS1.p2.1.m1.6.6.2.1.2.cmml" xref="S2.SS1.p2.1.m1.6.6.2.1.1"><log id="S2.SS1.p2.1.m1.4.4.cmml" xref="S2.SS1.p2.1.m1.4.4"></log><apply id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.cmml" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.1.cmml" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1">subscript</csymbol><ci id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.2.cmml" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.2">𝑥</ci><ci id="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.3.cmml" xref="S2.SS1.p2.1.m1.6.6.2.1.1.1.1.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.1.m1.6c">\log(ax_{i})=\log(a)+\log(x_{i})</annotation></semantics></math> with <math id="S2.SS1.p2.2.m2.1" class="ltx_Math" alttext="a" display="inline"><semantics id="S2.SS1.p2.2.m2.1a"><mi id="S2.SS1.p2.2.m2.1.1" xref="S2.SS1.p2.2.m2.1.1.cmml">a</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.2.m2.1b"><ci id="S2.SS1.p2.2.m2.1.1.cmml" xref="S2.SS1.p2.2.m2.1.1">𝑎</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.2.m2.1c">a</annotation></semantics></math> being a parameter that is trained later on, we can skip certain scaling values. In the case of logarithm, we can just use <math id="S2.SS1.p2.3.m3.2" class="ltx_Math" alttext="\log(x_{i}+b)" display="inline"><semantics id="S2.SS1.p2.3.m3.2a"><mrow id="S2.SS1.p2.3.m3.2.2.1" xref="S2.SS1.p2.3.m3.2.2.2.cmml"><mi id="S2.SS1.p2.3.m3.1.1" xref="S2.SS1.p2.3.m3.1.1.cmml">log</mi><mo id="S2.SS1.p2.3.m3.2.2.1a" xref="S2.SS1.p2.3.m3.2.2.2.cmml">⁡</mo><mrow id="S2.SS1.p2.3.m3.2.2.1.1" xref="S2.SS1.p2.3.m3.2.2.2.cmml"><mo stretchy="false" id="S2.SS1.p2.3.m3.2.2.1.1.2" xref="S2.SS1.p2.3.m3.2.2.2.cmml">(</mo><mrow id="S2.SS1.p2.3.m3.2.2.1.1.1" xref="S2.SS1.p2.3.m3.2.2.1.1.1.cmml"><msub id="S2.SS1.p2.3.m3.2.2.1.1.1.2" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2.cmml"><mi id="S2.SS1.p2.3.m3.2.2.1.1.1.2.2" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2.2.cmml">x</mi><mi id="S2.SS1.p2.3.m3.2.2.1.1.1.2.3" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2.3.cmml">i</mi></msub><mo id="S2.SS1.p2.3.m3.2.2.1.1.1.1" xref="S2.SS1.p2.3.m3.2.2.1.1.1.1.cmml">+</mo><mi id="S2.SS1.p2.3.m3.2.2.1.1.1.3" xref="S2.SS1.p2.3.m3.2.2.1.1.1.3.cmml">b</mi></mrow><mo stretchy="false" id="S2.SS1.p2.3.m3.2.2.1.1.3" xref="S2.SS1.p2.3.m3.2.2.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.3.m3.2b"><apply id="S2.SS1.p2.3.m3.2.2.2.cmml" xref="S2.SS1.p2.3.m3.2.2.1"><log id="S2.SS1.p2.3.m3.1.1.cmml" xref="S2.SS1.p2.3.m3.1.1"></log><apply id="S2.SS1.p2.3.m3.2.2.1.1.1.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1"><plus id="S2.SS1.p2.3.m3.2.2.1.1.1.1.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1.1"></plus><apply id="S2.SS1.p2.3.m3.2.2.1.1.1.2.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2"><csymbol cd="ambiguous" id="S2.SS1.p2.3.m3.2.2.1.1.1.2.1.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2">subscript</csymbol><ci id="S2.SS1.p2.3.m3.2.2.1.1.1.2.2.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2.2">𝑥</ci><ci id="S2.SS1.p2.3.m3.2.2.1.1.1.2.3.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1.2.3">𝑖</ci></apply><ci id="S2.SS1.p2.3.m3.2.2.1.1.1.3.cmml" xref="S2.SS1.p2.3.m3.2.2.1.1.1.3">𝑏</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.3.m3.2c">\log(x_{i}+b)</annotation></semantics></math> instead of <math id="S2.SS1.p2.4.m4.2" class="ltx_Math" alttext="\log(ax_{i}+b)" display="inline"><semantics id="S2.SS1.p2.4.m4.2a"><mrow id="S2.SS1.p2.4.m4.2.2.1" xref="S2.SS1.p2.4.m4.2.2.2.cmml"><mi id="S2.SS1.p2.4.m4.1.1" xref="S2.SS1.p2.4.m4.1.1.cmml">log</mi><mo id="S2.SS1.p2.4.m4.2.2.1a" xref="S2.SS1.p2.4.m4.2.2.2.cmml">⁡</mo><mrow id="S2.SS1.p2.4.m4.2.2.1.1" xref="S2.SS1.p2.4.m4.2.2.2.cmml"><mo stretchy="false" id="S2.SS1.p2.4.m4.2.2.1.1.2" xref="S2.SS1.p2.4.m4.2.2.2.cmml">(</mo><mrow id="S2.SS1.p2.4.m4.2.2.1.1.1" xref="S2.SS1.p2.4.m4.2.2.1.1.1.cmml"><mrow id="S2.SS1.p2.4.m4.2.2.1.1.1.2" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.cmml"><mi id="S2.SS1.p2.4.m4.2.2.1.1.1.2.2" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.2.cmml">a</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.4.m4.2.2.1.1.1.2.1" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.1.cmml">​</mo><msub id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.cmml"><mi id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.2" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.2.cmml">x</mi><mi id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.3" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.3.cmml">i</mi></msub></mrow><mo id="S2.SS1.p2.4.m4.2.2.1.1.1.1" xref="S2.SS1.p2.4.m4.2.2.1.1.1.1.cmml">+</mo><mi id="S2.SS1.p2.4.m4.2.2.1.1.1.3" xref="S2.SS1.p2.4.m4.2.2.1.1.1.3.cmml">b</mi></mrow><mo stretchy="false" id="S2.SS1.p2.4.m4.2.2.1.1.3" xref="S2.SS1.p2.4.m4.2.2.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.4.m4.2b"><apply id="S2.SS1.p2.4.m4.2.2.2.cmml" xref="S2.SS1.p2.4.m4.2.2.1"><log id="S2.SS1.p2.4.m4.1.1.cmml" xref="S2.SS1.p2.4.m4.1.1"></log><apply id="S2.SS1.p2.4.m4.2.2.1.1.1.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1"><plus id="S2.SS1.p2.4.m4.2.2.1.1.1.1.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.1"></plus><apply id="S2.SS1.p2.4.m4.2.2.1.1.1.2.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2"><times id="S2.SS1.p2.4.m4.2.2.1.1.1.2.1.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.1"></times><ci id="S2.SS1.p2.4.m4.2.2.1.1.1.2.2.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.2">𝑎</ci><apply id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3"><csymbol cd="ambiguous" id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.1.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3">subscript</csymbol><ci id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.2.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.2">𝑥</ci><ci id="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.3.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.2.3.3">𝑖</ci></apply></apply><ci id="S2.SS1.p2.4.m4.2.2.1.1.1.3.cmml" xref="S2.SS1.p2.4.m4.2.2.1.1.1.3">𝑏</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.4.m4.2c">\log(ax_{i}+b)</annotation></semantics></math> as base function as we can rewrite it to <math id="S2.SS1.p2.5.m5.3" class="ltx_Math" alttext="log(c)\log(x_{i}+d)" display="inline"><semantics id="S2.SS1.p2.5.m5.3a"><mrow id="S2.SS1.p2.5.m5.3.3" xref="S2.SS1.p2.5.m5.3.3.cmml"><mi id="S2.SS1.p2.5.m5.3.3.3" xref="S2.SS1.p2.5.m5.3.3.3.cmml">l</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.5.m5.3.3.2" xref="S2.SS1.p2.5.m5.3.3.2.cmml">​</mo><mi id="S2.SS1.p2.5.m5.3.3.4" xref="S2.SS1.p2.5.m5.3.3.4.cmml">o</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.5.m5.3.3.2a" xref="S2.SS1.p2.5.m5.3.3.2.cmml">​</mo><mi id="S2.SS1.p2.5.m5.3.3.5" xref="S2.SS1.p2.5.m5.3.3.5.cmml">g</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.5.m5.3.3.2b" xref="S2.SS1.p2.5.m5.3.3.2.cmml">​</mo><mrow id="S2.SS1.p2.5.m5.3.3.6.2" xref="S2.SS1.p2.5.m5.3.3.cmml"><mo stretchy="false" id="S2.SS1.p2.5.m5.3.3.6.2.1" xref="S2.SS1.p2.5.m5.3.3.cmml">(</mo><mi id="S2.SS1.p2.5.m5.1.1" xref="S2.SS1.p2.5.m5.1.1.cmml">c</mi><mo stretchy="false" id="S2.SS1.p2.5.m5.3.3.6.2.2" xref="S2.SS1.p2.5.m5.3.3.cmml">)</mo></mrow><mo lspace="0.167em" rspace="0em" id="S2.SS1.p2.5.m5.3.3.2c" xref="S2.SS1.p2.5.m5.3.3.2.cmml">​</mo><mrow id="S2.SS1.p2.5.m5.3.3.1.1" xref="S2.SS1.p2.5.m5.3.3.1.2.cmml"><mi id="S2.SS1.p2.5.m5.2.2" xref="S2.SS1.p2.5.m5.2.2.cmml">log</mi><mo id="S2.SS1.p2.5.m5.3.3.1.1a" xref="S2.SS1.p2.5.m5.3.3.1.2.cmml">⁡</mo><mrow id="S2.SS1.p2.5.m5.3.3.1.1.1" xref="S2.SS1.p2.5.m5.3.3.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.5.m5.3.3.1.1.1.2" xref="S2.SS1.p2.5.m5.3.3.1.2.cmml">(</mo><mrow id="S2.SS1.p2.5.m5.3.3.1.1.1.1" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.cmml"><msub id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.cmml"><mi id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.2" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.2.cmml">x</mi><mi id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.3" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.3.cmml">i</mi></msub><mo id="S2.SS1.p2.5.m5.3.3.1.1.1.1.1" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.1.cmml">+</mo><mi id="S2.SS1.p2.5.m5.3.3.1.1.1.1.3" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.3.cmml">d</mi></mrow><mo stretchy="false" id="S2.SS1.p2.5.m5.3.3.1.1.1.3" xref="S2.SS1.p2.5.m5.3.3.1.2.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.5.m5.3b"><apply id="S2.SS1.p2.5.m5.3.3.cmml" xref="S2.SS1.p2.5.m5.3.3"><times id="S2.SS1.p2.5.m5.3.3.2.cmml" xref="S2.SS1.p2.5.m5.3.3.2"></times><ci id="S2.SS1.p2.5.m5.3.3.3.cmml" xref="S2.SS1.p2.5.m5.3.3.3">𝑙</ci><ci id="S2.SS1.p2.5.m5.3.3.4.cmml" xref="S2.SS1.p2.5.m5.3.3.4">𝑜</ci><ci id="S2.SS1.p2.5.m5.3.3.5.cmml" xref="S2.SS1.p2.5.m5.3.3.5">𝑔</ci><ci id="S2.SS1.p2.5.m5.1.1.cmml" xref="S2.SS1.p2.5.m5.1.1">𝑐</ci><apply id="S2.SS1.p2.5.m5.3.3.1.2.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1"><log id="S2.SS1.p2.5.m5.2.2.cmml" xref="S2.SS1.p2.5.m5.2.2"></log><apply id="S2.SS1.p2.5.m5.3.3.1.1.1.1.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1"><plus id="S2.SS1.p2.5.m5.3.3.1.1.1.1.1.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.1"></plus><apply id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.1.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2">subscript</csymbol><ci id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.2.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.2">𝑥</ci><ci id="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.3.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.2.3">𝑖</ci></apply><ci id="S2.SS1.p2.5.m5.3.3.1.1.1.1.3.cmml" xref="S2.SS1.p2.5.m5.3.3.1.1.1.1.3">𝑑</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.5.m5.3c">log(c)\log(x_{i}+d)</annotation></semantics></math>. Then we can skip <math id="S2.SS1.p2.6.m6.1" class="ltx_Math" alttext="log(a)" display="inline"><semantics id="S2.SS1.p2.6.m6.1a"><mrow id="S2.SS1.p2.6.m6.1.2" xref="S2.SS1.p2.6.m6.1.2.cmml"><mi id="S2.SS1.p2.6.m6.1.2.2" xref="S2.SS1.p2.6.m6.1.2.2.cmml">l</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.6.m6.1.2.1" xref="S2.SS1.p2.6.m6.1.2.1.cmml">​</mo><mi id="S2.SS1.p2.6.m6.1.2.3" xref="S2.SS1.p2.6.m6.1.2.3.cmml">o</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.6.m6.1.2.1a" xref="S2.SS1.p2.6.m6.1.2.1.cmml">​</mo><mi id="S2.SS1.p2.6.m6.1.2.4" xref="S2.SS1.p2.6.m6.1.2.4.cmml">g</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p2.6.m6.1.2.1b" xref="S2.SS1.p2.6.m6.1.2.1.cmml">​</mo><mrow id="S2.SS1.p2.6.m6.1.2.5.2" xref="S2.SS1.p2.6.m6.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.6.m6.1.2.5.2.1" xref="S2.SS1.p2.6.m6.1.2.cmml">(</mo><mi id="S2.SS1.p2.6.m6.1.1" xref="S2.SS1.p2.6.m6.1.1.cmml">a</mi><mo stretchy="false" id="S2.SS1.p2.6.m6.1.2.5.2.2" xref="S2.SS1.p2.6.m6.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.6.m6.1b"><apply id="S2.SS1.p2.6.m6.1.2.cmml" xref="S2.SS1.p2.6.m6.1.2"><times id="S2.SS1.p2.6.m6.1.2.1.cmml" xref="S2.SS1.p2.6.m6.1.2.1"></times><ci id="S2.SS1.p2.6.m6.1.2.2.cmml" xref="S2.SS1.p2.6.m6.1.2.2">𝑙</ci><ci id="S2.SS1.p2.6.m6.1.2.3.cmml" xref="S2.SS1.p2.6.m6.1.2.3">𝑜</ci><ci id="S2.SS1.p2.6.m6.1.2.4.cmml" xref="S2.SS1.p2.6.m6.1.2.4">𝑔</ci><ci id="S2.SS1.p2.6.m6.1.1.cmml" xref="S2.SS1.p2.6.m6.1.1">𝑎</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.6.m6.1c">log(a)</annotation></semantics></math> as it is constant in the final model and therefore summed up by the final model’s intercept. The same applies to the exp-function, in which we can skip the multiplicative scaling value in the argument as we can rewrite <math id="S2.SS1.p2.7.m7.6" class="ltx_Math" alttext="\exp(x+a)=\exp(x)\exp(a)" display="inline"><semantics id="S2.SS1.p2.7.m7.6a"><mrow id="S2.SS1.p2.7.m7.6.6" xref="S2.SS1.p2.7.m7.6.6.cmml"><mrow id="S2.SS1.p2.7.m7.6.6.1.1" xref="S2.SS1.p2.7.m7.6.6.1.2.cmml"><mi id="S2.SS1.p2.7.m7.1.1" xref="S2.SS1.p2.7.m7.1.1.cmml">exp</mi><mo id="S2.SS1.p2.7.m7.6.6.1.1a" xref="S2.SS1.p2.7.m7.6.6.1.2.cmml">⁡</mo><mrow id="S2.SS1.p2.7.m7.6.6.1.1.1" xref="S2.SS1.p2.7.m7.6.6.1.2.cmml"><mo stretchy="false" id="S2.SS1.p2.7.m7.6.6.1.1.1.2" xref="S2.SS1.p2.7.m7.6.6.1.2.cmml">(</mo><mrow id="S2.SS1.p2.7.m7.6.6.1.1.1.1" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.cmml"><mi id="S2.SS1.p2.7.m7.6.6.1.1.1.1.2" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.2.cmml">x</mi><mo id="S2.SS1.p2.7.m7.6.6.1.1.1.1.1" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.1.cmml">+</mo><mi id="S2.SS1.p2.7.m7.6.6.1.1.1.1.3" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.3.cmml">a</mi></mrow><mo stretchy="false" id="S2.SS1.p2.7.m7.6.6.1.1.1.3" xref="S2.SS1.p2.7.m7.6.6.1.2.cmml">)</mo></mrow></mrow><mo id="S2.SS1.p2.7.m7.6.6.2" xref="S2.SS1.p2.7.m7.6.6.2.cmml">=</mo><mrow id="S2.SS1.p2.7.m7.6.6.3" xref="S2.SS1.p2.7.m7.6.6.3.cmml"><mrow id="S2.SS1.p2.7.m7.6.6.3.2.2" xref="S2.SS1.p2.7.m7.6.6.3.2.1.cmml"><mi id="S2.SS1.p2.7.m7.2.2" xref="S2.SS1.p2.7.m7.2.2.cmml">exp</mi><mo id="S2.SS1.p2.7.m7.6.6.3.2.2a" xref="S2.SS1.p2.7.m7.6.6.3.2.1.cmml">⁡</mo><mrow id="S2.SS1.p2.7.m7.6.6.3.2.2.1" xref="S2.SS1.p2.7.m7.6.6.3.2.1.cmml"><mo stretchy="false" id="S2.SS1.p2.7.m7.6.6.3.2.2.1.1" xref="S2.SS1.p2.7.m7.6.6.3.2.1.cmml">(</mo><mi id="S2.SS1.p2.7.m7.3.3" xref="S2.SS1.p2.7.m7.3.3.cmml">x</mi><mo stretchy="false" id="S2.SS1.p2.7.m7.6.6.3.2.2.1.2" xref="S2.SS1.p2.7.m7.6.6.3.2.1.cmml">)</mo></mrow></mrow><mo lspace="0.167em" rspace="0em" id="S2.SS1.p2.7.m7.6.6.3.1" xref="S2.SS1.p2.7.m7.6.6.3.1.cmml">​</mo><mrow id="S2.SS1.p2.7.m7.6.6.3.3.2" xref="S2.SS1.p2.7.m7.6.6.3.3.1.cmml"><mi id="S2.SS1.p2.7.m7.4.4" xref="S2.SS1.p2.7.m7.4.4.cmml">exp</mi><mo id="S2.SS1.p2.7.m7.6.6.3.3.2a" xref="S2.SS1.p2.7.m7.6.6.3.3.1.cmml">⁡</mo><mrow id="S2.SS1.p2.7.m7.6.6.3.3.2.1" xref="S2.SS1.p2.7.m7.6.6.3.3.1.cmml"><mo stretchy="false" id="S2.SS1.p2.7.m7.6.6.3.3.2.1.1" xref="S2.SS1.p2.7.m7.6.6.3.3.1.cmml">(</mo><mi id="S2.SS1.p2.7.m7.5.5" xref="S2.SS1.p2.7.m7.5.5.cmml">a</mi><mo stretchy="false" id="S2.SS1.p2.7.m7.6.6.3.3.2.1.2" xref="S2.SS1.p2.7.m7.6.6.3.3.1.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.7.m7.6b"><apply id="S2.SS1.p2.7.m7.6.6.cmml" xref="S2.SS1.p2.7.m7.6.6"><eq id="S2.SS1.p2.7.m7.6.6.2.cmml" xref="S2.SS1.p2.7.m7.6.6.2"></eq><apply id="S2.SS1.p2.7.m7.6.6.1.2.cmml" xref="S2.SS1.p2.7.m7.6.6.1.1"><exp id="S2.SS1.p2.7.m7.1.1.cmml" xref="S2.SS1.p2.7.m7.1.1"></exp><apply id="S2.SS1.p2.7.m7.6.6.1.1.1.1.cmml" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1"><plus id="S2.SS1.p2.7.m7.6.6.1.1.1.1.1.cmml" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.1"></plus><ci id="S2.SS1.p2.7.m7.6.6.1.1.1.1.2.cmml" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.2">𝑥</ci><ci id="S2.SS1.p2.7.m7.6.6.1.1.1.1.3.cmml" xref="S2.SS1.p2.7.m7.6.6.1.1.1.1.3">𝑎</ci></apply></apply><apply id="S2.SS1.p2.7.m7.6.6.3.cmml" xref="S2.SS1.p2.7.m7.6.6.3"><times id="S2.SS1.p2.7.m7.6.6.3.1.cmml" xref="S2.SS1.p2.7.m7.6.6.3.1"></times><apply id="S2.SS1.p2.7.m7.6.6.3.2.1.cmml" xref="S2.SS1.p2.7.m7.6.6.3.2.2"><exp id="S2.SS1.p2.7.m7.2.2.cmml" xref="S2.SS1.p2.7.m7.2.2"></exp><ci id="S2.SS1.p2.7.m7.3.3.cmml" xref="S2.SS1.p2.7.m7.3.3">𝑥</ci></apply><apply id="S2.SS1.p2.7.m7.6.6.3.3.1.cmml" xref="S2.SS1.p2.7.m7.6.6.3.3.2"><exp id="S2.SS1.p2.7.m7.4.4.cmml" xref="S2.SS1.p2.7.m7.4.4"></exp><ci id="S2.SS1.p2.7.m7.5.5.cmml" xref="S2.SS1.p2.7.m7.5.5">𝑎</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.7.m7.6c">\exp(x+a)=\exp(x)\exp(a)</annotation></semantics></math> and skip <math id="S2.SS1.p2.8.m8.2" class="ltx_Math" alttext="\exp(a)" display="inline"><semantics id="S2.SS1.p2.8.m8.2a"><mrow id="S2.SS1.p2.8.m8.2.3.2" xref="S2.SS1.p2.8.m8.2.3.1.cmml"><mi id="S2.SS1.p2.8.m8.1.1" xref="S2.SS1.p2.8.m8.1.1.cmml">exp</mi><mo id="S2.SS1.p2.8.m8.2.3.2a" xref="S2.SS1.p2.8.m8.2.3.1.cmml">⁡</mo><mrow id="S2.SS1.p2.8.m8.2.3.2.1" xref="S2.SS1.p2.8.m8.2.3.1.cmml"><mo stretchy="false" id="S2.SS1.p2.8.m8.2.3.2.1.1" xref="S2.SS1.p2.8.m8.2.3.1.cmml">(</mo><mi id="S2.SS1.p2.8.m8.2.2" xref="S2.SS1.p2.8.m8.2.2.cmml">a</mi><mo stretchy="false" id="S2.SS1.p2.8.m8.2.3.2.1.2" xref="S2.SS1.p2.8.m8.2.3.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p2.8.m8.2b"><apply id="S2.SS1.p2.8.m8.2.3.1.cmml" xref="S2.SS1.p2.8.m8.2.3.2"><exp id="S2.SS1.p2.8.m8.1.1.cmml" xref="S2.SS1.p2.8.m8.1.1"></exp><ci id="S2.SS1.p2.8.m8.2.2.cmml" xref="S2.SS1.p2.8.m8.2.2">𝑎</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p2.8.m8.2c">\exp(a)</annotation></semantics></math> in the final model.</p>
</div>
</section>
<section id="S2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.2 </span>Parameter Optimization with Variable Projection</h3>

<div id="S2.SS2.p1" class="ltx_para">
<p id="S2.SS2.p1.6" class="ltx_p">The generated base functions <math id="S2.SS2.p1.1.m1.1" class="ltx_Math" alttext="\bm{f}" display="inline"><semantics id="S2.SS2.p1.1.m1.1a"><mi id="S2.SS2.p1.1.m1.1.1" xref="S2.SS2.p1.1.m1.1.1.cmml">𝒇</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.1.m1.1b"><ci id="S2.SS2.p1.1.m1.1.1.cmml" xref="S2.SS2.p1.1.m1.1.1">𝒇</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.1.m1.1c">\bm{f}</annotation></semantics></math> are combined to one large linear model <math id="S2.SS2.p1.2.m2.3" class="ltx_Math" alttext="\hat{\Theta}(\bm{x})=l_{0}+\sum_{i}{l_{i}f_{i}(\bm{k},\bm{x})}" display="inline"><semantics id="S2.SS2.p1.2.m2.3a"><mrow id="S2.SS2.p1.2.m2.3.4" xref="S2.SS2.p1.2.m2.3.4.cmml"><mrow id="S2.SS2.p1.2.m2.3.4.2" xref="S2.SS2.p1.2.m2.3.4.2.cmml"><mover accent="true" id="S2.SS2.p1.2.m2.3.4.2.2" xref="S2.SS2.p1.2.m2.3.4.2.2.cmml"><mi mathvariant="normal" id="S2.SS2.p1.2.m2.3.4.2.2.2" xref="S2.SS2.p1.2.m2.3.4.2.2.2.cmml">Θ</mi><mo id="S2.SS2.p1.2.m2.3.4.2.2.1" xref="S2.SS2.p1.2.m2.3.4.2.2.1.cmml">^</mo></mover><mo lspace="0em" rspace="0em" id="S2.SS2.p1.2.m2.3.4.2.1" xref="S2.SS2.p1.2.m2.3.4.2.1.cmml">​</mo><mrow id="S2.SS2.p1.2.m2.3.4.2.3.2" xref="S2.SS2.p1.2.m2.3.4.2.cmml"><mo stretchy="false" id="S2.SS2.p1.2.m2.3.4.2.3.2.1" xref="S2.SS2.p1.2.m2.3.4.2.cmml">(</mo><mi id="S2.SS2.p1.2.m2.1.1" xref="S2.SS2.p1.2.m2.1.1.cmml">𝒙</mi><mo stretchy="false" id="S2.SS2.p1.2.m2.3.4.2.3.2.2" xref="S2.SS2.p1.2.m2.3.4.2.cmml">)</mo></mrow></mrow><mo id="S2.SS2.p1.2.m2.3.4.1" xref="S2.SS2.p1.2.m2.3.4.1.cmml">=</mo><mrow id="S2.SS2.p1.2.m2.3.4.3" xref="S2.SS2.p1.2.m2.3.4.3.cmml"><msub id="S2.SS2.p1.2.m2.3.4.3.2" xref="S2.SS2.p1.2.m2.3.4.3.2.cmml"><mi id="S2.SS2.p1.2.m2.3.4.3.2.2" xref="S2.SS2.p1.2.m2.3.4.3.2.2.cmml">l</mi><mn id="S2.SS2.p1.2.m2.3.4.3.2.3" xref="S2.SS2.p1.2.m2.3.4.3.2.3.cmml">0</mn></msub><mo rspace="0.055em" id="S2.SS2.p1.2.m2.3.4.3.1" xref="S2.SS2.p1.2.m2.3.4.3.1.cmml">+</mo><mrow id="S2.SS2.p1.2.m2.3.4.3.3" xref="S2.SS2.p1.2.m2.3.4.3.3.cmml"><msub id="S2.SS2.p1.2.m2.3.4.3.3.1" xref="S2.SS2.p1.2.m2.3.4.3.3.1.cmml"><mo id="S2.SS2.p1.2.m2.3.4.3.3.1.2" xref="S2.SS2.p1.2.m2.3.4.3.3.1.2.cmml">∑</mo><mi id="S2.SS2.p1.2.m2.3.4.3.3.1.3" xref="S2.SS2.p1.2.m2.3.4.3.3.1.3.cmml">i</mi></msub><mrow id="S2.SS2.p1.2.m2.3.4.3.3.2" xref="S2.SS2.p1.2.m2.3.4.3.3.2.cmml"><msub id="S2.SS2.p1.2.m2.3.4.3.3.2.2" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2.cmml"><mi id="S2.SS2.p1.2.m2.3.4.3.3.2.2.2" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2.2.cmml">l</mi><mi id="S2.SS2.p1.2.m2.3.4.3.3.2.2.3" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.SS2.p1.2.m2.3.4.3.3.2.1" xref="S2.SS2.p1.2.m2.3.4.3.3.2.1.cmml">​</mo><msub id="S2.SS2.p1.2.m2.3.4.3.3.2.3" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3.cmml"><mi id="S2.SS2.p1.2.m2.3.4.3.3.2.3.2" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3.2.cmml">f</mi><mi id="S2.SS2.p1.2.m2.3.4.3.3.2.3.3" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.SS2.p1.2.m2.3.4.3.3.2.1a" xref="S2.SS2.p1.2.m2.3.4.3.3.2.1.cmml">​</mo><mrow id="S2.SS2.p1.2.m2.3.4.3.3.2.4.2" xref="S2.SS2.p1.2.m2.3.4.3.3.2.4.1.cmml"><mo stretchy="false" id="S2.SS2.p1.2.m2.3.4.3.3.2.4.2.1" xref="S2.SS2.p1.2.m2.3.4.3.3.2.4.1.cmml">(</mo><mi id="S2.SS2.p1.2.m2.2.2" xref="S2.SS2.p1.2.m2.2.2.cmml">𝒌</mi><mo id="S2.SS2.p1.2.m2.3.4.3.3.2.4.2.2" xref="S2.SS2.p1.2.m2.3.4.3.3.2.4.1.cmml">,</mo><mi id="S2.SS2.p1.2.m2.3.3" xref="S2.SS2.p1.2.m2.3.3.cmml">𝒙</mi><mo stretchy="false" id="S2.SS2.p1.2.m2.3.4.3.3.2.4.2.3" xref="S2.SS2.p1.2.m2.3.4.3.3.2.4.1.cmml">)</mo></mrow></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.2.m2.3b"><apply id="S2.SS2.p1.2.m2.3.4.cmml" xref="S2.SS2.p1.2.m2.3.4"><eq id="S2.SS2.p1.2.m2.3.4.1.cmml" xref="S2.SS2.p1.2.m2.3.4.1"></eq><apply id="S2.SS2.p1.2.m2.3.4.2.cmml" xref="S2.SS2.p1.2.m2.3.4.2"><times id="S2.SS2.p1.2.m2.3.4.2.1.cmml" xref="S2.SS2.p1.2.m2.3.4.2.1"></times><apply id="S2.SS2.p1.2.m2.3.4.2.2.cmml" xref="S2.SS2.p1.2.m2.3.4.2.2"><ci id="S2.SS2.p1.2.m2.3.4.2.2.1.cmml" xref="S2.SS2.p1.2.m2.3.4.2.2.1">^</ci><ci id="S2.SS2.p1.2.m2.3.4.2.2.2.cmml" xref="S2.SS2.p1.2.m2.3.4.2.2.2">Θ</ci></apply><ci id="S2.SS2.p1.2.m2.1.1.cmml" xref="S2.SS2.p1.2.m2.1.1">𝒙</ci></apply><apply id="S2.SS2.p1.2.m2.3.4.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3"><plus id="S2.SS2.p1.2.m2.3.4.3.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.1"></plus><apply id="S2.SS2.p1.2.m2.3.4.3.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.2"><csymbol cd="ambiguous" id="S2.SS2.p1.2.m2.3.4.3.2.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.2">subscript</csymbol><ci id="S2.SS2.p1.2.m2.3.4.3.2.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.2.2">𝑙</ci><cn type="integer" id="S2.SS2.p1.2.m2.3.4.3.2.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3.2.3">0</cn></apply><apply id="S2.SS2.p1.2.m2.3.4.3.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3"><apply id="S2.SS2.p1.2.m2.3.4.3.3.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.1"><csymbol cd="ambiguous" id="S2.SS2.p1.2.m2.3.4.3.3.1.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.1">subscript</csymbol><sum id="S2.SS2.p1.2.m2.3.4.3.3.1.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.1.2"></sum><ci id="S2.SS2.p1.2.m2.3.4.3.3.1.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.1.3">𝑖</ci></apply><apply id="S2.SS2.p1.2.m2.3.4.3.3.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2"><times id="S2.SS2.p1.2.m2.3.4.3.3.2.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.1"></times><apply id="S2.SS2.p1.2.m2.3.4.3.3.2.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2"><csymbol cd="ambiguous" id="S2.SS2.p1.2.m2.3.4.3.3.2.2.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2">subscript</csymbol><ci id="S2.SS2.p1.2.m2.3.4.3.3.2.2.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2.2">𝑙</ci><ci id="S2.SS2.p1.2.m2.3.4.3.3.2.2.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.2.3">𝑖</ci></apply><apply id="S2.SS2.p1.2.m2.3.4.3.3.2.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3"><csymbol cd="ambiguous" id="S2.SS2.p1.2.m2.3.4.3.3.2.3.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3">subscript</csymbol><ci id="S2.SS2.p1.2.m2.3.4.3.3.2.3.2.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3.2">𝑓</ci><ci id="S2.SS2.p1.2.m2.3.4.3.3.2.3.3.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.3.3">𝑖</ci></apply><interval closure="open" id="S2.SS2.p1.2.m2.3.4.3.3.2.4.1.cmml" xref="S2.SS2.p1.2.m2.3.4.3.3.2.4.2"><ci id="S2.SS2.p1.2.m2.2.2.cmml" xref="S2.SS2.p1.2.m2.2.2">𝒌</ci><ci id="S2.SS2.p1.2.m2.3.3.cmml" xref="S2.SS2.p1.2.m2.3.3">𝒙</ci></interval></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.2.m2.3c">\hat{\Theta}(\bm{x})=l_{0}+\sum_{i}{l_{i}f_{i}(\bm{k},\bm{x})}</annotation></semantics></math> with <math id="S2.SS2.p1.3.m3.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.SS2.p1.3.m3.1a"><mi id="S2.SS2.p1.3.m3.1.1" xref="S2.SS2.p1.3.m3.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.3.m3.1b"><ci id="S2.SS2.p1.3.m3.1.1.cmml" xref="S2.SS2.p1.3.m3.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.3.m3.1c">\bm{l}</annotation></semantics></math> as vector of linear parameters and intercept and <math id="S2.SS2.p1.4.m4.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.SS2.p1.4.m4.1a"><mi id="S2.SS2.p1.4.m4.1.1" xref="S2.SS2.p1.4.m4.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.4.m4.1b"><ci id="S2.SS2.p1.4.m4.1.1.cmml" xref="S2.SS2.p1.4.m4.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.4.m4.1c">\bm{k}</annotation></semantics></math> as vector of nonlinear parameters. We use NLS to find the values in <math id="S2.SS2.p1.5.m5.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.SS2.p1.5.m5.1a"><mi id="S2.SS2.p1.5.m5.1.1" xref="S2.SS2.p1.5.m5.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.5.m5.1b"><ci id="S2.SS2.p1.5.m5.1.1.cmml" xref="S2.SS2.p1.5.m5.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.5.m5.1c">\bm{l}</annotation></semantics></math> and <math id="S2.SS2.p1.6.m6.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.SS2.p1.6.m6.1a"><mi id="S2.SS2.p1.6.m6.1.1" xref="S2.SS2.p1.6.m6.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.6.m6.1b"><ci id="S2.SS2.p1.6.m6.1.1.cmml" xref="S2.SS2.p1.6.m6.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.6.m6.1c">\bm{k}</annotation></semantics></math> that minimize the mean squared error (MSE) for given training data.</p>
</div>
<div id="S2.SS2.p2" class="ltx_para">
<p id="S2.SS2.p2.4" class="ltx_p">Since nonlinear least squares optimization is computationally expensive, we use a variable projection (VP) algorithm initially developed by Golub and Pereya <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib5" title="" class="ltx_ref">5</a>]</cite> for optimizing all parameters <math id="S2.SS2.p2.1.m1.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.SS2.p2.1.m1.1a"><mi id="S2.SS2.p2.1.m1.1.1" xref="S2.SS2.p2.1.m1.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.1.m1.1b"><ci id="S2.SS2.p2.1.m1.1.1.cmml" xref="S2.SS2.p2.1.m1.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.1.m1.1c">\bm{l}</annotation></semantics></math> and <math id="S2.SS2.p2.2.m2.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.SS2.p2.2.m2.1a"><mi id="S2.SS2.p2.2.m2.1.1" xref="S2.SS2.p2.2.m2.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.2.m2.1b"><ci id="S2.SS2.p2.2.m2.1.1.cmml" xref="S2.SS2.p2.2.m2.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.2.m2.1c">\bm{k}</annotation></semantics></math>. The advantage of VP over plain NLS optimization is that VP utilizes the generated model’s structure – a nonlinear model with several linear parameters <math id="S2.SS2.p2.3.m3.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.SS2.p2.3.m3.1a"><mi id="S2.SS2.p2.3.m3.1.1" xref="S2.SS2.p2.3.m3.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.3.m3.1b"><ci id="S2.SS2.p2.3.m3.1.1.cmml" xref="S2.SS2.p2.3.m3.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.3.m3.1c">\bm{l}</annotation></semantics></math>. Golub and Pereya call the given model structure a <span id="S2.SS2.p2.4.1" class="ltx_text ltx_font_italic">separable least squares problem</span>. This setting is common in engineering domains. VP optimizes nonlinear parameters in <math id="S2.SS2.p2.4.m4.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.SS2.p2.4.m4.1a"><mi id="S2.SS2.p2.4.m4.1.1" xref="S2.SS2.p2.4.m4.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.4.m4.1b"><ci id="S2.SS2.p2.4.m4.1.1.cmml" xref="S2.SS2.p2.4.m4.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.4.m4.1c">\bm{k}</annotation></semantics></math> iteratively, while optimal linear parameters are solved exactly via ordinary least squares. Given the high number of linear parameters in the model (due to the large number of base functions), the use of VP is an important performance aspect of FFX NLS. We use the effieicennt VP algorithm by Krogh <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib8" title="" class="ltx_ref">8</a>]</cite>.</p>
</div>
</section>
<section id="S2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.3 </span>Base Function Selection</h3>

<div id="S2.SS3.p1" class="ltx_para">
<p id="S2.SS3.p1.1" class="ltx_p">As we generate a very large number of base functions, we need to select the most relevant ones in order to create a both interpretable and well-generalizing model. The number of selected base functions is thereby a hyperparameter of FFX NLS. In plain FFX, the selection of relevant base functions and the optimization of parameters is done in one step with a ElasticNet regression <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib4" title="" class="ltx_ref">4</a>]</cite>, as only linear parameters need to be optimized.</p>
</div>
<div id="S2.SS3.p2" class="ltx_para">
<p id="S2.SS3.p2.4" class="ltx_p">Since we also need to optimize nonlinear parameters, no simple way to combine NLS optimization and regularization of linear parameters is available. The VP algorithm by Chen et al. <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib2" title="" class="ltx_ref">2</a>]</cite> uses Tikhonov regularization <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib4" title="" class="ltx_ref">4</a>]</cite>. However, in initial experiments this algorithm was not beneficial for FFX NLS to identify relevant terms. Tikhonov regularization shrinks linear parameters, however, it did not provide the necessary sparsification of linear parameters and the algorithm in <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib2" title="" class="ltx_ref">2</a>]</cite> is computationally more expensive. Alternatively, we use the already computed vector of nonlinear parameters <math id="S2.SS3.p2.1.m1.1" class="ltx_Math" alttext="\bm{k}" display="inline"><semantics id="S2.SS3.p2.1.m1.1a"><mi id="S2.SS3.p2.1.m1.1.1" xref="S2.SS3.p2.1.m1.1.1.cmml">𝒌</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.1.m1.1b"><ci id="S2.SS3.p2.1.m1.1.1.cmml" xref="S2.SS3.p2.1.m1.1.1">𝒌</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.1.m1.1c">\bm{k}</annotation></semantics></math> of the previous step as fixed constants and optimize the linear parameters <math id="S2.SS3.p2.2.m2.1" class="ltx_Math" alttext="\bm{l}" display="inline"><semantics id="S2.SS3.p2.2.m2.1a"><mi id="S2.SS3.p2.2.m2.1.1" xref="S2.SS3.p2.2.m2.1.1.cmml">𝒍</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.2.m2.1b"><ci id="S2.SS3.p2.2.m2.1.1.cmml" xref="S2.SS3.p2.2.m2.1.1">𝒍</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.2.m2.1c">\bm{l}</annotation></semantics></math> with a lasso regression <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib4" title="" class="ltx_ref">4</a>]</cite>. Terms with a linear parameter <math id="S2.SS3.p2.3.m3.1" class="ltx_Math" alttext="\neq 0" display="inline"><semantics id="S2.SS3.p2.3.m3.1a"><mrow id="S2.SS3.p2.3.m3.1.1" xref="S2.SS3.p2.3.m3.1.1.cmml"><mi id="S2.SS3.p2.3.m3.1.1.2" xref="S2.SS3.p2.3.m3.1.1.2.cmml"></mi><mo id="S2.SS3.p2.3.m3.1.1.1" xref="S2.SS3.p2.3.m3.1.1.1.cmml">≠</mo><mn id="S2.SS3.p2.3.m3.1.1.3" xref="S2.SS3.p2.3.m3.1.1.3.cmml">0</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.3.m3.1b"><apply id="S2.SS3.p2.3.m3.1.1.cmml" xref="S2.SS3.p2.3.m3.1.1"><neq id="S2.SS3.p2.3.m3.1.1.1.cmml" xref="S2.SS3.p2.3.m3.1.1.1"></neq><csymbol cd="latexml" id="S2.SS3.p2.3.m3.1.1.2.cmml" xref="S2.SS3.p2.3.m3.1.1.2">absent</csymbol><cn type="integer" id="S2.SS3.p2.3.m3.1.1.3.cmml" xref="S2.SS3.p2.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.3.m3.1c">\neq 0</annotation></semantics></math> are selected as most important base functions. We get the desired number of base functions by iteratively increasing the lasso regression’s <math id="S2.SS3.p2.4.m4.1" class="ltx_Math" alttext="\lambda" display="inline"><semantics id="S2.SS3.p2.4.m4.1a"><mi id="S2.SS3.p2.4.m4.1.1" xref="S2.SS3.p2.4.m4.1.1.cmml">λ</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.4.m4.1b"><ci id="S2.SS3.p2.4.m4.1.1.cmml" xref="S2.SS3.p2.4.m4.1.1">𝜆</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.4.m4.1c">\lambda</annotation></semantics></math> value in order to shrink more parameters to zero. Although this method ignores dependencies between linear and nonlinear parameters, it is still effective for selecting base functions both regarding runtime and further modelling accuracy.</p>
</div>
</section>
<section id="S2.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.4 </span>Training of Final Model and Bivariate Base Functions</h3>

<div id="S2.SS4.p1" class="ltx_para">
<p id="S2.SS4.p1.1" class="ltx_p">Similar to the parameter optimization in Section <a href="#S2.SS2" title="2.2 Parameter Optimization with Variable Projection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a>, we combine base functions to one single model and optimize all parameters again. In difference to the previous NLS optimization step, we take in this step only the most relevant base functions from Section <a href="#S2.SS3" title="2.3 Base Function Selection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.3</span></a>. As parameters are not independent from each other, we repeat this step to get optimal parameters for a subset of base functions.</p>
</div>
<div id="S2.SS4.p2" class="ltx_para">
<p id="S2.SS4.p2.1" class="ltx_p">To cover interactions between features, we also include bivariate base functions. For this we take pairwise combinations of the previously generated univariate base function and multiply them to get one new base function for each pair. To prevent a combinatorial explosion of base functions and nonlinear parameters in highly multidimensional problems, we only consider the ten most important univariate base functions from Section <a href="#S2.SS3" title="2.3 Base Function Selection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.3</span></a> for to create bivariate base functions. We take the resulting <math id="S2.SS4.p2.1.m1.3" class="ltx_Math" alttext="{10\choose 2}=45" display="inline"><semantics id="S2.SS4.p2.1.m1.3a"><mrow id="S2.SS4.p2.1.m1.3.4" xref="S2.SS4.p2.1.m1.3.4.cmml"><mrow id="S2.SS4.p2.1.m1.3.3.5" xref="S2.SS4.p2.1.m1.3.3.4.cmml"><mo id="S2.SS4.p2.1.m1.3.3.5.1" xref="S2.SS4.p2.1.m1.1.1.1.1.1.cmml">(</mo><mfrac linethickness="0pt" id="S2.SS4.p2.1.m1.3.3.3.3" xref="S2.SS4.p2.1.m1.3.3.4.cmml"><mn id="S2.SS4.p2.1.m1.2.2.2.2.2" xref="S2.SS4.p2.1.m1.2.2.2.2.2.cmml">10</mn><mn id="S2.SS4.p2.1.m1.3.3.3.3.3" xref="S2.SS4.p2.1.m1.3.3.3.3.3.cmml">2</mn></mfrac><mo id="S2.SS4.p2.1.m1.3.3.5.2" xref="S2.SS4.p2.1.m1.1.1.1.1.1.cmml">)</mo></mrow><mo id="S2.SS4.p2.1.m1.3.4.1" xref="S2.SS4.p2.1.m1.3.4.1.cmml">=</mo><mn id="S2.SS4.p2.1.m1.3.4.2" xref="S2.SS4.p2.1.m1.3.4.2.cmml">45</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS4.p2.1.m1.3b"><apply id="S2.SS4.p2.1.m1.3.4.cmml" xref="S2.SS4.p2.1.m1.3.4"><eq id="S2.SS4.p2.1.m1.3.4.1.cmml" xref="S2.SS4.p2.1.m1.3.4.1"></eq><apply id="S2.SS4.p2.1.m1.3.3.4.cmml" xref="S2.SS4.p2.1.m1.3.3.5"><csymbol cd="latexml" id="S2.SS4.p2.1.m1.1.1.1.1.1.cmml" xref="S2.SS4.p2.1.m1.3.3.5.1">binomial</csymbol><cn type="integer" id="S2.SS4.p2.1.m1.2.2.2.2.2.cmml" xref="S2.SS4.p2.1.m1.2.2.2.2.2">10</cn><cn type="integer" id="S2.SS4.p2.1.m1.3.3.3.3.3.cmml" xref="S2.SS4.p2.1.m1.3.3.3.3.3">2</cn></apply><cn type="integer" id="S2.SS4.p2.1.m1.3.4.2.cmml" xref="S2.SS4.p2.1.m1.3.4.2">45</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS4.p2.1.m1.3c">{10\choose 2}=45</annotation></semantics></math> bivariate functions and append them to our existing list of univariate base functions. Then we repeat the steps described in Section <a href="#S2.SS2" title="2.2 Parameter Optimization with Variable Projection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a> and <a href="#S2.SS3" title="2.3 Base Function Selection ‣ 2 Algorithm Description ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.3</span></a> to determine the most important base functions across both univariate and bivariate base function for the final model. We consider only ten univariate base functions as a larger number did not provide any benefit in achieving our objectives.</p>
</div>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Experimental Setup</h2>

<div id="S3.p1" class="ltx_para">
<p id="S3.p1.1" class="ltx_p">We use <span id="S3.p1.1.1" class="ltx_text ltx_font_italic">PennML</span> benchmark suite <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib10" title="" class="ltx_ref">10</a>]</cite> for all experiments. This benchmark consist of over 90 regression problem and provides a performance overview of several common regression algorithms. We apply both FFX and FFX NLS on these problems to compare the algorithms’ accuracies with each other and with other common regression algorithms. Thereby we take the experimental results from <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib7" title="" class="ltx_ref">7</a>]</cite> for a comprehensive comparison as this work shows the results of the currently most accurate regression algorithms like GP CoOp as implemented in <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib1" title="" class="ltx_ref">1</a>]</cite> for the PennML data.</p>
</div>
<div id="S3.p2" class="ltx_para">
<p id="S3.p2.1" class="ltx_p">We apply the same modelling workflow as in <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib10" title="" class="ltx_ref">10</a>]</cite> in our experiments. For every regression problem in the benchmark suite, we repeat the modelling ten times. We shuffle the dataset in every repetition and take the first 75% of observations as training set and the remaining ones as test set. We perform a grid search with a 5-fold cross validation for each shuffled dataset for hyperparameter tuning and train one final model with these best hyperparameters. Eventually, we get ten different models for each original regression problem. Hyperparameters for all other algorithms are described in <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib7" title="" class="ltx_ref">7</a>]</cite>. We use the following hyperparameter sets for both FFX and FFX NLS:</p>
</div>
<div id="S3.p3" class="ltx_para">
<ul id="S3.I1" class="ltx_itemize">
<li id="S3.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i1.p1" class="ltx_para">
<p id="S3.I1.i1.p1.1" class="ltx_p">Max. number of base function in final model <math id="S3.I1.i1.p1.1.m1.6" class="ltx_Math" alttext="\in\{3,5,10,20,30,50\}" display="inline"><semantics id="S3.I1.i1.p1.1.m1.6a"><mrow id="S3.I1.i1.p1.1.m1.6.7" xref="S3.I1.i1.p1.1.m1.6.7.cmml"><mi id="S3.I1.i1.p1.1.m1.6.7.2" xref="S3.I1.i1.p1.1.m1.6.7.2.cmml"></mi><mo id="S3.I1.i1.p1.1.m1.6.7.1" xref="S3.I1.i1.p1.1.m1.6.7.1.cmml">∈</mo><mrow id="S3.I1.i1.p1.1.m1.6.7.3.2" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml"><mo stretchy="false" id="S3.I1.i1.p1.1.m1.6.7.3.2.1" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">{</mo><mn id="S3.I1.i1.p1.1.m1.1.1" xref="S3.I1.i1.p1.1.m1.1.1.cmml">3</mn><mo id="S3.I1.i1.p1.1.m1.6.7.3.2.2" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">,</mo><mn id="S3.I1.i1.p1.1.m1.2.2" xref="S3.I1.i1.p1.1.m1.2.2.cmml">5</mn><mo id="S3.I1.i1.p1.1.m1.6.7.3.2.3" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">,</mo><mn id="S3.I1.i1.p1.1.m1.3.3" xref="S3.I1.i1.p1.1.m1.3.3.cmml">10</mn><mo id="S3.I1.i1.p1.1.m1.6.7.3.2.4" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">,</mo><mn id="S3.I1.i1.p1.1.m1.4.4" xref="S3.I1.i1.p1.1.m1.4.4.cmml">20</mn><mo id="S3.I1.i1.p1.1.m1.6.7.3.2.5" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">,</mo><mn id="S3.I1.i1.p1.1.m1.5.5" xref="S3.I1.i1.p1.1.m1.5.5.cmml">30</mn><mo id="S3.I1.i1.p1.1.m1.6.7.3.2.6" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">,</mo><mn id="S3.I1.i1.p1.1.m1.6.6" xref="S3.I1.i1.p1.1.m1.6.6.cmml">50</mn><mo stretchy="false" id="S3.I1.i1.p1.1.m1.6.7.3.2.7" xref="S3.I1.i1.p1.1.m1.6.7.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.I1.i1.p1.1.m1.6b"><apply id="S3.I1.i1.p1.1.m1.6.7.cmml" xref="S3.I1.i1.p1.1.m1.6.7"><in id="S3.I1.i1.p1.1.m1.6.7.1.cmml" xref="S3.I1.i1.p1.1.m1.6.7.1"></in><csymbol cd="latexml" id="S3.I1.i1.p1.1.m1.6.7.2.cmml" xref="S3.I1.i1.p1.1.m1.6.7.2">absent</csymbol><set id="S3.I1.i1.p1.1.m1.6.7.3.1.cmml" xref="S3.I1.i1.p1.1.m1.6.7.3.2"><cn type="integer" id="S3.I1.i1.p1.1.m1.1.1.cmml" xref="S3.I1.i1.p1.1.m1.1.1">3</cn><cn type="integer" id="S3.I1.i1.p1.1.m1.2.2.cmml" xref="S3.I1.i1.p1.1.m1.2.2">5</cn><cn type="integer" id="S3.I1.i1.p1.1.m1.3.3.cmml" xref="S3.I1.i1.p1.1.m1.3.3">10</cn><cn type="integer" id="S3.I1.i1.p1.1.m1.4.4.cmml" xref="S3.I1.i1.p1.1.m1.4.4">20</cn><cn type="integer" id="S3.I1.i1.p1.1.m1.5.5.cmml" xref="S3.I1.i1.p1.1.m1.5.5">30</cn><cn type="integer" id="S3.I1.i1.p1.1.m1.6.6.cmml" xref="S3.I1.i1.p1.1.m1.6.6">50</cn></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i1.p1.1.m1.6c">\in\{3,5,10,20,30,50\}</annotation></semantics></math></p>
</div>
</li>
<li id="S3.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i2.p1" class="ltx_para">
<p id="S3.I1.i2.p1.1" class="ltx_p">Use bivariate base functions <math id="S3.I1.i2.p1.1.m1.2" class="ltx_Math" alttext="\in\{\text{true},\text{false}\}" display="inline"><semantics id="S3.I1.i2.p1.1.m1.2a"><mrow id="S3.I1.i2.p1.1.m1.2.3" xref="S3.I1.i2.p1.1.m1.2.3.cmml"><mi id="S3.I1.i2.p1.1.m1.2.3.2" xref="S3.I1.i2.p1.1.m1.2.3.2.cmml"></mi><mo id="S3.I1.i2.p1.1.m1.2.3.1" xref="S3.I1.i2.p1.1.m1.2.3.1.cmml">∈</mo><mrow id="S3.I1.i2.p1.1.m1.2.3.3.2" xref="S3.I1.i2.p1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.I1.i2.p1.1.m1.2.3.3.2.1" xref="S3.I1.i2.p1.1.m1.2.3.3.1.cmml">{</mo><mtext id="S3.I1.i2.p1.1.m1.1.1" xref="S3.I1.i2.p1.1.m1.1.1a.cmml">true</mtext><mo id="S3.I1.i2.p1.1.m1.2.3.3.2.2" xref="S3.I1.i2.p1.1.m1.2.3.3.1.cmml">,</mo><mtext id="S3.I1.i2.p1.1.m1.2.2" xref="S3.I1.i2.p1.1.m1.2.2a.cmml">false</mtext><mo stretchy="false" id="S3.I1.i2.p1.1.m1.2.3.3.2.3" xref="S3.I1.i2.p1.1.m1.2.3.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.I1.i2.p1.1.m1.2b"><apply id="S3.I1.i2.p1.1.m1.2.3.cmml" xref="S3.I1.i2.p1.1.m1.2.3"><in id="S3.I1.i2.p1.1.m1.2.3.1.cmml" xref="S3.I1.i2.p1.1.m1.2.3.1"></in><csymbol cd="latexml" id="S3.I1.i2.p1.1.m1.2.3.2.cmml" xref="S3.I1.i2.p1.1.m1.2.3.2">absent</csymbol><set id="S3.I1.i2.p1.1.m1.2.3.3.1.cmml" xref="S3.I1.i2.p1.1.m1.2.3.3.2"><ci id="S3.I1.i2.p1.1.m1.1.1a.cmml" xref="S3.I1.i2.p1.1.m1.1.1"><mtext id="S3.I1.i2.p1.1.m1.1.1.cmml" xref="S3.I1.i2.p1.1.m1.1.1">true</mtext></ci><ci id="S3.I1.i2.p1.1.m1.2.2a.cmml" xref="S3.I1.i2.p1.1.m1.2.2"><mtext id="S3.I1.i2.p1.1.m1.2.2.cmml" xref="S3.I1.i2.p1.1.m1.2.2">false</mtext></ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i2.p1.1.m1.2c">\in\{\text{true},\text{false}\}</annotation></semantics></math></p>
</div>
</li>
<li id="S3.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i3.p1" class="ltx_para">
<p id="S3.I1.i3.p1.1" class="ltx_p">Use nonlinear functions <math id="S3.I1.i3.p1.1.m1.2" class="ltx_Math" alttext="\in\{\text{true},\text{false}\}" display="inline"><semantics id="S3.I1.i3.p1.1.m1.2a"><mrow id="S3.I1.i3.p1.1.m1.2.3" xref="S3.I1.i3.p1.1.m1.2.3.cmml"><mi id="S3.I1.i3.p1.1.m1.2.3.2" xref="S3.I1.i3.p1.1.m1.2.3.2.cmml"></mi><mo id="S3.I1.i3.p1.1.m1.2.3.1" xref="S3.I1.i3.p1.1.m1.2.3.1.cmml">∈</mo><mrow id="S3.I1.i3.p1.1.m1.2.3.3.2" xref="S3.I1.i3.p1.1.m1.2.3.3.1.cmml"><mo stretchy="false" id="S3.I1.i3.p1.1.m1.2.3.3.2.1" xref="S3.I1.i3.p1.1.m1.2.3.3.1.cmml">{</mo><mtext id="S3.I1.i3.p1.1.m1.1.1" xref="S3.I1.i3.p1.1.m1.1.1a.cmml">true</mtext><mo id="S3.I1.i3.p1.1.m1.2.3.3.2.2" xref="S3.I1.i3.p1.1.m1.2.3.3.1.cmml">,</mo><mtext id="S3.I1.i3.p1.1.m1.2.2" xref="S3.I1.i3.p1.1.m1.2.2a.cmml">false</mtext><mo stretchy="false" id="S3.I1.i3.p1.1.m1.2.3.3.2.3" xref="S3.I1.i3.p1.1.m1.2.3.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.I1.i3.p1.1.m1.2b"><apply id="S3.I1.i3.p1.1.m1.2.3.cmml" xref="S3.I1.i3.p1.1.m1.2.3"><in id="S3.I1.i3.p1.1.m1.2.3.1.cmml" xref="S3.I1.i3.p1.1.m1.2.3.1"></in><csymbol cd="latexml" id="S3.I1.i3.p1.1.m1.2.3.2.cmml" xref="S3.I1.i3.p1.1.m1.2.3.2">absent</csymbol><set id="S3.I1.i3.p1.1.m1.2.3.3.1.cmml" xref="S3.I1.i3.p1.1.m1.2.3.3.2"><ci id="S3.I1.i3.p1.1.m1.1.1a.cmml" xref="S3.I1.i3.p1.1.m1.1.1"><mtext id="S3.I1.i3.p1.1.m1.1.1.cmml" xref="S3.I1.i3.p1.1.m1.1.1">true</mtext></ci><ci id="S3.I1.i3.p1.1.m1.2.2a.cmml" xref="S3.I1.i3.p1.1.m1.2.2"><mtext id="S3.I1.i3.p1.1.m1.2.2.cmml" xref="S3.I1.i3.p1.1.m1.2.2">false</mtext></ci></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i3.p1.1.m1.2c">\in\{\text{true},\text{false}\}</annotation></semantics></math></p>
</div>
</li>
<li id="S3.I1.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i4.p1" class="ltx_para">
<p id="S3.I1.i4.p1.1" class="ltx_p">FFX only: L1 ratio <math id="S3.I1.i4.p1.1.m1.3" class="ltx_Math" alttext="\in\{0,0.5,1\}" display="inline"><semantics id="S3.I1.i4.p1.1.m1.3a"><mrow id="S3.I1.i4.p1.1.m1.3.4" xref="S3.I1.i4.p1.1.m1.3.4.cmml"><mi id="S3.I1.i4.p1.1.m1.3.4.2" xref="S3.I1.i4.p1.1.m1.3.4.2.cmml"></mi><mo id="S3.I1.i4.p1.1.m1.3.4.1" xref="S3.I1.i4.p1.1.m1.3.4.1.cmml">∈</mo><mrow id="S3.I1.i4.p1.1.m1.3.4.3.2" xref="S3.I1.i4.p1.1.m1.3.4.3.1.cmml"><mo stretchy="false" id="S3.I1.i4.p1.1.m1.3.4.3.2.1" xref="S3.I1.i4.p1.1.m1.3.4.3.1.cmml">{</mo><mn id="S3.I1.i4.p1.1.m1.1.1" xref="S3.I1.i4.p1.1.m1.1.1.cmml">0</mn><mo id="S3.I1.i4.p1.1.m1.3.4.3.2.2" xref="S3.I1.i4.p1.1.m1.3.4.3.1.cmml">,</mo><mn id="S3.I1.i4.p1.1.m1.2.2" xref="S3.I1.i4.p1.1.m1.2.2.cmml">0.5</mn><mo id="S3.I1.i4.p1.1.m1.3.4.3.2.3" xref="S3.I1.i4.p1.1.m1.3.4.3.1.cmml">,</mo><mn id="S3.I1.i4.p1.1.m1.3.3" xref="S3.I1.i4.p1.1.m1.3.3.cmml">1</mn><mo stretchy="false" id="S3.I1.i4.p1.1.m1.3.4.3.2.4" xref="S3.I1.i4.p1.1.m1.3.4.3.1.cmml">}</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.I1.i4.p1.1.m1.3b"><apply id="S3.I1.i4.p1.1.m1.3.4.cmml" xref="S3.I1.i4.p1.1.m1.3.4"><in id="S3.I1.i4.p1.1.m1.3.4.1.cmml" xref="S3.I1.i4.p1.1.m1.3.4.1"></in><csymbol cd="latexml" id="S3.I1.i4.p1.1.m1.3.4.2.cmml" xref="S3.I1.i4.p1.1.m1.3.4.2">absent</csymbol><set id="S3.I1.i4.p1.1.m1.3.4.3.1.cmml" xref="S3.I1.i4.p1.1.m1.3.4.3.2"><cn type="integer" id="S3.I1.i4.p1.1.m1.1.1.cmml" xref="S3.I1.i4.p1.1.m1.1.1">0</cn><cn type="float" id="S3.I1.i4.p1.1.m1.2.2.cmml" xref="S3.I1.i4.p1.1.m1.2.2">0.5</cn><cn type="integer" id="S3.I1.i4.p1.1.m1.3.3.cmml" xref="S3.I1.i4.p1.1.m1.3.3">1</cn></set></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.I1.i4.p1.1.m1.3c">\in\{0,0.5,1\}</annotation></semantics></math></p>
</div>
</li>
</ul>
</div>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Results</h2>

<figure id="S4.F1" class="ltx_figure"><svg id="S4.F1.1.pic1" class="ltx_picture ltx_centering" height="270.56" overflow="visible" version="1.1" width="490.55"><g transform="translate(0,270.56) matrix(1 0 0 -1 0 0)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><clipPath id="pgfcp1"><path d="M 0 0 h 490 v 270 h -490 Z"></path></clipPath><g clip-path="url(#pgfcp1)"><g></g><g stroke-linecap="butt" fill="#FFFFFF" stroke="#000000" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-opacity="0.000000" stroke-width="0.0pt"><path d="M 43.06 107.99 L 490 107.99 L 490 270 L 43.06 270 L 43.06 107.99 Z" style="stroke:none"></path></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 57.17 29.78)"><foreignObject width="59.77" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.6.6.6.6.1.1" class="ltx_text ltx_font_bold" style="color:#262626;">FFX NLS</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 78.45 66.09)"><foreignObject width="28.44" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.7.7.7.7.1.1" class="ltx_text ltx_font_bold" style="color:#262626;">FFX</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 99.74 77.91)"><foreignObject width="20.28" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.8.8.8.8.1.1" class="ltx_text" style="color:#262626;">GP</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 121.02 37.79)"><foreignObject width="60.25" height="12.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.9.9.9.9.1.1" class="ltx_text" style="color:#262626;">GP CoOp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 142.3 59.39)"><foreignObject width="38.72" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.10.10.10.10.1.1" class="ltx_text" style="color:#262626;">OSGP</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 163.58 19.27)"><foreignObject width="78.7" height="12.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.11.11.11.11.1.1" class="ltx_text" style="color:#262626;">OSGP CoOp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 184.87 71.18)"><foreignObject width="26.98" height="8.65" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.12.12.12.12.1.1" class="ltx_text" style="color:#262626;">gsgp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 206.15 79.36)"><foreignObject width="18.83" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.13.13.13.13.1.1" class="ltx_text" style="color:#262626;">afp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 227.43 66.59)"><foreignObject width="31.56" height="8.65" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.14.14.14.14.1.1" class="ltx_text" style="color:#262626;">mrgp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 248.72 67.01)"><foreignObject width="31.13" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.15.15.15.15.1.1" class="ltx_text" style="color:#262626;">eplex</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 270 43.87)"><foreignObject width="54.2" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.16.16.16.16.1.1" class="ltx_text" style="color:#262626;">eplex-1m</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 291.28 50.73)"><foreignObject width="46.97" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.17.17.17.17.1.1" class="ltx_text" style="color:#262626;">xgboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 312.57 37.96)"><foreignObject width="59.69" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.18.18.18.18.1.1" class="ltx_text" style="color:#262626;">gradboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 333.85 75.12)"><foreignObject width="23.06" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.19.19.19.19.1.1" class="ltx_text" style="color:#262626;">mlp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 355.13 88.58)"><foreignObject width="9.65" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.20.20.20.20.1.1" class="ltx_text" style="color:#262626;">rf</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 376.42 27.2)"><foreignObject width="70.8" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.21.21.21.21.1.1" class="ltx_text" style="color:#262626;">kernel-ridge</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 397.7 43.4)"><foreignObject width="54.27" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.22.22.22.22.1.1" class="ltx_text" style="color:#262626;">adaboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 418.98 43.21)"><foreignObject width="54.85" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.23.23.23.23.1.1" class="ltx_text" style="color:#262626;">lasso-lars</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 440.26 41.4)"><foreignObject width="56.65" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.24.24.24.24.1.1" class="ltx_text" style="color:#262626;">linear-svr</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 461.55 0)"><foreignObject width="97.9" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.25.25.25.25.1.1" class="ltx_text" style="color:#262626;">linear-regression</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 482.83 13.85)"><foreignObject width="84.1" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.26.26.26.26.1.1" class="ltx_text" style="color:#262626;">sgd-regression</span></foreignObject></g><g></g><g><clipPath id="pgfcp2"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp2)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 107.99 L 490 107.99" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 26.39 103.16)"><foreignObject width="6.92" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{0}" display="inline"><semantics id="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1">0</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F1.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1c">\displaystyle{0}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp3"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp3)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 144.81 L 490 144.81" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 26.39 139.99)"><foreignObject width="6.92" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{5}" display="inline"><semantics id="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1" xref="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml">5</mn><annotation-xml encoding="MathML-Content" id="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1">5</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F1.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1c">\displaystyle{5}</annotation></semantics></math><!-- %****␣ranking_training.pgf␣Line␣225␣**** --></foreignObject></g><g></g><g><clipPath id="pgfcp4"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp4)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 181.63 L 490 181.63" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 176.81)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10}" display="inline"><semantics id="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1" xref="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml">10</mn><annotation-xml encoding="MathML-Content" id="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1">10</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F1.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1c">\displaystyle{10}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp5"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp5)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 218.45 L 490 218.45" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 213.63)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{15}" display="inline"><semantics id="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1" xref="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml">15</mn><annotation-xml encoding="MathML-Content" id="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1">15</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F1.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1c">\displaystyle{15}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp6"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp6)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 255.27 L 490 255.27" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 250.45)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{20}" display="inline"><semantics id="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1" xref="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.cmml">20</mn><annotation-xml encoding="MathML-Content" id="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1">20</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F1.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1c">\displaystyle{20}</annotation></semantics></math></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 10.43 109.78)"><foreignObject width="158.43" height="13.84" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F1.1.pic1.27.27.27.27.1.1" class="ltx_text" style="color:#262626;">Median ranking (training)</span></foreignObject></g><g></g><g><clipPath id="pgfcp7"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp7)"><g stroke-linecap="butt" fill="#A9A9A9" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 45.18 181.63 L 62.21 181.63 L 62.21 186.61 L 57.95 188.99 L 62.21 191.38 L 62.21 196.36 L 45.18 196.36 L 45.18 191.38 L 49.44 188.99 L 45.18 186.61 L 45.18 181.63 L 45.18 181.63 Z"></path></g></g><g><clipPath id="pgfcp8"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp8)"><g stroke-linecap="butt" fill="#A9A9A9" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 66.47 203.72 L 83.49 203.72 L 83.49 221.04 L 79.24 225.81 L 83.49 230.58 L 83.49 233.18 L 66.47 233.18 L 66.47 230.58 L 70.72 225.81 L 66.47 221.04 L 66.47 203.72 L 66.47 203.72 Z"></path></g></g><g><clipPath id="pgfcp9"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp9)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 87.75 203.72 L 104.78 203.72 L 104.78 208.7 L 100.52 211.09 L 104.78 213.47 L 104.78 218.45 L 87.75 218.45 L 87.75 213.47 L 92.01 211.09 L 87.75 208.7 L 87.75 203.72 L 87.75 203.72 Z"></path></g></g><g><clipPath id="pgfcp10"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp10)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 109.03 152.17 L 126.06 152.17 L 126.06 157.15 L 121.8 159.54 L 126.06 161.92 L 126.06 166.9 L 109.03 166.9 L 109.03 161.92 L 113.29 159.54 L 109.03 157.15 L 109.03 152.17 L 109.03 152.17 Z"></path></g></g><g><clipPath id="pgfcp11"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp11)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 130.32 188.99 L 147.34 188.99 L 147.34 200.14 L 143.09 203.72 L 147.34 207.3 L 147.34 211.09 L 130.32 211.09 L 130.32 207.3 L 134.57 203.72 L 130.32 200.14 L 130.32 188.99 L 130.32 188.99 Z"></path></g></g><g><clipPath id="pgfcp12"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp12)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 151.6 152.17 L 168.63 152.17 L 168.63 157.15 L 164.37 159.54 L 168.63 161.92 L 168.63 166.9 L 151.6 166.9 L 151.6 161.92 L 155.86 159.54 L 151.6 157.15 L 151.6 152.17 L 151.6 152.17 Z"></path></g></g><g><clipPath id="pgfcp13"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp13)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 172.88 144.81 L 189.91 144.81 L 189.91 172.09 L 185.65 181.63 L 189.91 191.17 L 189.91 203.72 L 172.88 203.72 L 172.88 191.17 L 177.14 181.63 L 172.88 172.09 L 172.88 144.81 L 172.88 144.81 Z"></path></g></g><g><clipPath id="pgfcp14"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp14)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 194.17 225.81 L 211.19 225.81 L 211.19 228.41 L 206.94 233.18 L 211.19 237.95 L 211.19 255.27 L 194.17 255.27 L 194.17 237.95 L 198.42 233.18 L 194.17 228.41 L 194.17 225.81 L 194.17 225.81 Z"></path></g></g><g><clipPath id="pgfcp15"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp15)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 215.45 122.71 L 232.48 122.71 L 232.48 134.16 L 228.22 137.44 L 232.48 140.72 L 232.48 142.97 L 215.45 142.97 L 215.45 140.72 L 219.71 137.44 L 215.45 134.16 L 215.45 122.71 L 215.45 122.71 Z"></path></g></g><g><clipPath id="pgfcp16"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp16)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 236.73 188.99 L 253.76 188.99 L 253.76 200.14 L 249.5 203.72 L 253.76 207.3 L 253.76 211.09 L 236.73 211.09 L 236.73 207.3 L 240.99 203.72 L 236.73 200.14 L 236.73 188.99 L 236.73 188.99 Z"></path></g></g><g><clipPath id="pgfcp17"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp17)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 258.01 174.26 L 275.04 174.26 L 275.04 170.69 L 270.78 174.26 L 275.04 177.84 L 275.04 196.36 L 258.01 196.36 L 258.01 177.84 L 262.27 174.26 L 258.01 170.69 L 258.01 174.26 L 258.01 174.26 Z"></path></g></g><g><clipPath id="pgfcp18"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp18)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 279.3 122.71 L 296.32 122.71 L 296.32 125.2 L 292.07 126.4 L 296.32 127.59 L 296.32 130.08 L 279.3 130.08 L 279.3 127.59 L 283.55 126.4 L 279.3 125.2 L 279.3 122.71 L 279.3 122.71 Z"></path></g></g><g><clipPath id="pgfcp19"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp19)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 300.58 115.35 L 317.61 115.35 L 317.61 114.16 L 313.35 115.35 L 317.61 116.54 L 317.61 122.71 L 300.58 122.71 L 300.58 116.54 L 304.84 115.35 L 300.58 114.16 L 300.58 115.35 L 300.58 115.35 Z"></path></g></g><g><clipPath id="pgfcp20"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp20)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 321.86 130.08 L 338.89 130.08 L 338.89 146.21 L 334.63 152.17 L 338.89 158.13 L 338.89 166.9 L 321.86 166.9 L 321.86 158.13 L 326.12 152.17 L 321.86 146.21 L 321.86 130.08 L 321.86 130.08 Z"></path></g></g><g><clipPath id="pgfcp21"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp21)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 343.15 130.08 L 360.17 130.08 L 360.17 141.23 L 355.92 144.81 L 360.17 148.39 L 360.17 152.17 L 343.15 152.17 L 343.15 148.39 L 347.4 144.81 L 343.15 141.23 L 343.15 130.08 L 343.15 130.08 Z"></path></g></g><g><clipPath id="pgfcp22"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp22)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 364.43 137.44 L 381.46 137.44 L 381.46 153.57 L 377.2 159.54 L 381.46 165.5 L 381.46 174.26 L 364.43 174.26 L 364.43 165.5 L 368.69 159.54 L 364.43 153.57 L 364.43 137.44 L 364.43 137.44 Z"></path></g></g><g><clipPath id="pgfcp23"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp23)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 385.71 159.54 L 402.74 159.54 L 402.74 176.86 L 398.48 181.63 L 402.74 186.4 L 402.74 188.99 L 385.71 188.99 L 385.71 186.4 L 389.97 181.63 L 385.71 176.86 L 385.71 159.54 L 385.71 159.54 Z"></path></g></g><g><clipPath id="pgfcp24"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp24)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 407 240.54 L 424.02 240.54 L 424.02 245.52 L 419.77 247.91 L 424.02 250.29 L 424.02 255.27 L 407 255.27 L 407 250.29 L 411.25 247.91 L 407 245.52 L 407 240.54 L 407 240.54 Z"></path></g></g><g><clipPath id="pgfcp25"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp25)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 428.28 233.18 L 445.3 233.18 L 445.3 238.16 L 441.05 240.54 L 445.3 242.93 L 445.3 247.91 L 428.28 247.91 L 428.28 242.93 L 432.54 240.54 L 428.28 238.16 L 428.28 233.18 L 428.28 233.18 Z"></path></g></g><g><clipPath id="pgfcp26"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp26)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 449.56 233.18 L 466.59 233.18 L 466.59 238.16 L 462.33 240.54 L 466.59 242.93 L 466.59 247.91 L 449.56 247.91 L 449.56 242.93 L 453.82 240.54 L 449.56 238.16 L 449.56 233.18 L 449.56 233.18 Z"></path></g></g><g><clipPath id="pgfcp27"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp27)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 470.85 247.91 L 487.87 247.91 L 487.87 252.89 L 483.62 255.27 L 487.87 257.66 L 487.87 262.64 L 470.85 262.64 L 470.85 257.66 L 475.1 255.27 L 470.85 252.89 L 470.85 247.91 L 470.85 247.91 Z"></path></g></g><g><clipPath id="pgfcp28"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp28)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 53.7 181.63 L 53.7 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp29"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp29)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 53.7 196.36 L 53.7 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp30"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp30)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.44 159.54 L 57.95 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp31"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp31)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.44 218.45 L 57.95 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp32"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp32)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 74.98 203.72 L 74.98 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp33"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp33)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 74.98 233.18 L 74.98 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp34"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp34)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 70.72 159.54 L 79.24 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp35"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp35)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 70.72 262.64 L 79.24 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp36"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp36)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 96.26 203.72 L 96.26 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp37"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp37)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 96.26 218.45 L 96.26 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp38"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp38)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 92.01 181.63 L 100.52 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp39"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp39)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 92.01 240.54 L 100.52 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp40"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp40)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 117.55 152.17 L 117.55 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp41"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp41)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 117.55 166.9 L 117.55 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp42"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp42)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 113.29 137.44 L 121.8 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp43"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp43)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 113.29 188.99 L 121.8 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp44"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp44)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 138.83 188.99 L 138.83 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp45"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp45)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 138.83 211.09 L 138.83 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp46"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp46)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 134.57 159.54 L 143.09 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp47"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp47)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 134.57 240.54 L 143.09 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp48"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp48)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 160.11 152.17 L 160.11 130.08" style="fill:none"></path></g></g><g><clipPath id="pgfcp49"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp49)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 160.11 166.9 L 160.11 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp50"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp50)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 155.86 130.08 L 164.37 130.08" style="fill:none"></path></g></g><g><clipPath id="pgfcp51"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp51)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 155.86 188.99 L 164.37 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp52"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp52)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 181.39 144.81 L 181.39 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp53"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp53)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 181.39 203.72 L 181.39 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp54"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp54)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 177.14 122.71 L 185.65 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp55"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp55)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 177.14 262.64 L 185.65 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp56"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp56)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 202.68 225.81 L 202.68 196.36" style="fill:none"></path></g></g><g><clipPath id="pgfcp57"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp57)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 202.68 255.27 L 202.68 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp58"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp58)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 198.42 196.36 L 206.94 196.36" style="fill:none"></path></g></g><g><clipPath id="pgfcp59"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp59)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 198.42 262.64 L 206.94 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp60"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp60)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 223.96 122.71 L 223.96 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp61"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp61)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 223.96 142.97 L 223.96 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp62"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp62)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 219.71 115.35 L 228.22 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp63"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp63)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 219.71 166.9 L 228.22 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp64"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp64)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 245.25 188.99 L 245.25 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp65"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp65)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 245.25 211.09 L 245.25 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp66"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp66)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 240.99 166.9 L 249.5 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp67"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp67)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 240.99 240.54 L 249.5 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp68"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp68)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 266.53 174.26 L 266.53 144.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp69"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp69)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 266.53 196.36 L 266.53 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp70"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp70)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 262.27 144.81 L 270.78 144.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp71"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp71)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 262.27 225.81 L 270.78 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp72"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp72)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 287.81 122.71 L 287.81 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp73"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp73)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 287.81 130.08 L 287.81 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp74"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp74)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 283.55 115.35 L 292.07 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp75"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp75)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 283.55 137.44 L 292.07 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp76"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp76)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 309.09 115.35 L 309.09 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp77"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp77)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 309.09 122.71 L 309.09 130.08" style="fill:none"></path></g></g><g><clipPath id="pgfcp78"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp78)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 304.84 115.35 L 313.35 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp79"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp79)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 304.84 130.08 L 313.35 130.08" style="fill:none"></path></g></g><g><clipPath id="pgfcp80"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp80)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 330.38 130.08 L 330.38 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp81"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp81)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 330.38 166.9 L 330.38 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp82"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp82)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 326.12 115.35 L 334.63 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp83"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp83)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 326.12 218.45 L 334.63 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp84"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp84)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 351.66 130.08 L 351.66 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp85"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp85)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 351.66 152.17 L 351.66 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp86"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp86)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 347.4 115.35 L 355.92 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp87"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp87)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 347.4 181.63 L 355.92 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp88"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp88)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 372.94 137.44 L 372.94 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp89"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp89)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 372.94 174.26 L 372.94 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp90"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp90)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 368.69 115.35 L 377.2 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp91"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp91)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 368.69 225.81 L 377.2 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp92"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp92)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 394.23 159.54 L 394.23 130.08" style="fill:none"></path></g></g><g><clipPath id="pgfcp93"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp93)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 394.23 188.99 L 394.23 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp94"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp94)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 389.97 130.08 L 398.48 130.08" style="fill:none"></path></g></g><g><clipPath id="pgfcp95"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp95)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 389.97 225.81 L 398.48 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp96"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp96)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 415.51 240.54 L 415.51 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp97"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp97)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 415.51 255.27 L 415.51 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp98"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp98)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 411.25 225.81 L 419.77 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp99"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp99)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 411.25 262.64 L 419.77 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp100"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp100)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 436.79 233.18 L 436.79 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp101"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp101)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 436.79 247.91 L 436.79 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp102"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp102)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 432.54 218.45 L 441.05 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp103"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp103)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 432.54 262.64 L 441.05 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp104"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp104)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 458.07 233.18 L 458.07 211.09" style="fill:none"></path></g></g><g><clipPath id="pgfcp105"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp105)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 458.07 247.91 L 458.07 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp106"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp106)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 453.82 211.09 L 462.33 211.09" style="fill:none"></path></g></g><g><clipPath id="pgfcp107"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp107)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 453.82 262.64 L 462.33 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp108"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp108)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 479.36 247.91 L 479.36 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp109"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp109)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 479.36 262.64 L 479.36 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp110"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp110)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 475.1 233.18 L 483.62 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp111"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp111)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 475.1 262.64 L 483.62 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp112"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp112)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.44 188.99 L 57.95 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp113"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp113)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 70.72 225.81 L 79.24 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp114"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp114)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 92.01 211.09 L 100.52 211.09" style="fill:none"></path></g></g><g><clipPath id="pgfcp115"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp115)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 113.29 159.54 L 121.8 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp116"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp116)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 134.57 203.72 L 143.09 203.72" style="fill:none"></path></g></g><g><clipPath id="pgfcp117"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp117)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 155.86 159.54 L 164.37 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp118"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp118)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 177.14 181.63 L 185.65 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp119"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp119)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 198.42 233.18 L 206.94 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp120"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp120)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 219.71 137.44 L 228.22 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp121"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp121)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 240.99 203.72 L 249.5 203.72" style="fill:none"></path></g></g><g><clipPath id="pgfcp122"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp122)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 262.27 174.26 L 270.78 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp123"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp123)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 283.55 126.4 L 292.07 126.4" style="fill:none"></path></g></g><g><clipPath id="pgfcp124"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp124)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 304.84 115.35 L 313.35 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp125"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp125)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 326.12 152.17 L 334.63 152.17" style="fill:none"></path></g></g><g><clipPath id="pgfcp126"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp126)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 347.4 144.81 L 355.92 144.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp127"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp127)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 368.69 159.54 L 377.2 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp128"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp128)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 389.97 181.63 L 398.48 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp129"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp129)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 411.25 247.91 L 419.77 247.91" style="fill:none"></path></g></g><g><clipPath id="pgfcp130"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp130)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 432.54 240.54 L 441.05 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp131"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp131)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 453.82 240.54 L 462.33 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp132"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp132)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 475.1 255.27 L 483.62 255.27" style="fill:none"></path></g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 43.06 107.99 L 43.06 270" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 490 107.99 L 490 270" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 43.06 107.99 L 490 107.99" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 43.06 270 L 490 270" style="fill:none"></path></g><g></g></g></g></svg>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure"><span id="S4.F1.3.1.1" class="ltx_text" style="font-size:90%;">Figure 1</span>: </span><span id="S4.F1.4.2" class="ltx_text" style="font-size:90%;">Distribution of median rankings of the mean squared error on the training set.</span></figcaption>
</figure>
<figure id="S4.F2" class="ltx_figure"><svg id="S4.F2.1.pic1" class="ltx_picture ltx_centering" height="270.56" overflow="visible" version="1.1" width="490.55"><g transform="translate(0,270.56) matrix(1 0 0 -1 0 0)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><clipPath id="pgfcp133"><path d="M 0 0 h 490 v 270 h -490 Z"></path></clipPath><g clip-path="url(#pgfcp133)"><g></g><g stroke-linecap="butt" fill="#FFFFFF" stroke="#000000" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-opacity="0.000000" stroke-width="0.0pt"><path d="M 43.06 107.99 L 490 107.99 L 490 270 L 43.06 270 L 43.06 107.99 Z" style="stroke:none"></path></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 57.17 29.78)"><foreignObject width="59.77" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.6.6.6.6.1.1" class="ltx_text ltx_font_bold" style="color:#262626;">FFX NLS</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 78.45 66.09)"><foreignObject width="28.44" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.7.7.7.7.1.1" class="ltx_text ltx_font_bold" style="color:#262626;">FFX</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 99.74 77.91)"><foreignObject width="20.28" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.8.8.8.8.1.1" class="ltx_text" style="color:#262626;">GP</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 121.02 37.79)"><foreignObject width="60.25" height="12.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.9.9.9.9.1.1" class="ltx_text" style="color:#262626;">GP CoOp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 142.3 59.39)"><foreignObject width="38.72" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.10.10.10.10.1.1" class="ltx_text" style="color:#262626;">OSGP</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 163.58 19.27)"><foreignObject width="78.7" height="12.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.11.11.11.11.1.1" class="ltx_text" style="color:#262626;">OSGP CoOp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 184.87 71.18)"><foreignObject width="26.98" height="8.65" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.12.12.12.12.1.1" class="ltx_text" style="color:#262626;">gsgp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 206.15 79.36)"><foreignObject width="18.83" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.13.13.13.13.1.1" class="ltx_text" style="color:#262626;">afp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 227.43 66.59)"><foreignObject width="31.56" height="8.65" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.14.14.14.14.1.1" class="ltx_text" style="color:#262626;">mrgp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 248.72 67.01)"><foreignObject width="31.13" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.15.15.15.15.1.1" class="ltx_text" style="color:#262626;">eplex</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 270 43.87)"><foreignObject width="54.2" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.16.16.16.16.1.1" class="ltx_text" style="color:#262626;">eplex-1m</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 291.28 50.73)"><foreignObject width="46.97" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.17.17.17.17.1.1" class="ltx_text" style="color:#262626;">xgboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 312.57 37.96)"><foreignObject width="59.69" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.18.18.18.18.1.1" class="ltx_text" style="color:#262626;">gradboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 333.85 75.12)"><foreignObject width="23.06" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.19.19.19.19.1.1" class="ltx_text" style="color:#262626;">mlp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 355.13 88.58)"><foreignObject width="9.65" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.20.20.20.20.1.1" class="ltx_text" style="color:#262626;">rf</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 376.42 27.2)"><foreignObject width="70.8" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.21.21.21.21.1.1" class="ltx_text" style="color:#262626;">kernel-ridge</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 397.7 43.4)"><foreignObject width="54.27" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.22.22.22.22.1.1" class="ltx_text" style="color:#262626;">adaboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 418.98 43.21)"><foreignObject width="54.85" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.23.23.23.23.1.1" class="ltx_text" style="color:#262626;">lasso-lars</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 440.26 41.4)"><foreignObject width="56.65" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.24.24.24.24.1.1" class="ltx_text" style="color:#262626;">linear-svr</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 461.55 0)"><foreignObject width="97.9" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.25.25.25.25.1.1" class="ltx_text" style="color:#262626;">linear-regression</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 482.83 13.85)"><foreignObject width="84.1" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.26.26.26.26.1.1" class="ltx_text" style="color:#262626;">sgd-regression</span></foreignObject></g><g></g><g><clipPath id="pgfcp134"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp134)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 107.99 L 490 107.99" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 26.39 103.16)"><foreignObject width="6.92" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{0}" display="inline"><semantics id="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1">0</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F2.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1c">\displaystyle{0}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp135"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp135)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 144.81 L 490 144.81" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 26.39 139.99)"><foreignObject width="6.92" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{5}" display="inline"><semantics id="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1" xref="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml">5</mn><annotation-xml encoding="MathML-Content" id="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1">5</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F2.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1c">\displaystyle{5}</annotation></semantics></math><!-- %****␣ranking_test.pgf␣Line␣225␣**** --></foreignObject></g><g></g><g><clipPath id="pgfcp136"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp136)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 181.63 L 490 181.63" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 176.81)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10}" display="inline"><semantics id="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1" xref="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml">10</mn><annotation-xml encoding="MathML-Content" id="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1">10</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F2.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1c">\displaystyle{10}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp137"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp137)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 218.45 L 490 218.45" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 213.63)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{15}" display="inline"><semantics id="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1" xref="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml">15</mn><annotation-xml encoding="MathML-Content" id="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1">15</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F2.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1c">\displaystyle{15}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp138"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp138)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 43.06 255.27 L 490 255.27" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 250.45)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{20}" display="inline"><semantics id="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1" xref="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.cmml">20</mn><annotation-xml encoding="MathML-Content" id="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1">20</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F2.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1c">\displaystyle{20}</annotation></semantics></math></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 10.43 122.44)"><foreignObject width="133.1" height="13.84" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F2.1.pic1.27.27.27.27.1.1" class="ltx_text" style="color:#262626;">Median ranking (test)</span></foreignObject></g><g></g><g><clipPath id="pgfcp139"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp139)"><g stroke-linecap="butt" fill="#A9A9A9" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 45.18 137.44 L 62.21 137.44 L 62.21 179.45 L 57.95 188.99 L 62.21 198.53 L 62.21 196.36 L 45.18 196.36 L 45.18 198.53 L 49.44 188.99 L 45.18 179.45 L 45.18 137.44 L 45.18 137.44 Z"></path></g></g><g><clipPath id="pgfcp140"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp140)"><g stroke-linecap="butt" fill="#A9A9A9" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 66.47 174.26 L 83.49 174.26 L 83.49 201.55 L 79.24 211.09 L 83.49 220.63 L 83.49 233.18 L 66.47 233.18 L 66.47 220.63 L 70.72 211.09 L 66.47 201.55 L 66.47 174.26 L 66.47 174.26 Z"></path></g></g><g><clipPath id="pgfcp141"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp141)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 87.75 176.11 L 104.78 176.11 L 104.78 184.52 L 100.52 188.99 L 104.78 193.46 L 104.78 203.72 L 87.75 203.72 L 87.75 193.46 L 92.01 188.99 L 87.75 184.52 L 87.75 176.11 L 87.75 176.11 Z"></path></g></g><g><clipPath id="pgfcp142"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp142)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 109.03 122.71 L 126.06 122.71 L 126.06 119.14 L 121.8 122.71 L 126.06 126.29 L 126.06 144.81 L 109.03 144.81 L 109.03 126.29 L 113.29 122.71 L 109.03 119.14 L 109.03 122.71 L 109.03 122.71 Z"></path></g></g><g><clipPath id="pgfcp143"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp143)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 130.32 166.9 L 147.34 166.9 L 147.34 183.03 L 143.09 188.99 L 147.34 194.96 L 147.34 203.72 L 130.32 203.72 L 130.32 194.96 L 134.57 188.99 L 130.32 183.03 L 130.32 166.9 L 130.32 166.9 Z"></path></g></g><g><clipPath id="pgfcp144"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp144)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 151.6 115.35 L 168.63 115.35 L 168.63 115.86 L 164.37 122.71 L 168.63 129.57 L 168.63 157.69 L 151.6 157.69 L 151.6 129.57 L 155.86 122.71 L 151.6 115.86 L 151.6 115.35 L 151.6 115.35 Z"></path></g></g><g><clipPath id="pgfcp145"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp145)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 172.88 225.81 L 189.91 225.81 L 189.91 241.94 L 185.65 247.91 L 189.91 253.87 L 189.91 262.64 L 172.88 262.64 L 172.88 253.87 L 177.14 247.91 L 172.88 241.94 L 172.88 225.81 L 172.88 225.81 Z"></path></g></g><g><clipPath id="pgfcp146"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp146)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 194.17 203.72 L 211.19 203.72 L 211.19 214.87 L 206.94 218.45 L 211.19 222.03 L 211.19 225.81 L 194.17 225.81 L 194.17 222.03 L 198.42 218.45 L 194.17 214.87 L 194.17 203.72 L 194.17 203.72 Z"></path></g></g><g><clipPath id="pgfcp147"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp147)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 215.45 130.08 L 232.48 130.08 L 232.48 167.53 L 228.22 188.99 L 232.48 210.46 L 232.48 262.64 L 215.45 262.64 L 215.45 210.46 L 219.71 188.99 L 215.45 167.53 L 215.45 130.08 L 215.45 130.08 Z"></path></g></g><g><clipPath id="pgfcp148"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp148)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 236.73 166.9 L 253.76 166.9 L 253.76 169.79 L 249.5 174.26 L 253.76 178.74 L 253.76 194.52 L 236.73 194.52 L 236.73 178.74 L 240.99 174.26 L 236.73 169.79 L 236.73 166.9 L 236.73 166.9 Z"></path></g></g><g><clipPath id="pgfcp149"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp149)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 258.01 131.92 L 275.04 131.92 L 275.04 131.78 L 270.78 137.44 L 275.04 143.11 L 275.04 166.9 L 258.01 166.9 L 258.01 143.11 L 262.27 137.44 L 258.01 131.78 L 258.01 131.92 L 258.01 131.92 Z"></path></g></g><g><clipPath id="pgfcp150"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp150)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 279.3 137.44 L 296.32 137.44 L 296.32 145.02 L 292.07 152.17 L 296.32 159.33 L 296.32 181.63 L 279.3 181.63 L 279.3 159.33 L 283.55 152.17 L 279.3 145.02 L 279.3 137.44 L 279.3 137.44 Z"></path></g></g><g><clipPath id="pgfcp151"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp151)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 300.58 144.81 L 317.61 144.81 L 317.61 146.51 L 313.35 152.17 L 317.61 157.84 L 317.61 179.79 L 300.58 179.79 L 300.58 157.84 L 304.84 152.17 L 300.58 146.51 L 300.58 144.81 L 300.58 144.81 Z"></path></g></g><g><clipPath id="pgfcp152"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp152)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 321.86 137.44 L 338.89 137.44 L 338.89 150 L 334.63 159.54 L 338.89 169.08 L 338.89 196.36 L 321.86 196.36 L 321.86 169.08 L 326.12 159.54 L 321.86 150 L 321.86 137.44 L 321.86 137.44 Z"></path></g></g><g><clipPath id="pgfcp153"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp153)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 343.15 159.54 L 360.17 159.54 L 360.17 159.75 L 355.92 166.9 L 360.17 174.06 L 360.17 203.72 L 343.15 203.72 L 343.15 174.06 L 347.4 166.9 L 343.15 159.75 L 343.15 159.54 L 343.15 159.54 Z"></path></g></g><g><clipPath id="pgfcp154"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp154)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 364.43 146.65 L 381.46 146.65 L 381.46 170 L 377.2 181.63 L 381.46 193.26 L 381.46 218.45 L 364.43 218.45 L 364.43 193.26 L 368.69 181.63 L 364.43 170 L 364.43 146.65 L 364.43 146.65 Z"></path></g></g><g><clipPath id="pgfcp155"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp155)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 385.71 166.9 L 402.74 166.9 L 402.74 174.47 L 398.48 181.63 L 402.74 188.78 L 402.74 211.09 L 385.71 211.09 L 385.71 188.78 L 389.97 181.63 L 385.71 174.47 L 385.71 166.9 L 385.71 166.9 Z"></path></g></g><g><clipPath id="pgfcp156"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp156)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 407 211.09 L 424.02 211.09 L 424.02 227.22 L 419.77 233.18 L 424.02 239.14 L 424.02 247.91 L 407 247.91 L 407 239.14 L 411.25 233.18 L 407 227.22 L 407 211.09 L 407 211.09 Z"></path></g></g><g><clipPath id="pgfcp157"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp157)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 428.28 211.09 L 445.3 211.09 L 445.3 234.58 L 441.05 240.54 L 445.3 246.51 L 445.3 247.91 L 428.28 247.91 L 428.28 246.51 L 432.54 240.54 L 428.28 234.58 L 428.28 211.09 L 428.28 211.09 Z"></path></g></g><g><clipPath id="pgfcp158"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp158)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 449.56 225.81 L 466.59 225.81 L 466.59 236.97 L 462.33 240.54 L 466.59 244.12 L 466.59 247.91 L 449.56 247.91 L 449.56 244.12 L 453.82 240.54 L 449.56 236.97 L 449.56 225.81 L 449.56 225.81 Z"></path></g></g><g><clipPath id="pgfcp159"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp159)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 470.85 225.81 L 487.87 225.81 L 487.87 235.77 L 483.62 240.54 L 487.87 245.31 L 487.87 255.27 L 470.85 255.27 L 470.85 245.31 L 475.1 240.54 L 470.85 235.77 L 470.85 225.81 L 470.85 225.81 Z"></path></g></g><g><clipPath id="pgfcp160"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp160)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 53.7 137.44 L 53.7 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp161"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp161)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 53.7 196.36 L 53.7 247.91" style="fill:none"></path></g></g><g><clipPath id="pgfcp162"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp162)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 49.44 115.35 L 57.95 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp163"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp163)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 49.44 247.91 L 57.95 247.91" style="fill:none"></path></g></g><g><clipPath id="pgfcp164"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp164)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 74.98 174.26 L 74.98 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp165"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp165)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 74.98 233.18 L 74.98 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp166"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp166)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 70.72 115.35 L 79.24 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp167"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp167)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 70.72 262.64 L 79.24 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp168"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp168)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 96.26 176.11 L 96.26 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp169"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp169)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 96.26 203.72 L 96.26 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp170"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp170)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 92.01 137.44 L 100.52 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp171"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp171)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 92.01 225.81 L 100.52 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp172"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp172)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 117.55 122.71 L 117.55 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp173"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp173)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 117.55 144.81 L 117.55 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp174"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp174)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 113.29 115.35 L 121.8 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp175"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp175)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 113.29 174.26 L 121.8 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp176"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp176)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 138.83 166.9 L 138.83 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp177"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp177)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 138.83 203.72 L 138.83 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp178"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp178)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 134.57 122.71 L 143.09 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp179"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp179)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 134.57 233.18 L 143.09 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp180"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp180)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 160.11 115.35 L 160.11 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp181"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp181)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 160.11 157.69 L 160.11 211.09" style="fill:none"></path></g></g><g><clipPath id="pgfcp182"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp182)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 155.86 115.35 L 164.37 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp183"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp183)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 155.86 211.09 L 164.37 211.09" style="fill:none"></path></g></g><g><clipPath id="pgfcp184"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp184)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 181.39 225.81 L 181.39 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp185"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp185)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 181.39 262.64 L 181.39 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp186"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp186)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 177.14 174.26 L 185.65 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp187"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp187)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 177.14 262.64 L 185.65 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp188"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp188)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 202.68 203.72 L 202.68 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp189"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp189)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 202.68 225.81 L 202.68 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp190"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp190)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 198.42 174.26 L 206.94 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp191"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp191)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 198.42 255.27 L 206.94 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp192"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp192)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 223.96 130.08 L 223.96 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp193"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp193)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 223.96 262.64 L 223.96 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp194"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp194)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 219.71 115.35 L 228.22 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp195"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp195)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 219.71 262.64 L 228.22 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp196"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp196)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 245.25 166.9 L 245.25 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp197"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp197)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 245.25 194.52 L 245.25 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp198"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp198)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 240.99 137.44 L 249.5 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp199"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp199)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 240.99 233.18 L 249.5 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp200"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp200)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 266.53 131.92 L 266.53 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp201"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp201)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 266.53 166.9 L 266.53 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp202"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp202)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 262.27 115.35 L 270.78 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp203"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp203)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 262.27 218.45 L 270.78 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp204"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp204)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 287.81 137.44 L 287.81 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp205"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp205)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 287.81 181.63 L 287.81 247.91" style="fill:none"></path></g></g><g><clipPath id="pgfcp206"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp206)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 283.55 115.35 L 292.07 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp207"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp207)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 283.55 247.91 L 292.07 247.91" style="fill:none"></path></g></g><g><clipPath id="pgfcp208"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp208)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 309.09 144.81 L 309.09 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp209"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp209)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 309.09 179.79 L 309.09 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp210"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp210)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 304.84 122.71 L 313.35 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp211"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp211)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 304.84 225.81 L 313.35 225.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp212"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp212)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 330.38 137.44 L 330.38 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp213"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp213)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 330.38 196.36 L 330.38 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp214"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp214)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 326.12 115.35 L 334.63 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp215"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp215)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 326.12 255.27 L 334.63 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp216"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp216)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 351.66 159.54 L 351.66 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp217"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp217)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 351.66 203.72 L 351.66 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp218"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp218)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 347.4 115.35 L 355.92 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp219"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp219)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 347.4 255.27 L 355.92 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp220"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp220)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 372.94 146.65 L 372.94 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp221"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp221)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 372.94 218.45 L 372.94 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp222"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp222)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 368.69 115.35 L 377.2 115.35" style="fill:none"></path></g></g><g><clipPath id="pgfcp223"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp223)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 368.69 255.27 L 377.2 255.27" style="fill:none"></path></g></g><g><clipPath id="pgfcp224"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp224)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 394.23 166.9 L 394.23 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp225"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp225)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 394.23 211.09 L 394.23 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp226"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp226)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 389.97 122.71 L 398.48 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp227"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp227)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 389.97 262.64 L 398.48 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp228"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp228)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 415.51 211.09 L 415.51 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp229"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp229)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 415.51 247.91 L 415.51 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp230"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp230)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 411.25 166.9 L 419.77 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp231"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp231)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 411.25 262.64 L 419.77 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp232"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp232)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 436.79 211.09 L 436.79 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp233"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp233)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 436.79 247.91 L 436.79 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp234"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp234)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 432.54 159.54 L 441.05 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp235"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp235)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 432.54 262.64 L 441.05 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp236"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp236)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 458.07 225.81 L 458.07 196.36" style="fill:none"></path></g></g><g><clipPath id="pgfcp237"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp237)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 458.07 247.91 L 458.07 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp238"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp238)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 453.82 196.36 L 462.33 196.36" style="fill:none"></path></g></g><g><clipPath id="pgfcp239"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp239)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 453.82 262.64 L 462.33 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp240"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp240)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 479.36 225.81 L 479.36 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp241"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp241)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 479.36 255.27 L 479.36 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp242"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp242)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 475.1 181.63 L 483.62 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp243"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp243)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 475.1 262.64 L 483.62 262.64" style="fill:none"></path></g></g><g><clipPath id="pgfcp244"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp244)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 49.44 188.99 L 57.95 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp245"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp245)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 70.72 211.09 L 79.24 211.09" style="fill:none"></path></g></g><g><clipPath id="pgfcp246"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp246)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 92.01 188.99 L 100.52 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp247"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp247)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 113.29 122.71 L 121.8 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp248"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp248)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 134.57 188.99 L 143.09 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp249"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp249)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 155.86 122.71 L 164.37 122.71" style="fill:none"></path></g></g><g><clipPath id="pgfcp250"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp250)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 177.14 247.91 L 185.65 247.91" style="fill:none"></path></g></g><g><clipPath id="pgfcp251"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp251)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 198.42 218.45 L 206.94 218.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp252"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp252)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 219.71 188.99 L 228.22 188.99" style="fill:none"></path></g></g><g><clipPath id="pgfcp253"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp253)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 240.99 174.26 L 249.5 174.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp254"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp254)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 262.27 137.44 L 270.78 137.44" style="fill:none"></path></g></g><g><clipPath id="pgfcp255"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp255)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 283.55 152.17 L 292.07 152.17" style="fill:none"></path></g></g><g><clipPath id="pgfcp256"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp256)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 304.84 152.17 L 313.35 152.17" style="fill:none"></path></g></g><g><clipPath id="pgfcp257"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp257)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 326.12 159.54 L 334.63 159.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp258"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp258)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 347.4 166.9 L 355.92 166.9" style="fill:none"></path></g></g><g><clipPath id="pgfcp259"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp259)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 368.69 181.63 L 377.2 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp260"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp260)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 389.97 181.63 L 398.48 181.63" style="fill:none"></path></g></g><g><clipPath id="pgfcp261"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp261)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 411.25 233.18 L 419.77 233.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp262"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp262)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 432.54 240.54 L 441.05 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp263"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp263)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 453.82 240.54 L 462.33 240.54" style="fill:none"></path></g></g><g><clipPath id="pgfcp264"><path d="M 43.06 107.99 h 446.94 v 162.01 h -446.94 Z"></path></clipPath></g><g clip-path="url(#pgfcp264)"><g stroke-linecap="round" stroke="#656565" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 475.1 240.54 L 483.62 240.54" style="fill:none"></path></g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 43.06 107.99 L 43.06 270" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 490 107.99 L 490 270" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 43.06 107.99 L 490 107.99" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 43.06 270 L 490 270" style="fill:none"></path></g><g></g></g></g></svg>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure"><span id="S4.F2.3.1.1" class="ltx_text" style="font-size:90%;">Figure 2</span>: </span><span id="S4.F2.4.2" class="ltx_text" style="font-size:90%;">Distribution of median rankings of the mean squared error on the test set.</span></figcaption>
</figure>
<div id="S4.p1" class="ltx_para">
<p id="S4.p1.1" class="ltx_p">As in the original experiments for this benchmark suite <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib10" title="" class="ltx_ref">10</a>]</cite>, we first calculate the median (mean squared) error of the ten models of each regression problem. Then we rank all regression methods by their median error for each problem. Figure <a href="#S4.F1" title="Figure 1 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> and <a href="#S4.F2" title="Figure 2 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> show the distribution of ranks for each algorithm for training and test across all regression problems in the benchmark suite. E.g.  in Figure <a href="#S4.F1" title="Figure 1 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>, gradient boosting was for most problems the most accurate algorithm in training.</p>
</div>
<div id="S4.p2" class="ltx_para">
<p id="S4.p2.1" class="ltx_p">Figure <a href="#S4.F1" title="Figure 1 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> and <a href="#S4.F2" title="Figure 2 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> show the distribution of rankings of each algorithm’s median rank on the training set and test set. The optimization of nonlinear parameters in FFX NLS provided to more accurate models than FFX both in training and test. In comparison to other algorithms, FFX as well as FFX NLS perform better than linear models, which are on the right of Figure <a href="#S4.F1" title="Figure 1 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> (like linear regression or lasso regression). This is expected given both FFX methods can cover nonlinear dependencies and interactions in the data in contrast to purely linear methods. However, both algorithms perform worse than boosting methods or GP with NLS (GP CoOp) by Kommenda et al. <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib7" title="" class="ltx_ref">7</a>]</cite>. Also this is plausible, as both methods have a more powerful search algorithm and a larger hypothesis space than FFX methods with their restricted model structures.</p>
</div>
<div id="S4.p3" class="ltx_para">
<p id="S4.p3.1" class="ltx_p">To analyze the size of models, we count the number of syntactical symbols in a model. E.g. the model <math id="S4.p3.1.m1.1" class="ltx_Math" alttext="c_{0}+\log{x_{1}+c_{2}}" display="inline"><semantics id="S4.p3.1.m1.1a"><mrow id="S4.p3.1.m1.1.1" xref="S4.p3.1.m1.1.1.cmml"><msub id="S4.p3.1.m1.1.1.2" xref="S4.p3.1.m1.1.1.2.cmml"><mi id="S4.p3.1.m1.1.1.2.2" xref="S4.p3.1.m1.1.1.2.2.cmml">c</mi><mn id="S4.p3.1.m1.1.1.2.3" xref="S4.p3.1.m1.1.1.2.3.cmml">0</mn></msub><mo id="S4.p3.1.m1.1.1.1" xref="S4.p3.1.m1.1.1.1.cmml">+</mo><mrow id="S4.p3.1.m1.1.1.3" xref="S4.p3.1.m1.1.1.3.cmml"><mi id="S4.p3.1.m1.1.1.3.1" xref="S4.p3.1.m1.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="S4.p3.1.m1.1.1.3a" xref="S4.p3.1.m1.1.1.3.cmml">⁡</mo><msub id="S4.p3.1.m1.1.1.3.2" xref="S4.p3.1.m1.1.1.3.2.cmml"><mi id="S4.p3.1.m1.1.1.3.2.2" xref="S4.p3.1.m1.1.1.3.2.2.cmml">x</mi><mn id="S4.p3.1.m1.1.1.3.2.3" xref="S4.p3.1.m1.1.1.3.2.3.cmml">1</mn></msub></mrow><mo id="S4.p3.1.m1.1.1.1a" xref="S4.p3.1.m1.1.1.1.cmml">+</mo><msub id="S4.p3.1.m1.1.1.4" xref="S4.p3.1.m1.1.1.4.cmml"><mi id="S4.p3.1.m1.1.1.4.2" xref="S4.p3.1.m1.1.1.4.2.cmml">c</mi><mn id="S4.p3.1.m1.1.1.4.3" xref="S4.p3.1.m1.1.1.4.3.cmml">2</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S4.p3.1.m1.1b"><apply id="S4.p3.1.m1.1.1.cmml" xref="S4.p3.1.m1.1.1"><plus id="S4.p3.1.m1.1.1.1.cmml" xref="S4.p3.1.m1.1.1.1"></plus><apply id="S4.p3.1.m1.1.1.2.cmml" xref="S4.p3.1.m1.1.1.2"><csymbol cd="ambiguous" id="S4.p3.1.m1.1.1.2.1.cmml" xref="S4.p3.1.m1.1.1.2">subscript</csymbol><ci id="S4.p3.1.m1.1.1.2.2.cmml" xref="S4.p3.1.m1.1.1.2.2">𝑐</ci><cn type="integer" id="S4.p3.1.m1.1.1.2.3.cmml" xref="S4.p3.1.m1.1.1.2.3">0</cn></apply><apply id="S4.p3.1.m1.1.1.3.cmml" xref="S4.p3.1.m1.1.1.3"><log id="S4.p3.1.m1.1.1.3.1.cmml" xref="S4.p3.1.m1.1.1.3.1"></log><apply id="S4.p3.1.m1.1.1.3.2.cmml" xref="S4.p3.1.m1.1.1.3.2"><csymbol cd="ambiguous" id="S4.p3.1.m1.1.1.3.2.1.cmml" xref="S4.p3.1.m1.1.1.3.2">subscript</csymbol><ci id="S4.p3.1.m1.1.1.3.2.2.cmml" xref="S4.p3.1.m1.1.1.3.2.2">𝑥</ci><cn type="integer" id="S4.p3.1.m1.1.1.3.2.3.cmml" xref="S4.p3.1.m1.1.1.3.2.3">1</cn></apply></apply><apply id="S4.p3.1.m1.1.1.4.cmml" xref="S4.p3.1.m1.1.1.4"><csymbol cd="ambiguous" id="S4.p3.1.m1.1.1.4.1.cmml" xref="S4.p3.1.m1.1.1.4">subscript</csymbol><ci id="S4.p3.1.m1.1.1.4.2.cmml" xref="S4.p3.1.m1.1.1.4.2">𝑐</ci><cn type="integer" id="S4.p3.1.m1.1.1.4.3.cmml" xref="S4.p3.1.m1.1.1.4.3">2</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p3.1.m1.1c">c_{0}+\log{x_{1}+c_{2}}</annotation></semantics></math> has a complexity of six. We use this measure because it takes the added scaling terms within function arguments in FFX NLS into account. Figure <a href="#S4.F3.sf1" title="In Figure 3 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3(a)</span></a> shows that both the size of models and the number of base function within models produced by FFX NLS and FFX are similar. FFX NLS models are just slightly shorter.</p>
</div>
<figure id="S4.F3" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="S4.F3.sf1" class="ltx_figure ltx_figure_panel ltx_align_center"><svg id="S4.F3.sf1.pic1" class="ltx_picture ltx_centering" height="180.6" overflow="visible" version="1.1" width="220.68"><g transform="translate(0,180.6) matrix(1 0 0 -1 0 0)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><clipPath id="pgfcp265"><path d="M 0 0 h 220.12 v 180.05 h -220.12 Z"></path></clipPath><g clip-path="url(#pgfcp265)"><g></g><g stroke-linecap="butt" fill="#FFFFFF" stroke="#000000" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-opacity="0.000000" stroke-width="0.0pt"><path d="M 48.46 22.07 L 220.12 22.07 L 220.12 180.05 L 48.46 180.05 L 48.46 22.07 Z" style="stroke:none"></path></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 61.49 2.89)"><foreignObject width="59.77" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F3.sf1.pic1.5.5.5.5.1.1" class="ltx_text" style="color:#262626;">FFX NLS</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 162.99 2.89)"><foreignObject width="28.44" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F3.sf1.pic1.6.6.6.6.1.1" class="ltx_text" style="color:#262626;">FFX</span></foreignObject></g><g></g><g><clipPath id="pgfcp266"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp266)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 48.46 26.88 L 220.12 26.88" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 31.79 22.06)"><foreignObject width="6.92" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{0}" display="inline"><semantics id="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1">0</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1c">\displaystyle{0}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp267"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp267)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 48.46 66.34 L 220.12 66.34" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 24.85 61.51)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{50}" display="inline"><semantics id="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml">50</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1">50</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1c">\displaystyle{50}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp268"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp268)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 48.46 105.79 L 220.12 105.79" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 17.9 100.97)"><foreignObject width="20.76" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{100}" display="inline"><semantics id="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml">100</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1">100</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1c">\displaystyle{100}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp269"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp269)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 48.46 145.25 L 220.12 145.25" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 17.9 140.42)"><foreignObject width="20.76" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{150}" display="inline"><semantics id="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml">150</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1">150</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1c">\displaystyle{150}</annotation></semantics></math></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 9.66 61.08)"><foreignObject width="79.56" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F3.sf1.pic1.7.7.7.7.1.1" class="ltx_text" style="color:#262626;">Model length</span></foreignObject></g><g></g><g><clipPath id="pgfcp270"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp270)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 57.04 50.75 L 125.71 50.75 L 125.71 58.33 L 108.54 60.81 L 125.71 63.3 L 125.71 91.39 L 57.04 91.39 L 57.04 63.3 L 74.21 60.81 L 57.04 58.33 L 57.04 50.75 L 57.04 50.75 Z"></path></g></g><g><clipPath id="pgfcp271"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp271)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 142.87 57.66 L 211.54 57.66 L 211.54 65.06 L 194.37 67.92 L 211.54 70.77 L 211.54 104.21 L 142.87 104.21 L 142.87 70.77 L 160.04 67.92 L 142.87 65.06 L 142.87 57.66 L 142.87 57.66 Z"></path></g></g><g><clipPath id="pgfcp272"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp272)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 91.37 50.75 L 91.37 37.14" style="fill:none"></path></g></g><g><clipPath id="pgfcp273"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp273)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 91.37 91.39 L 91.37 137.36" style="fill:none"></path></g></g><g><clipPath id="pgfcp274"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp274)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 74.21 37.14 L 108.54 37.14" style="fill:none"></path></g></g><g><clipPath id="pgfcp275"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp275)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 74.21 137.36 L 108.54 137.36" style="fill:none"></path></g></g><g><clipPath id="pgfcp276"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp276)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 177.21 57.66 L 177.21 29.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp277"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp277)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 177.21 104.21 L 177.21 172.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp278"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp278)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 160.04 29.25 L 194.37 29.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp279"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp279)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 160.04 172.87 L 194.37 172.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp280"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp280)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 74.21 60.81 L 108.54 60.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp281"><path d="M 48.46 22.07 h 171.67 v 157.98 h -171.67 Z"></path></clipPath></g><g clip-path="url(#pgfcp281)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 160.04 67.92 L 194.37 67.92" style="fill:none"></path></g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 48.46 22.07 L 48.46 180.05" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 220.12 22.07 L 220.12 180.05" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 48.46 22.07 L 220.12 22.07" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 48.46 180.05 L 220.12 180.05" style="fill:none"></path></g><g></g></g></g></svg>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S4.F3.sf1.2.1.1" class="ltx_text" style="font-size:90%;">(a)</span> </span><span id="S4.F3.sf1.3.2" class="ltx_text" style="font-size:90%;">Length per model.</span></figcaption>
</figure>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<figure id="S4.F3.sf2" class="ltx_figure ltx_figure_panel ltx_align_center"><svg id="S4.F3.sf2.pic1" class="ltx_picture ltx_centering" height="180.6" overflow="visible" version="1.1" width="220.65"><g transform="translate(0,180.6) matrix(1 0 0 -1 0 0)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><clipPath id="pgfcp282"><path d="M 0 0 h 220.1 v 180.05 h -220.1 Z"></path></clipPath><g clip-path="url(#pgfcp282)"><g></g><g stroke-linecap="butt" fill="#FFFFFF" stroke="#000000" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-opacity="0.000000" stroke-width="0.0pt"><path d="M 41.51 22.07 L 220.1 22.07 L 220.1 180.05 L 41.51 180.05 L 41.51 22.07 Z" style="stroke:none"></path></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 56.27 2.89)"><foreignObject width="59.77" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F3.sf2.pic1.5.5.5.5.1.1" class="ltx_text" style="color:#262626;">FFX NLS</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 161.23 2.89)"><foreignObject width="28.44" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F3.sf2.pic1.6.6.6.6.1.1" class="ltx_text" style="color:#262626;">FFX</span></foreignObject></g><g></g><g><clipPath id="pgfcp283"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp283)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 41.51 33.88 L 220.1 33.88" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 24.85 29.06)"><foreignObject width="6.92" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{0}" display="inline"><semantics id="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1">0</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1c">\displaystyle{0}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp284"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp284)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 41.51 80.21 L 220.1 80.21" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 17.9 75.39)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10}" display="inline"><semantics id="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml">10</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1">10</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1c">\displaystyle{10}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp285"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp285)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 41.51 126.54 L 220.1 126.54" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 17.9 121.71)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{20}" display="inline"><semantics id="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml">20</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1">20</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1c">\displaystyle{20}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp286"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp286)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 41.51 172.87 L 220.1 172.87" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 17.9 168.04)"><foreignObject width="13.84" height="8.92" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{30}" display="inline"><semantics id="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1a"><mn mathcolor="#262626" id="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1" xref="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml">30</mn><annotation-xml encoding="MathML-Content" id="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1b"><cn type="integer" id="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1">30</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.F3.sf2.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1c">\displaystyle{30}</annotation></semantics></math></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 12.35 49.48)"><foreignObject width="103.16" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F3.sf2.pic1.7.7.7.7.1.1" class="ltx_text" style="color:#262626;">Nubmer of terms</span></foreignObject></g><g></g><g><clipPath id="pgfcp287"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp287)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 50.44 61.68 L 121.88 61.68 L 121.88 76.52 L 104.02 80.21 L 121.88 83.9 L 121.88 121.9 L 50.44 121.9 L 50.44 83.9 L 68.3 80.21 L 50.44 76.52 L 50.44 61.68 L 50.44 61.68 Z"></path></g></g><g><clipPath id="pgfcp288"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp288)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 139.73 70.94 L 211.17 70.94 L 211.17 76.8 L 193.31 80.21 L 211.17 83.62 L 211.17 126.54 L 139.73 126.54 L 139.73 83.62 L 157.59 80.21 L 139.73 76.8 L 139.73 70.94 L 139.73 70.94 Z"></path></g></g><g><clipPath id="pgfcp289"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp289)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 86.16 61.68 L 86.16 47.78" style="fill:none"></path></g></g><g><clipPath id="pgfcp290"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp290)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 86.16 121.9 L 86.16 172.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp291"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp291)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 68.3 47.78 L 104.02 47.78" style="fill:none"></path></g></g><g><clipPath id="pgfcp292"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp292)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 68.3 172.87 L 104.02 172.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp293"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp293)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 175.45 70.94 L 175.45 29.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp294"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp294)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 175.45 126.54 L 175.45 172.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp295"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp295)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 157.59 29.25 L 193.31 29.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp296"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp296)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 157.59 172.87 L 193.31 172.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp297"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp297)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 68.3 80.21 L 104.02 80.21" style="fill:none"></path></g></g><g><clipPath id="pgfcp298"><path d="M 41.51 22.07 h 178.59 v 157.98 h -178.59 Z"></path></clipPath></g><g clip-path="url(#pgfcp298)"><g stroke-linecap="round" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 157.59 80.21 L 193.31 80.21" style="fill:none"></path></g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 41.51 22.07 L 41.51 180.05" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 220.1 22.07 L 220.1 180.05" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 41.51 22.07 L 220.1 22.07" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 41.51 180.05 L 220.1 180.05" style="fill:none"></path></g><g></g></g></g></svg>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S4.F3.sf2.2.1.1" class="ltx_text" style="font-size:90%;">(b)</span> </span><span id="S4.F3.sf2.3.2" class="ltx_text" style="font-size:90%;">Number of base functions per model.</span></figcaption>
</figure>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S4.F3.2.1.1" class="ltx_text" style="font-size:90%;">Figure 3</span>: </span><span id="S4.F3.3.2" class="ltx_text" style="font-size:90%;">Distribution complexity of all models from FFX and FFX NLS.</span></figcaption>
</figure>
<figure id="S4.F4" class="ltx_figure"><svg id="S4.F4.1.pic1" class="ltx_picture ltx_centering" height="251.77" overflow="visible" version="1.1" width="480.56"><g transform="translate(0,251.77) matrix(1 0 0 -1 0 0)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><clipPath id="pgfcp299"><path d="M 0 0 h 480 v 250 h -480 Z"></path></clipPath><g clip-path="url(#pgfcp299)"><g></g><g stroke-linecap="butt" fill="#FFFFFF" stroke="#000000" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-opacity="0.000000" stroke-width="0.0pt"><path d="M 49.29 107.99 L 480 107.99 L 480 245.18 L 49.29 245.18 L 49.29 107.99 Z" style="stroke:none"></path></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 63.01 29.78)"><foreignObject width="59.77" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.6.6.6.6.1.1" class="ltx_text ltx_font_bold" style="color:#262626;">FFX NLS</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 83.52 66.09)"><foreignObject width="28.44" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.7.7.7.7.1.1" class="ltx_text ltx_font_bold" style="color:#262626;">FFX</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 104.03 77.91)"><foreignObject width="20.28" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.8.8.8.8.1.1" class="ltx_text" style="color:#262626;">GP</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 124.54 37.79)"><foreignObject width="60.25" height="12.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.9.9.9.9.1.1" class="ltx_text" style="color:#262626;">GP CoOp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 145.05 59.39)"><foreignObject width="38.72" height="9.46" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.10.10.10.10.1.1" class="ltx_text" style="color:#262626;">OSGP</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 165.56 19.27)"><foreignObject width="78.7" height="12.15" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.11.11.11.11.1.1" class="ltx_text" style="color:#262626;">OSGP CoOp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 186.07 71.18)"><foreignObject width="26.98" height="8.65" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.12.12.12.12.1.1" class="ltx_text" style="color:#262626;">gsgp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 206.58 79.36)"><foreignObject width="18.83" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.13.13.13.13.1.1" class="ltx_text" style="color:#262626;">afp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 227.1 66.59)"><foreignObject width="31.56" height="8.65" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.14.14.14.14.1.1" class="ltx_text" style="color:#262626;">mrgp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 247.61 67.01)"><foreignObject width="31.13" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.15.15.15.15.1.1" class="ltx_text" style="color:#262626;">eplex</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 268.12 43.87)"><foreignObject width="54.2" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.16.16.16.16.1.1" class="ltx_text" style="color:#262626;">eplex-1m</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 288.63 50.73)"><foreignObject width="46.97" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.17.17.17.17.1.1" class="ltx_text" style="color:#262626;">xgboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 309.14 37.96)"><foreignObject width="59.69" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.18.18.18.18.1.1" class="ltx_text" style="color:#262626;">gradboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 329.65 75.12)"><foreignObject width="23.06" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.19.19.19.19.1.1" class="ltx_text" style="color:#262626;">mlp</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 350.16 88.58)"><foreignObject width="9.65" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.20.20.20.20.1.1" class="ltx_text" style="color:#262626;">rf</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 370.67 27.2)"><foreignObject width="70.8" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.21.21.21.21.1.1" class="ltx_text" style="color:#262626;">kernel-ridge</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 391.18 43.4)"><foreignObject width="54.27" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.22.22.22.22.1.1" class="ltx_text" style="color:#262626;">adaboost</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 411.69 43.21)"><foreignObject width="54.85" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.23.23.23.23.1.1" class="ltx_text" style="color:#262626;">lasso-lars</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 432.2 41.4)"><foreignObject width="56.65" height="9.61" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.24.24.24.24.1.1" class="ltx_text" style="color:#262626;">linear-svr</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 452.71 0)"><foreignObject width="97.9" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.25.25.25.25.1.1" class="ltx_text" style="color:#262626;">linear-regression</span></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 473.22 13.85)"><foreignObject width="84.1" height="12.3" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.26.26.26.26.1.1" class="ltx_text" style="color:#262626;">sgd-regression</span></foreignObject></g><g></g><g><clipPath id="pgfcp300"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp300)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.29 122.41 L 480 122.41" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 117.59)"><foreignObject width="17.71" height="11.41" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10^{0}}" display="inline"><semantics id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1a"><msup id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml"><mn mathcolor="#262626" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2.cmml">10</mn><mn mathcolor="#262626" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1b"><apply id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1">superscript</csymbol><cn type="integer" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2">10</cn><cn type="integer" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.1.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1c">\displaystyle{10^{0}}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp301"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp301)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.29 158.61 L 480 158.61" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 153.79)"><foreignObject width="17.71" height="11.41" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10^{1}}" display="inline"><semantics id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1a"><msup id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml"><mn mathcolor="#262626" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.2" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.2.cmml">10</mn><mn mathcolor="#262626" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.3" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1b"><apply id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1">superscript</csymbol><cn type="integer" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.2">10</cn><cn type="integer" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.1.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.m1.1c">\displaystyle{10^{1}}</annotation></semantics></math><!-- %****␣runtime.pgf␣Line␣225␣**** --></foreignObject></g><g></g><g><clipPath id="pgfcp302"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp302)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.29 187.47 L 480 187.47" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 182.65)"><foreignObject width="17.71" height="11.41" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10^{2}}" display="inline"><semantics id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1a"><msup id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml"><mn mathcolor="#262626" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.2" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.2.cmml">10</mn><mn mathcolor="#262626" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.3" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.3.cmml">2</mn></msup><annotation-xml encoding="MathML-Content" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1b"><apply id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1">superscript</csymbol><cn type="integer" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.2">10</cn><cn type="integer" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1.1.3">2</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.1.pic1.3.3.3.3.3.3.3.3.3.3.3.3.1.1.1.1.1.1.m1.1c">\displaystyle{10^{2}}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp303"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp303)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.29 216.32 L 480 216.32" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 211.5)"><foreignObject width="17.71" height="11.41" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10^{3}}" display="inline"><semantics id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1a"><msup id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml"><mn mathcolor="#262626" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.2" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.2.cmml">10</mn><mn mathcolor="#262626" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.3" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.3.cmml">3</mn></msup><annotation-xml encoding="MathML-Content" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1b"><apply id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1">superscript</csymbol><cn type="integer" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.2">10</cn><cn type="integer" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.1.pic1.4.4.4.4.4.4.4.4.4.4.4.4.1.1.1.1.1.1.m1.1c">\displaystyle{10^{3}}</annotation></semantics></math></foreignObject></g><g></g><g><clipPath id="pgfcp304"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp304)"><g stroke-linecap="round" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="0.803pt"><path d="M 49.29 245.18 L 480 245.18" style="fill:none"></path></g></g><g stroke="#262626" fill="#262626" transform="matrix(1.0 0.0 0.0 1.0 19.44 240.35)"><foreignObject width="17.71" height="11.41" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\displaystyle{10^{4}}" display="inline"><semantics id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1a"><msup id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.cmml"><mn mathcolor="#262626" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.2" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.2.cmml">10</mn><mn mathcolor="#262626" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.3" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.3.cmml">4</mn></msup><annotation-xml encoding="MathML-Content" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1b"><apply id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.cmml" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1">superscript</csymbol><cn type="integer" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.2">10</cn><cn type="integer" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1.1.3">4</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.1.pic1.5.5.5.5.5.5.5.5.5.5.5.5.1.1.1.1.1.1.m1.1c">\displaystyle{10^{4}}</annotation></semantics></math></foreignObject></g><g></g><g stroke="#262626" fill="#262626" transform="matrix(0.0 1.0 -1.0 0.0 10.43 123.17)"><foreignObject width="107.2" height="13.84" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S4.F4.1.pic1.27.27.27.27.1.1" class="ltx_text" style="color:#262626;">Runtime (in secs)</span></foreignObject></g><g></g><g><clipPath id="pgfcp305"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp305)"><g stroke-linecap="butt" fill="#A9A9A9" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 51.34 106.38 L 67.75 106.38 L 67.75 142.82 L 51.34 142.82 L 51.34 106.38 Z"></path></g></g><g><clipPath id="pgfcp306"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp306)"><g stroke-linecap="butt" fill="#A9A9A9" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 71.85 106.38 L 88.26 106.38 L 88.26 108.43 L 71.85 108.43 L 71.85 106.38 Z"></path></g></g><g><clipPath id="pgfcp307"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp307)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 92.36 106.38 L 108.77 106.38 L 108.77 178.34 L 92.36 178.34 L 92.36 106.38 Z"></path></g></g><g><clipPath id="pgfcp308"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp308)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 112.87 106.38 L 129.28 106.38 L 129.28 228.46 L 112.87 228.46 L 112.87 106.38 Z"></path></g></g><g><clipPath id="pgfcp309"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp309)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 133.38 106.38 L 149.79 106.38 L 149.79 179.61 L 133.38 179.61 L 133.38 106.38 Z"></path></g></g><g><clipPath id="pgfcp310"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp310)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 153.89 106.38 L 170.3 106.38 L 170.3 229.85 L 153.89 229.85 L 153.89 106.38 Z"></path></g></g><g><clipPath id="pgfcp311"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp311)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 174.4 106.38 L 190.81 106.38 L 190.81 189.25 L 174.4 189.25 L 174.4 106.38 Z"></path></g></g><g><clipPath id="pgfcp312"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp312)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 194.91 106.38 L 211.32 106.38 L 211.32 191.01 L 194.91 191.01 L 194.91 106.38 Z"></path></g></g><g><clipPath id="pgfcp313"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp313)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 215.42 106.38 L 231.83 106.38 L 231.83 230.71 L 215.42 230.71 L 215.42 106.38 Z"></path></g></g><g><clipPath id="pgfcp314"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp314)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 235.93 106.38 L 252.34 106.38 L 252.34 194.73 L 235.93 194.73 L 235.93 106.38 Z"></path></g></g><g><clipPath id="pgfcp315"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp315)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 256.44 106.38 L 272.85 106.38 L 272.85 228.65 L 256.44 228.65 L 256.44 106.38 Z"></path></g></g><g><clipPath id="pgfcp316"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp316)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 276.95 106.38 L 293.36 106.38 L 293.36 122.51 L 276.95 122.51 L 276.95 106.38 Z"></path></g></g><g><clipPath id="pgfcp317"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp317)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 297.46 106.38 L 313.87 106.38 L 313.87 116.63 L 297.46 116.63 L 297.46 106.38 Z"></path></g></g><g><clipPath id="pgfcp318"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp318)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 317.97 106.38 L 334.38 106.38 L 334.38 120.43 L 317.97 120.43 L 317.97 106.38 Z"></path></g></g><g><clipPath id="pgfcp319"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp319)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 338.48 106.38 L 354.89 106.38 L 354.89 143.59 L 338.48 143.59 L 338.48 106.38 Z"></path></g></g><g><clipPath id="pgfcp320"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp320)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 358.99 106.38 L 375.4 106.38 L 375.4 107.49 L 358.99 107.49 L 358.99 106.38 Z"></path></g></g><g><clipPath id="pgfcp321"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp321)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 379.5 106.38 L 395.91 106.38 L 395.91 144.82 L 379.5 144.82 L 379.5 106.38 Z"></path></g></g><g><clipPath id="pgfcp322"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp322)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 400.01 106.38 L 416.42 106.38 L 416.42 106.64 L 400.01 106.64 L 400.01 106.38 Z"></path></g></g><g><clipPath id="pgfcp323"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp323)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 420.52 106.38 L 436.93 106.38 L 436.93 106.78 L 420.52 106.78 L 420.52 106.38 Z"></path></g></g><g><clipPath id="pgfcp324"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp324)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 441.03 106.38 L 457.44 106.38 L 457.44 106.85 L 441.03 106.85 L 441.03 106.38 Z"></path></g></g><g><clipPath id="pgfcp325"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp325)"><g stroke-linecap="butt" fill="#D3D3D3" stroke="#7F7F7F" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="1.00375pt"><path d="M 461.54 106.38 L 477.95 106.38 L 477.95 106.54 L 461.54 106.54 L 461.54 106.38 Z"></path></g></g><g><clipPath id="pgfcp326"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp326)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 59.54 130.69 L 59.54 148.12" style="fill:none"></path></g></g><g><clipPath id="pgfcp327"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp327)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 58.52 130.69 L 60.57 130.69" style="fill:none"></path></g></g><g><clipPath id="pgfcp328"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp328)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 58.52 148.12 L 60.57 148.12" style="fill:none"></path></g></g><g><clipPath id="pgfcp329"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp329)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 80.05 107.49 L 80.05 109.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp330"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp330)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 79.03 107.49 L 81.08 107.49" style="fill:none"></path></g></g><g><clipPath id="pgfcp331"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp331)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 79.03 109.45 L 81.08 109.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp332"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp332)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 100.56 176.66 L 100.56 179.88" style="fill:none"></path></g></g><g><clipPath id="pgfcp333"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp333)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 99.54 176.66 L 101.59 176.66" style="fill:none"></path></g></g><g><clipPath id="pgfcp334"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp334)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 99.54 179.88 L 101.59 179.88" style="fill:none"></path></g></g><g><clipPath id="pgfcp335"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp335)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 121.07 225.83 L 121.07 230.62" style="fill:none"></path></g></g><g><clipPath id="pgfcp336"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp336)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 120.05 225.83 L 122.1 225.83" style="fill:none"></path></g></g><g><clipPath id="pgfcp337"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp337)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 120.05 230.62 L 122.1 230.62" style="fill:none"></path></g></g><g><clipPath id="pgfcp338"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp338)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 141.58 177.69 L 141.58 181.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp339"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp339)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 140.56 177.69 L 142.61 177.69" style="fill:none"></path></g></g><g><clipPath id="pgfcp340"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp340)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 140.56 181.25 L 142.61 181.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp341"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp341)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 162.09 226.74 L 162.09 232.32" style="fill:none"></path></g></g><g><clipPath id="pgfcp342"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp342)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 161.07 226.74 L 163.12 226.74" style="fill:none"></path></g></g><g><clipPath id="pgfcp343"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp343)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 161.07 232.32 L 163.12 232.32" style="fill:none"></path></g></g><g><clipPath id="pgfcp344"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp344)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 182.6 188.29 L 182.6 190.21" style="fill:none"></path></g></g><g><clipPath id="pgfcp345"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp345)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 181.58 188.29 L 183.63 188.29" style="fill:none"></path></g></g><g><clipPath id="pgfcp346"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp346)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 181.58 190.21 L 183.63 190.21" style="fill:none"></path></g></g><g><clipPath id="pgfcp347"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp347)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 203.11 190.55 L 203.11 191.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp348"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp348)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 202.09 190.55 L 204.14 190.55" style="fill:none"></path></g></g><g><clipPath id="pgfcp349"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp349)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 202.09 191.45 L 204.14 191.45" style="fill:none"></path></g></g><g><clipPath id="pgfcp350"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp350)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 223.62 230.29 L 223.62 231.03" style="fill:none"></path></g></g><g><clipPath id="pgfcp351"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp351)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 222.6 230.29 L 224.65 230.29" style="fill:none"></path></g></g><g><clipPath id="pgfcp352"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp352)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 222.6 231.03 L 224.65 231.03" style="fill:none"></path></g></g><g><clipPath id="pgfcp353"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp353)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 244.13 194.18 L 244.13 195.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp354"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp354)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 243.11 194.18 L 245.16 194.18" style="fill:none"></path></g></g><g><clipPath id="pgfcp355"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp355)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 243.11 195.26 L 245.16 195.26" style="fill:none"></path></g></g><g><clipPath id="pgfcp356"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp356)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 264.64 228.13 L 264.64 229.23" style="fill:none"></path></g></g><g><clipPath id="pgfcp357"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp357)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 263.62 228.13 L 265.67 228.13" style="fill:none"></path></g></g><g><clipPath id="pgfcp358"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp358)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 263.62 229.23 L 265.67 229.23" style="fill:none"></path></g></g><g><clipPath id="pgfcp359"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp359)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 285.15 118.53 L 285.15 127.19" style="fill:none"></path></g></g><g><clipPath id="pgfcp360"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp360)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 284.13 118.53 L 286.18 118.53" style="fill:none"></path></g></g><g><clipPath id="pgfcp361"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp361)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 284.13 127.19 L 286.18 127.19" style="fill:none"></path></g></g><g><clipPath id="pgfcp362"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp362)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 305.66 115.12 L 305.66 118.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp363"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp363)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 304.64 115.12 L 306.69 115.12" style="fill:none"></path></g></g><g><clipPath id="pgfcp364"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp364)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 304.64 118.25 L 306.69 118.25" style="fill:none"></path></g></g><g><clipPath id="pgfcp365"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp365)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 326.17 118.86 L 326.17 122.02" style="fill:none"></path></g></g><g><clipPath id="pgfcp366"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp366)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 325.15 118.86 L 327.2 118.86" style="fill:none"></path></g></g><g><clipPath id="pgfcp367"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp367)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 325.15 122.02 L 327.2 122.02" style="fill:none"></path></g></g><g><clipPath id="pgfcp368"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp368)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 346.68 142.28 L 346.68 144.97" style="fill:none"></path></g></g><g><clipPath id="pgfcp369"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp369)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 345.66 142.28 L 347.71 142.28" style="fill:none"></path></g></g><g><clipPath id="pgfcp370"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp370)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 345.66 144.97 L 347.71 144.97" style="fill:none"></path></g></g><g><clipPath id="pgfcp371"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp371)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 367.19 107.24 L 367.19 107.76" style="fill:none"></path></g></g><g><clipPath id="pgfcp372"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp372)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 366.17 107.24 L 368.22 107.24" style="fill:none"></path></g></g><g><clipPath id="pgfcp373"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp373)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 366.17 107.76 L 368.22 107.76" style="fill:none"></path></g></g><g><clipPath id="pgfcp374"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp374)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 387.7 142.81 L 387.7 146.59" style="fill:none"></path></g></g><g><clipPath id="pgfcp375"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp375)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 386.68 142.81 L 388.73 142.81" style="fill:none"></path></g></g><g><clipPath id="pgfcp376"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp376)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 386.68 146.59 L 388.73 146.59" style="fill:none"></path></g></g><g><clipPath id="pgfcp377"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp377)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 408.21 106.62 L 408.21 106.67" style="fill:none"></path></g></g><g><clipPath id="pgfcp378"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp378)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 407.19 106.62 L 409.24 106.62" style="fill:none"></path></g></g><g><clipPath id="pgfcp379"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp379)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 407.19 106.67 L 409.24 106.67" style="fill:none"></path></g></g><g><clipPath id="pgfcp380"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp380)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 428.72 106.69 L 428.72 106.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp381"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp381)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 427.7 106.69 L 429.75 106.69" style="fill:none"></path></g></g><g><clipPath id="pgfcp382"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp382)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 427.7 106.87 L 429.75 106.87" style="fill:none"></path></g></g><g><clipPath id="pgfcp383"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp383)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 449.23 106.79 L 449.23 106.92" style="fill:none"></path></g></g><g><clipPath id="pgfcp384"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp384)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 448.21 106.79 L 450.26 106.79" style="fill:none"></path></g></g><g><clipPath id="pgfcp385"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp385)"><g stroke-linecap="round" stroke="#424242" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="round" stroke-width="1.00375pt"><path d="M 448.21 106.92 L 450.26 106.92" style="fill:none"></path></g></g><g><clipPath id="pgfcp386"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp386)"><g stroke-linecap="round"></g></g><g><clipPath id="pgfcp387"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp387)"><g stroke-linecap="round"></g></g><g><clipPath id="pgfcp388"><path d="M 49.29 107.99 h 430.71 v 137.19 h -430.71 Z"></path></clipPath></g><g clip-path="url(#pgfcp388)"><g stroke-linecap="round"></g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 49.29 107.99 L 49.29 245.18" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 480 107.99 L 480 245.18" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 49.29 107.99 L 480 107.99" style="fill:none"></path></g><g></g><g stroke-linecap="rect" stroke="#CCCCCC" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linejoin="miter" stroke-width="0.803pt"><path d="M 49.29 245.18 L 480 245.18" style="fill:none"></path></g><g></g></g></g></svg>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure"><span id="S4.F4.3.1.1" class="ltx_text" style="font-size:90%;">Figure 4</span>: </span><span id="S4.F4.4.2" class="ltx_text" style="font-size:90%;">Median runtime of all algorithms.</span></figcaption>
</figure>
<div id="S4.p4" class="ltx_para">
<p id="S4.p4.1" class="ltx_p">Figure <a href="#S4.F4" title="Figure 4 ‣ 4 Results ‣ Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a> shows the median runtime across all problems per algorithm. While one FFX run takes less than a second for most problems, the runtime of FFX NLS ranges between one and a few seconds. Although the runtime increase is relatively large, it is still feasible to perform large grid searches in reasonable amount of time with FFX NLS. Both algorithms beat some GP-based algorithms and are on a similar level as boosting algorithms.</p>
</div>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Conclusion</h2>

<div id="S5.p1" class="ltx_para">
<p id="S5.p1.1" class="ltx_p">We proposed an extension of the model structure of the FFX algorithm. We added nonlinear scaling parameter which were optimized using the variable projection algorithm by Krogh <cite class="ltx_cite ltx_citemacro_cite">[<a href="#bib.bib8" title="" class="ltx_ref">8</a>]</cite> and separate base function selection, which we called FFX NLS. We achieved a large improvement in test accuracy in comparison to plain FFX on the PennML benchmark suite while providing models of similar complexity. Although the runtime of FFX NLS is higher than the one of FFX, it still finishes training within seconds.</p>
</div>
<div id="S5.p2" class="ltx_para">
<p id="S5.p2.1" class="ltx_p">However, FFX and FFX NLS still perform worse than many symbolic regression algorithms. Big advantages of FFX NLS towards GP-based algorithms are its short runtime, the low number of hyperparameters and the simple, comprehensible model structure. Compared to boosting algorithms, FFX NLS performs worse in accuracy but similar in runtime. However, boosting algorithms are considered black box methods and allow no readability of its models.</p>
</div>
<div id="S5.p3" class="ltx_para">
<p id="S5.p3.1" class="ltx_p">To sum it up, FFX is a promising tool for quick data exploration. It identifies interactions as well as nonlinear relations with simple hyperparameter configuration and quick execution. However, further improvements regarding accuracy are needed. Potential improvements are the combination of regularization and parameter optimization, as this are separate steps right now that deliberately ignore dependencies between linear and nonlinear parameters.</p>
</div>
<section id="S5.SS0.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.0.1 </span>Acknowledgements</h4>

<div id="S5.SS0.SSS1.p1" class="ltx_para">
<p id="S5.SS0.SSS1.p1.1" class="ltx_p">The authors gratefully acknowledge support by the Christian Doppler Research Association and the Federal Ministry for Digital and Economic Affairs within the <em id="S5.SS0.SSS1.p1.1.1" class="ltx_emph ltx_font_italic">Josef Ressel Center for Symbolic Regression</em>.</p>
</div>
</section>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[1]</span>
<span class="ltx_bibblock">
Burlacu, B., Kronberger, G., Kommenda, M.: Operon c++ an efficient genetic
programming framework for symbolic regression. In: Proceedings of the 2020
Genetic and Evolutionary Computation Conference Companion. pp. 1562–1570
(2020)

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[2]</span>
<span class="ltx_bibblock">
Chen, G.Y., Gan, M., Chen, C.P., Li, H.X.: A regularized variable projection
algorithm for separable nonlinear least-squares problems. IEEE Transactions
on Automatic Control <span id="bib.bib2.1.1" class="ltx_text ltx_font_bold">64</span>(2), 526–537 (2018)

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[3]</span>
<span class="ltx_bibblock">
de França, F.O.: A greedy search tree heuristic for symbolic regression.
Information Sciences <span id="bib.bib3.1.1" class="ltx_text ltx_font_bold">442</span>, 18–32 (2018)

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[4]</span>
<span class="ltx_bibblock">
Friedman, J.H.: The elements of statistical learning: Data mining, inference,
and prediction. springer open (2017)

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[5]</span>
<span class="ltx_bibblock">
Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear
least squares problems whose variables separate. SIAM Journal on numerical
analysis <span id="bib.bib5.1.1" class="ltx_text ltx_font_bold">10</span>(2), 413–432 (1973)

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[6]</span>
<span class="ltx_bibblock">
Kammerer, L., Kronberger, G., Burlacu, B., Winkler, S.M., Kommenda, M.,
Affenzeller, M.: Symbolic regression by exhaustive search: reducing the
search space using syntactical constraints and efficient semantic structure
deduplication. Genetic programming theory and practice <span id="bib.bib6.1.1" class="ltx_text ltx_font_bold">17</span>, 79–99
(2020)

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[7]</span>
<span class="ltx_bibblock">
Kommenda, M., Burlacu, B., Kronberger, G., Affenzeller, M.: Parameter
identification for symbolic regression using nonlinear least squares. Genetic
Programming and Evolvable Machines <span id="bib.bib7.1.1" class="ltx_text ltx_font_bold">21</span>(3), 471–501 (2020)

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[8]</span>
<span class="ltx_bibblock">
Krogh, F.T.: Efficient implementation of a variable projection algorithm for
nonlinear least squares problems. Communications of the ACM <span id="bib.bib8.1.1" class="ltx_text ltx_font_bold">17</span>(3),
167–169 (1974)

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[9]</span>
<span class="ltx_bibblock">
McConaghy, T.: Ffx: Fast, scalable, deterministic symbolic regression
technology. In: Genetic Programming Theory and Practice IX, pp. 235–260.
Springer (2011)

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_tag_bibitem">[10]</span>
<span class="ltx_bibblock">
Orzechowski, P., La Cava, W., Moore, J.H.: Where are we now? a large benchmark
study of recent symbolic regression methods. In: Proceedings of the Genetic
and Evolutionary Computation Conference. pp. 1183–1190 (2018)

</span>
</li>
</ul>
</section>
<div class="ltx_pagination ltx_role_newpage"></div>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2209.09674" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2209.09675" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2209.09675">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2209.09675" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2209.09677" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Thu Mar 14 00:39:47 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2209.09675
Fast Function Extraction (FFX)  is a deterministic algorithm for solving symbolic regression problems. We improve the accuracy of FFX by adding parameters to the arguments of nonlinear functions. Instead of only optimizing linear parameters, we optimize these additional nonlinear parameters with separable nonlinear least squared optimization using a variable projection algorithm. Both FFX and our new algorithm is applied on the PennML benchmark suite. We show that the proposed extensions of FFX leads to higher accuracy while providing models of similar length and with only a small increase in runtime on the given data. Our results are compared to a large set of regression methods that were already published for the given benchmark suite.
abstract
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
Abstract
2209.09675
1Symbolic Regression and FFXSymbolic regression is a machine learning task in which we try to identify mathematical formulas that cover linear and nonlinear relations within given data. The most common algorithm for solving symbolic regression is genetic programming (GP) , which optimizes a population of mathematical models using crossover and mutation. GP is in theory capable of finding models of any syntactical structure and complexity. However, disadvantages of GP are its stochasticity, its long algorithm runtime for nontrivial problems and its complex hyperparameter settings. These characteristics led to the development of non-evolutionary algorithms which produce more restricted models but have advantages in determinism, runtime or complexity of hyperparameters.One of the first algorithms that was developed to tackle the shortages of GP is Fast Function Extraction (FFX) . FFX generates a large set ofbase functionsfirst, then it learns a regularized linear model using these base functions as terms. The set of base functions is a combination of the original features with several nonlinear functions such asorand a predefined set of real-valued exponent values. Examples of base functions for problems with featuresare,,,,,or.The structure of FFX models withfeaturesis outlined in Equation1. Parametersare linear parameters of a model withbase functions. They are learned with ElasticNet regression. ElasticNet regression identifies only the most relevant base functions due to its regularization by setting linear parametersof irrelevant base functions to zero (sparsification) . Therefore, learned models contain only a subset of the original base functions.(1) 1.1Motivation and ObjectivesA disadvantage of FFX is that only linear parameters are optimized. Parameters within nonlinear functions are not present. For example functions with featureand nonlinear parameterssuch asorhave to be approximated by FFX with a linear model of several base functions. In this work, we extend the capabilities of FFX by adding such a real-valued parameterto each generated base functions. Adding several nonlinear parameters to the possible model structure should allow to fit additive models with fewer base functions, as we have more degrees of freedom per base function.The introduced nonlinear parameters are optimized in combination with the linear parameters by separable nonlinear least squares optmization (NLS)  using a variable projection algorithm. We call this new algorithmFFX with Nonlinear Least Squares Optimization(FFX NLS) . We test whether FFX NLS leads to higher accuracy than the original implementation on thePennMLbenchmark suiteand compare the complexity of the generated models. Given, that we have more degrees of freedom to fit data with a single base function, we expect to find that it produces models with a lower number of base functions and therefore simpler models than the original FFX algorithm.
introduction
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
1Symbolic Regression and FFX
2209.09675
2Algorithm DescriptionSimilar to FFX, FFX NLS runs in several steps. First, base functions are generated. Then the most relevant base functions and the model’s parameters are determined. In difference to FFX, the selection of relevant base function and the optimization of parameters are separate steps. FFX NLS performs the following four steps that are described in detail in the next sections:1.Generate a list of all univariate base functions(cf. Section2.1) .2.Optimize all parametersandof a nonlinear modelthat consists of the base functions as terms.is a vector of all nonlinear parameters.are linear parameters,are features (cf. Section2.2) .3.Select most important base functions ofwith a regularized linear model and the nonlinear parametersfrom the previous step (cf. Section2.3) .4.The final model is created by optimizing all parameters again with nonlinear least squares optimization but only using the most important base function (cf. Section2.4) .2.1Base FunctionsIn step 1, we generate all univariate base functionswith placeholders for nonlinear parameters. For each featurewe create base functions of structurewith,and two scaling valuesandwith. The scaling valuesandare placeholder for nonlinear parameters that will be optimized later on. We also include bivariate base functions, which are described in Section2.4.We utilize the linear structure in the final model as well as mathematical identities of the used nonlinear functions to reduce the number of nonlinear parameters. Due to mathematical identities such aswithbeing a parameter that is trained later on, we can skip certain scaling values. In the case of logarithm, we can just useinstead ofas base function as we can rewrite it to. Then we can skipas it is constant in the final model and therefore summed up by the final model’s intercept. The same applies to the exp-function, in which we can skip the multiplicative scaling value in the argument as we can rewriteand skipin the final model.2.2Parameter Optimization with Variable ProjectionThe generated base functionsare combined to one large linear modelwithas vector of linear parameters and intercept andas vector of nonlinear parameters. We use NLS to find the values inandthat minimize the mean squared error (MSE)  for given training data.Since nonlinear least squares optimization is computationally expensive, we use a variable projection (VP)  algorithm initially developed by Golub and Pereyafor optimizing all parametersand. The advantage of VP over plain NLS optimization is that VP utilizes the generated model’s structure – a nonlinear model with several linear parameters. Golub and Pereya call the given model structure aseparable least squares problem. This setting is common in engineering domains. VP optimizes nonlinear parameters initeratively, while optimal linear parameters are solved exactly via ordinary least squares. Given the high number of linear parameters in the model (due to the large number of base functions) , the use of VP is an important performance aspect of FFX NLS. We use the effieicennt VP algorithm by Krogh.2.3Base Function SelectionAs we generate a very large number of base functions, we need to select the most relevant ones in order to create a both interpretable and well-generalizing model. The number of selected base functions is thereby a hyperparameter of FFX NLS. In plain FFX, the selection of relevant base functions and the optimization of parameters is done in one step with a ElasticNet regression, as only linear parameters need to be optimized.Since we also need to optimize nonlinear parameters, no simple way to combine NLS optimization and regularization of linear parameters is available. The VP algorithm by Chen et al.uses Tikhonov regularization. However, in initial experiments this algorithm was not beneficial for FFX NLS to identify relevant terms. Tikhonov regularization shrinks linear parameters, however, it did not provide the necessary sparsification of linear parameters and the algorithm inis computationally more expensive. Alternatively, we use the already computed vector of nonlinear parametersof the previous step as fixed constants and optimize the linear parameterswith a lasso regression. Terms with a linear parameterare selected as most important base functions. We get the desired number of base functions by iteratively increasing the lasso regression’svalue in order to shrink more parameters to zero. Although this method ignores dependencies between linear and nonlinear parameters, it is still effective for selecting base functions both regarding runtime and further modelling accuracy.2.4Training of Final Model and Bivariate Base FunctionsSimilar to the parameter optimization in Section2.2, we combine base functions to one single model and optimize all parameters again. In difference to the previous NLS optimization step, we take in this step only the most relevant base functions from Section2.3. As parameters are not independent from each other, we repeat this step to get optimal parameters for a subset of base functions.To cover interactions between features, we also include bivariate base functions. For this we take pairwise combinations of the previously generated univariate base function and multiply them to get one new base function for each pair. To prevent a combinatorial explosion of base functions and nonlinear parameters in highly multidimensional problems, we only consider the ten most important univariate base functions from Section2.3for to create bivariate base functions. We take the resultingbivariate functions and append them to our existing list of univariate base functions. Then we repeat the steps described in Section2.2and2.3to determine the most important base functions across both univariate and bivariate base function for the final model. We consider only ten univariate base functions as a larger number did not provide any benefit in achieving our objectives.
body
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
2Algorithm Description
2209.09675
3Experimental SetupWe usePennMLbenchmark suitefor all experiments. This benchmark consist of over 90 regression problem and provides a performance overview of several common regression algorithms. We apply both FFX and FFX NLS on these problems to compare the algorithms’ accuracies with each other and with other common regression algorithms. Thereby we take the experimental results fromfor a comprehensive comparison as this work shows the results of the currently most accurate regression algorithms like GP CoOp as implemented infor the PennML data.We apply the same modelling workflow as inin our experiments. For every regression problem in the benchmark suite, we repeat the modelling ten times. We shuffle the dataset in every repetition and take the first 75% of observations as training set and the remaining ones as test set. We perform a grid search with a 5-fold cross validation for each shuffled dataset for hyperparameter tuning and train one final model with these best hyperparameters. Eventually, we get ten different models for each original regression problem. Hyperparameters for all other algorithms are described in. We use the following hyperparameter sets for both FFX and FFX NLS:•Max. number of base function in final model•Use bivariate base functions•Use nonlinear functions•FFX only: L1 ratio
body
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
3Experimental Setup
2209.09675
4ResultsFFX NLSFFXGPGP CoOpOSGPOSGP CoOpgsgpafpmrgpeplexeplex-1mxgboostgradboostmlprfkernel-ridgeadaboostlasso-larslinear-svrlinear-regressionsgd-regressionMedian ranking (training) Figure 1:Distribution of median rankings of the mean squared error on the training set.FFX NLSFFXGPGP CoOpOSGPOSGP CoOpgsgpafpmrgpeplexeplex-1mxgboostgradboostmlprfkernel-ridgeadaboostlasso-larslinear-svrlinear-regressionsgd-regressionMedian ranking (test) Figure 2:Distribution of median rankings of the mean squared error on the test set.As in the original experiments for this benchmark suite, we first calculate the median (mean squared)  error of the ten models of each regression problem. Then we rank all regression methods by their median error for each problem. Figure1and2show the distribution of ranks for each algorithm for training and test across all regression problems in the benchmark suite. E.g.  in Figure1, gradient boosting was for most problems the most accurate algorithm in training.Figure1and2show the distribution of rankings of each algorithm’s median rank on the training set and test set. The optimization of nonlinear parameters in FFX NLS provided to more accurate models than FFX both in training and test. In comparison to other algorithms, FFX as well as FFX NLS perform better than linear models, which are on the right of Figure1(like linear regression or lasso regression) . This is expected given both FFX methods can cover nonlinear dependencies and interactions in the data in contrast to purely linear methods. However, both algorithms perform worse than boosting methods or GP with NLS (GP CoOp)  by Kommenda et al.. Also this is plausible, as both methods have a more powerful search algorithm and a larger hypothesis space than FFX methods with their restricted model structures.To analyze the size of models, we count the number of syntactical symbols in a model. E.g. the modelhas a complexity of six. We use this measure because it takes the added scaling terms within function arguments in FFX NLS into account. Figure3(a) shows that both the size of models and the number of base function within models produced by FFX NLS and FFX are similar. FFX NLS models are just slightly shorter.FFX NLSFFXModel length(a) Length per model.FFX NLSFFXNubmer of terms(b) Number of base functions per model.Figure 3:Distribution complexity of all models from FFX and FFX NLS.FFX NLSFFXGPGP CoOpOSGPOSGP CoOpgsgpafpmrgpeplexeplex-1mxgboostgradboostmlprfkernel-ridgeadaboostlasso-larslinear-svrlinear-regressionsgd-regressionRuntime (in secs) Figure 4:Median runtime of all algorithms.Figure4shows the median runtime across all problems per algorithm. While one FFX run takes less than a second for most problems, the runtime of FFX NLS ranges between one and a few seconds. Although the runtime increase is relatively large, it is still feasible to perform large grid searches in reasonable amount of time with FFX NLS. Both algorithms beat some GP-based algorithms and are on a similar level as boosting algorithms.
body
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
4Results
2209.09675
5ConclusionWe proposed an extension of the model structure of the FFX algorithm. We added nonlinear scaling parameter which were optimized using the variable projection algorithm by Kroghand separate base function selection, which we called FFX NLS. We achieved a large improvement in test accuracy in comparison to plain FFX on the PennML benchmark suite while providing models of similar complexity. Although the runtime of FFX NLS is higher than the one of FFX, it still finishes training within seconds.However, FFX and FFX NLS still perform worse than many symbolic regression algorithms. Big advantages of FFX NLS towards GP-based algorithms are its short runtime, the low number of hyperparameters and the simple, comprehensible model structure. Compared to boosting algorithms, FFX NLS performs worse in accuracy but similar in runtime. However, boosting algorithms are considered black box methods and allow no readability of its models.To sum it up, FFX is a promising tool for quick data exploration. It identifies interactions as well as nonlinear relations with simple hyperparameter configuration and quick execution. However, further improvements regarding accuracy are needed. Potential improvements are the combination of regularization and parameter optimization, as this are separate steps right now that deliberately ignore dependencies between linear and nonlinear parameters.5.0.1AcknowledgementsThe authors gratefully acknowledge support by the Christian Doppler Research Association and the Federal Ministry for Digital and Economic Affairs within theJosef Ressel Center for Symbolic Regression.
conclusion
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
5Conclusion