<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2404.08819] The Illusion of State in State-Space Models</title><meta property="og:description" content="State-space models (SSMs) have emerged as a potential alternative architecture for building large language models (LLMs) compared to the previously ubiquitous transformer architecture. One theoretical weakness of trans…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="The Illusion of State in State-Space Models">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="The Illusion of State in State-Space Models">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2404.08819">

<!--Generated on Sun May  5 21:23:09 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="keywords" lang="en" content="Machine Learning,  ICML">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document">
<h1 class="ltx_title ltx_title_document">The Illusion of State in State-Space Models</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">William Merrill
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Jackson Petty
</span></span>
<span class="ltx_author_before">  </span><span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Ashish Sabharwal
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id1.1" class="ltx_p">State-space models (SSMs) have emerged as a potential alternative architecture for building large language models (LLMs) compared to the previously ubiquitous transformer architecture. One theoretical weakness of transformers is that they cannot express certain kinds of sequential computation and state tracking <cite class="ltx_cite ltx_citemacro_citep">(Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>, which SSMs are explicitly designed to address via their close architectural similarity to recurrent neural networks (RNNs). <em id="id1.1.1" class="ltx_emph ltx_font_italic">But do SSMs truly have an advantage (over transformers) in expressive power for state tracking?</em> Surprisingly, the answer is no. Our analysis reveals that the expressive power of SSMs is limited very similarly to transformers: SSMs cannot express computation outside the complexity class <math id="id1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="id1.1.m1.1a"><msup id="id1.1.m1.1.1" xref="id1.1.m1.1.1.cmml"><mi id="id1.1.m1.1.1.2" xref="id1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="id1.1.m1.1.1.3" xref="id1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="id1.1.m1.1b"><apply id="id1.1.m1.1.1.cmml" xref="id1.1.m1.1.1"><csymbol cd="ambiguous" id="id1.1.m1.1.1.1.cmml" xref="id1.1.m1.1.1">superscript</csymbol><ci id="id1.1.m1.1.1.2.cmml" xref="id1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="id1.1.m1.1.1.3.cmml" xref="id1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="id1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>. In particular, this means they cannot solve simple state-tracking problems like permutation composition. It follows that SSMs are provably unable to accurately track chess moves with certain notation, evaluate code, or track entities in a long narrative. To supplement our formal analysis, we report experiments showing that Mamba-style SSMs indeed struggle with state tracking. Thus, despite its recurrent formulation, the “state” in an SSM is an illusion: SSMs have similar expressiveness limitations to non-recurrent models like transformers, which may fundamentally limit their ability to solve real-world state-tracking problems.</p>
</div>
<div class="ltx_keywords">Machine Learning, ICML
</div>
<div id="p1" class="ltx_para">
<span id="p1.1" class="ltx_ERROR undefined">\setchessboard</span>
<p id="p1.2" class="ltx_p">showmover=false








  




</p>
</div>
<div id="p3" class="ltx_para">
<br class="ltx_break">
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<figure id="S1.F1" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_1">
<div id="S1.F1.7" class="ltx_inline-block ltx_figure_panel ltx_align_center ltx_transformed_outer" style="width:173.4pt;height:4.7pt;vertical-align:-1.2pt;"><span class="ltx_transformed_inner" style="transform:translate(-97.3pt,2.0pt) scale(0.471188747693782,0.471188747693782) ;"><span id="S1.F1.7.8" class="ltx_ERROR undefined">\chessboard</span>
<p id="S1.F1.7.7" class="ltx_p">[
setpieces=qa8, qb8, rc8, qd8, qe8, Pa2, Pb2, Ka1, kh1,
addpgf=
 <svg id="S1.F1.1.1.pic1" class="ltx_picture" height="6.42" overflow="visible" version="1.1" width="3.1"><g transform="translate(0,6.42) matrix(1 0 0 -1 0 0) translate(2.41,0) translate(0,3.21)" fill="#804D00" stroke="#804D00" stroke-width="0.4pt" color="#804D00"><g stroke-width="1.0pt"><path d="M 0 0 C 0 0 0 0 1.16 0" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
 <svg id="S1.F1.2.2.pic2" class="ltx_picture" height="1.38" overflow="visible" version="1.1" width="1.38"><g transform="translate(0,1.38) matrix(1 0 0 -1 0 0) translate(0.69,0) translate(0,0.69)" fill="#B38033" stroke="#B38033" stroke-width="0.4pt" color="#B38033"><g stroke-width="1.0pt"><path d="M 0 0 L 1.16 0" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
 <svg id="S1.F1.3.3.pic3" class="ltx_picture" height="1.38" overflow="visible" version="1.1" width="1.38"><g transform="translate(0,1.38) matrix(1 0 0 -1 0 0) translate(0.69,0) translate(0,0.69)" fill="#E6B366" stroke="#E6B366" stroke-width="0.4pt" color="#E6B366"><g stroke-width="1.0pt"><path d="M 0 0 C 0 0 0 0 1.16 0" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
 <svg id="S1.F1.4.4.pic4" class="ltx_picture" height="1.38" overflow="visible" version="1.1" width="1.38"><g transform="translate(0,1.38) matrix(1 0 0 -1 0 0) translate(0.69,0) translate(0,0.69)" fill="#004D80" stroke="#004D80" stroke-width="0.4pt" color="#004D80"><g stroke-width="1.0pt"><path d="M 0 0 L 1.16 0" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
 <svg id="S1.F1.5.5.pic5" class="ltx_picture" height="1.38" overflow="visible" version="1.1" width="1.38"><g transform="translate(0,1.38) matrix(1 0 0 -1 0 0) translate(0.69,0) translate(0,0.69)" fill="#3380B3" stroke="#3380B3" stroke-width="0.4pt" color="#3380B3"><g stroke-width="1.0pt"><path d="M 0 0 L 1.16 0" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
 <svg id="S1.F1.6.6.pic6" class="ltx_picture" height="1.38" overflow="visible" version="1.1" width="1.38"><g transform="translate(0,1.38) matrix(1 0 0 -1 0 0) translate(0.69,0) translate(0,0.69)" fill="#66B3E6" stroke="#66B3E6" stroke-width="0.4pt" color="#66B3E6"><g stroke-width="1.0pt"><path d="M 0 0 L 1.16 0" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
 <svg id="S1.F1.7.7.pic7" class="ltx_picture" height="1.38" overflow="visible" version="1.1" width="1.38"><g transform="translate(0,1.38) matrix(1 0 0 -1 0 0) translate(0.69,0) translate(0,0.69)" fill="#4D4D4D" stroke="#4D4D4D" stroke-width="0.4pt" color="#4D4D4D"><g stroke-width="1.0pt"><path d="M 1.16 0 L 1.16 0" style="fill:none"></path></g><g transform="matrix(-1.0 0.0 0.0 -1.0 1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -1.16 0)" stroke-dasharray="none" stroke-dashoffset="0.0pt" stroke-linecap="round" stroke-linejoin="round" stroke-width="0.8pt"><path d="M -2.41 3.21 C -2.21 2.01 0 0.2 0.6 0 C 0 -0.2 -2.21 -2.01 -2.41 -3.21" style="fill:none"></path></g></g></svg>
,
]</p>
</span></div>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1">
<p id="S1.F1.11" class="ltx_p ltx_figure_panel ltx_align_center">
<span id="S1.F1.11.1" class="ltx_inline-block ltx_parbox ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:411.9pt;"><span id="S1.F1.11.1.1" class="ltx_ERROR undefined">\pyth</span>
<span id="S1.F1.11.1.2" class="ltx_p"><span id="S1.F1.11.1.2.1" class="ltx_text" style="font-size:90%;">x = [0, 0, 1, 0, 0]
<br class="ltx_break"><span id="S1.F1.11.1.2.1.1" class="ltx_ERROR undefined">\pyth</span>x[1], x[3] = x[3], x[1] # Swap 1, 3</span></span>
</span></p>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1">
<p id="S1.F1.12" class="ltx_p ltx_figure_panel ltx_align_center">
<span id="S1.F1.12.1" class="ltx_inline-block ltx_parbox ltx_align_middle ltx_framed ltx_framed_rectangle" style="width:411.9pt;">
<span id="S1.F1.12.1.1" class="ltx_p"><em id="S1.F1.12.1.1.1" class="ltx_emph ltx_font_italic">Alice, Bob, Carl, Dan, and Emma each have a coin. All are dimes except Carl’s.
Alice and Carl trade coins.</em></span>
</span></p>
</div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S1.F1.13.2.1" class="ltx_text" style="font-size:90%;">Figure 1</span>: </span><span id="S1.F1.9.1" class="ltx_text" style="font-size:90%;">
We prove that SSMs, like transformers, cannot solve inherently sequential problems like permutation composition (<math id="S1.F1.9.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S1.F1.9.1.m1.1b"><msub id="S1.F1.9.1.m1.1.1" xref="S1.F1.9.1.m1.1.1.cmml"><mi id="S1.F1.9.1.m1.1.1.2" xref="S1.F1.9.1.m1.1.1.2.cmml">S</mi><mn id="S1.F1.9.1.m1.1.1.3" xref="S1.F1.9.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S1.F1.9.1.m1.1c"><apply id="S1.F1.9.1.m1.1.1.cmml" xref="S1.F1.9.1.m1.1.1"><csymbol cd="ambiguous" id="S1.F1.9.1.m1.1.1.1.cmml" xref="S1.F1.9.1.m1.1.1">subscript</csymbol><ci id="S1.F1.9.1.m1.1.1.2.cmml" xref="S1.F1.9.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S1.F1.9.1.m1.1.1.3.cmml" xref="S1.F1.9.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.9.1.m1.1d">S_{5}</annotation></semantics></math>), which lies at the heart of state-tracking problems like tracking chess moves in source-target notation (see <a href="#S3.SS2" title="3.2 Encoding 𝑆₅ in Chess State Tracking ‣ 3 State Tracking ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">3.2</span></a>), evaluating Python code, or entity tracking. Thus, SSMs cannot, in general, solve these problems either.
</span></figcaption>
</figure>
<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Recent theoretical work <cite class="ltx_cite ltx_citemacro_citep">(Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite> has shown that models built upon the transformer architecture are incapable of expressing inherently sequential computation. These results reveal a surprising limitation of transformers: they cannot express simple kinds of <em id="S1.p1.1.1" class="ltx_emph ltx_font_italic">state tracking</em> problems, such as composing sequences of permutations, which even simple recurrent neural networks (RNNs) can naturally express.
In a different line of work, state space model (SSM) architectures <cite class="ltx_cite ltx_citemacro_citep">(Gu et al., <a href="#bib.bib9" title="" class="ltx_ref">2021</a>, <a href="#bib.bib10" title="" class="ltx_ref">2022a</a>; Fu et al., <a href="#bib.bib7" title="" class="ltx_ref">2023</a>; Gu &amp; Dao, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>; Wang et al., <a href="#bib.bib31" title="" class="ltx_ref">2024</a>)</cite> have been introduced as an alternative to transformers, with the goal of achieving similar expressive power to RNNs for handling problems that are naturally stateful and sequential <cite class="ltx_cite ltx_citemacro_citep">(Gu et al., <a href="#bib.bib9" title="" class="ltx_ref">2021</a>, <a href="#bib.bib11" title="" class="ltx_ref">2022b</a>)</cite>.
<em id="S1.p1.1.2" class="ltx_emph ltx_font_italic">But does the seemingly stateful design of SSMs truly enable them to solve sequential and state-tracking problems that transformers cannot?</em>
If so, this would be a promising property of SSMs because state tracking is at the heart of large language model (LLM) capabilities such as tracking entities in a narrative <cite class="ltx_cite ltx_citemacro_citep">(Heim, <a href="#bib.bib13" title="" class="ltx_ref">1983</a>; Kim &amp; Schuster, <a href="#bib.bib19" title="" class="ltx_ref">2023</a>)</cite>, playing chess under certain notation<span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>The hardness of chess state tracking holds with (source, target) notation, but standard notation may make state tracking easier.</span></span></span>, or evaluating code. This would motivate further research into SSM architectures and their deployment as the next generation of LLMs.</p>
</div>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">In this work, we show that the apparent stateful design of SSMs is an <em id="S1.p2.1.1" class="ltx_emph ltx_font_italic">illusion</em> as far as their expressive power is concerned. In contrast to the suggestion by <cite class="ltx_cite ltx_citemacro_citet">Gu et al. (<a href="#bib.bib9" title="" class="ltx_ref">2021</a>, <a href="#bib.bib11" title="" class="ltx_ref">2022b</a>)</cite> (and, perhaps, a broader belief in the community) that SSMs have expressive power for state tracking similar to RNNs, we prove theoretically that linear and Mamba-style SSMs, like transformers, cannot express inherently sequential problems, including state-tracking problems like composing permutations that RNNs can easily express. Further, our experiments confirm our formal predictions: both transformers and these SSMs cannot learn to compose permutations with a fixed number of layers, whereas RNNs can compose permutations with just a single layer.
Our results imply that arguments that SSMs have an advantage over transformers due to being “more recurrent” or capable of tracking state are misguided. In fact, the SSM architectures we consider are just as theoretically unequipped for state tracking and recurrent computation as transformers are.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.3" class="ltx_p">We first establish the theoretical weakness of linear SSMs and near generalizations by proving they are in the complexity class <math id="S1.p3.1.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S1.p3.1.m1.1a"><mi id="S1.p3.1.m1.1.1" xref="S1.p3.1.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S1.p3.1.m1.1b"><ci id="S1.p3.1.m1.1.1.cmml" xref="S1.p3.1.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.1.m1.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S1.p3.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S1.p3.2.m2.1a"><msup id="S1.p3.2.m2.1.1" xref="S1.p3.2.m2.1.1.cmml"><mi id="S1.p3.2.m2.1.1.2" xref="S1.p3.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S1.p3.2.m2.1.1.3" xref="S1.p3.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S1.p3.2.m2.1b"><apply id="S1.p3.2.m2.1.1.cmml" xref="S1.p3.2.m2.1.1"><csymbol cd="ambiguous" id="S1.p3.2.m2.1.1.1.cmml" xref="S1.p3.2.m2.1.1">superscript</csymbol><ci id="S1.p3.2.m2.1.1.2.cmml" xref="S1.p3.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S1.p3.2.m2.1.1.3.cmml" xref="S1.p3.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math>, which has been previously shown for transformers <cite class="ltx_cite ltx_citemacro_citep">(Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>. This implies these SSMs cannot solve inherently sequential problems (formally, problems that are <math id="S1.p3.3.m3.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S1.p3.3.m3.1a"><msup id="S1.p3.3.m3.1.1" xref="S1.p3.3.m3.1.1.cmml"><mi id="S1.p3.3.m3.1.1.2" xref="S1.p3.3.m3.1.1.2.cmml">𝖭𝖢</mi><mn id="S1.p3.3.m3.1.1.3" xref="S1.p3.3.m3.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S1.p3.3.m3.1b"><apply id="S1.p3.3.m3.1.1.cmml" xref="S1.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S1.p3.3.m3.1.1.1.cmml" xref="S1.p3.3.m3.1.1">superscript</csymbol><ci id="S1.p3.3.m3.1.1.2.cmml" xref="S1.p3.3.m3.1.1.2">𝖭𝖢</ci><cn type="integer" id="S1.p3.3.m3.1.1.3.cmml" xref="S1.p3.3.m3.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.3.m3.1c">\mathsf{NC}^{1}</annotation></semantics></math>-hard), including state-tracking problems like permutation composition <cite class="ltx_cite ltx_citemacro_citep">(Liu et al., <a href="#bib.bib21" title="" class="ltx_ref">2023</a>)</cite>. Permutation composition is a fundamental problem at the heart of many real-world state-tracking problems such as playing chess, evaluating code, or tracking entities in a narrative (<a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a>), implying solutions to these problems, too, cannot be expressed by SSMs, at least in the worst case.</p>
</div>
<figure id="S1.F2" class="ltx_figure"><svg id="S1.F2.pic1" class="ltx_picture ltx_centering" height="111.25" overflow="visible" version="1.1" width="249.62"><g transform="translate(0,111.25) matrix(1 0 0 -1 0 0) translate(124.81,0)" fill="#000000" stroke="#000000" stroke-width="0.4pt"><g fill="#FFCCCB"><path d="M 124.53 55.62 C 124.53 86.19 68.78 110.97 0 110.97 C -68.78 110.97 -124.53 86.19 -124.53 55.62 C -124.53 25.06 -68.78 0.28 0 0.28 C 68.78 0.28 124.53 25.06 124.53 55.62 Z M 0 55.62"></path></g><g fill="#CBC3E3"><path d="M 96.86 41.79 C 96.86 64.71 53.49 83.3 0 83.3 C -53.49 83.3 -96.86 64.71 -96.86 41.79 C -96.86 18.86 -53.49 0.28 0 0.28 C 53.49 0.28 96.86 18.86 96.86 41.79 Z M 0 41.79"></path></g><g transform="matrix(1.0 0.0 0.0 1.0 -60.24 88.83)" fill="#000000" stroke="#000000"><foreignObject width="27.8" height="7.69" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S1.F2.pic1.4.4.4.1.1" class="ltx_text ltx_font_italic" style="font-size:80%;">Chess</span></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 -94.15 75.17)" fill="#000000" stroke="#000000"><foreignObject width="10.53" height="9.56" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1a"><msub id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml"><mi mathsize="80%" id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2.cmml">S</mi><mn mathsize="80%" id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1b"><apply id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1">subscript</csymbol><ci id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.m1.1c">S_{5}</annotation></semantics></math></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 32.45 88.83)" fill="#000000" stroke="#000000"><foreignObject width="24.6" height="7.69" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S1.F2.pic1.5.5.5.1.1" class="ltx_text ltx_font_italic" style="font-size:80%;">Code</span></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 67.88 75.17)" fill="#000000" stroke="#000000"><foreignObject width="37.73" height="7.56" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S1.F2.pic1.6.6.6.1.1" class="ltx_text ltx_font_italic" style="font-size:80%;">Entities</span></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 -60.65 38.01)" fill="#000000" stroke="#000000"><foreignObject width="26.81" height="7.56" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S1.F2.pic1.7.7.7.1.1" class="ltx_text" style="font-size:80%;">SSMs</span></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 7.71 37.94)" fill="#000000" stroke="#000000"><foreignObject width="63.56" height="7.69" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><span id="S1.F2.pic1.8.8.8.1.1" class="ltx_text" style="font-size:80%;">Transformers</span></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 -11.93 66.73)" fill="#000000" stroke="#000000"><foreignObject width="23.86" height="11.95" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1a"><msup id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml"><mi id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.2" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.3" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1b"><apply id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1">superscript</csymbol><ci id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.pic1.2.2.2.2.2.2.2.2.2.2.2.2.1.1.1.1.1.1.1.1.1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math></foreignObject></g><g transform="matrix(1.0 0.0 0.0 1.0 -12.12 94.41)" fill="#000000" stroke="#000000"><foreignObject width="24.25" height="11.95" transform="matrix(1 0 0 -1 0 16.6)" overflow="visible"><math id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1a"><msup id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.cmml"><mi id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.2" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.2.cmml">𝖭𝖢</mi><mn id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.3" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1b"><apply id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.cmml" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.1.cmml" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1">superscript</csymbol><ci id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.2.cmml" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.2">𝖭𝖢</ci><cn type="integer" id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.3.cmml" xref="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.pic1.3.3.3.3.3.3.3.3.3.3.3.3.2.1.1.1.1.1.1.1.1.m1.1c">\mathsf{NC}^{1}</annotation></semantics></math></foreignObject></g></g></svg>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S1.F2.13.6.1" class="ltx_text" style="font-size:90%;">Figure 2</span>: </span><span id="S1.F2.10.5" class="ltx_text" style="font-size:90%;">Complexity hierarchy within <math id="S1.F2.6.1.m1.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S1.F2.6.1.m1.1b"><msup id="S1.F2.6.1.m1.1.1" xref="S1.F2.6.1.m1.1.1.cmml"><mi id="S1.F2.6.1.m1.1.1.2" xref="S1.F2.6.1.m1.1.1.2.cmml">𝖭𝖢</mi><mn id="S1.F2.6.1.m1.1.1.3" xref="S1.F2.6.1.m1.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S1.F2.6.1.m1.1c"><apply id="S1.F2.6.1.m1.1.1.cmml" xref="S1.F2.6.1.m1.1.1"><csymbol cd="ambiguous" id="S1.F2.6.1.m1.1.1.1.cmml" xref="S1.F2.6.1.m1.1.1">superscript</csymbol><ci id="S1.F2.6.1.m1.1.1.2.cmml" xref="S1.F2.6.1.m1.1.1.2">𝖭𝖢</ci><cn type="integer" id="S1.F2.6.1.m1.1.1.3.cmml" xref="S1.F2.6.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.6.1.m1.1d">\mathsf{NC}^{1}</annotation></semantics></math>. Transformers can only recognize languages within <math id="S1.F2.7.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S1.F2.7.2.m2.1b"><msup id="S1.F2.7.2.m2.1.1" xref="S1.F2.7.2.m2.1.1.cmml"><mi id="S1.F2.7.2.m2.1.1.2" xref="S1.F2.7.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S1.F2.7.2.m2.1.1.3" xref="S1.F2.7.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S1.F2.7.2.m2.1c"><apply id="S1.F2.7.2.m2.1.1.cmml" xref="S1.F2.7.2.m2.1.1"><csymbol cd="ambiguous" id="S1.F2.7.2.m2.1.1.1.cmml" xref="S1.F2.7.2.m2.1.1">superscript</csymbol><ci id="S1.F2.7.2.m2.1.1.2.cmml" xref="S1.F2.7.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S1.F2.7.2.m2.1.1.3.cmml" xref="S1.F2.7.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.7.2.m2.1d">\mathsf{TC}^{0}</annotation></semantics></math> <cite class="ltx_cite ltx_citemacro_citep">(Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>, and we show the same for SSMs (<a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorems</span> <span class="ltx_text ltx_ref_tag">4.1</span></a> and <a href="#S4.Thmtheorem2" title="Theorem 4.2. ‣ 4.2 Simulating S6 with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.2</span></a>). Thus, both architectures cannot express the “hard state tracking” captured by <math id="S1.F2.8.3.m3.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S1.F2.8.3.m3.1b"><msup id="S1.F2.8.3.m3.1.1" xref="S1.F2.8.3.m3.1.1.cmml"><mi id="S1.F2.8.3.m3.1.1.2" xref="S1.F2.8.3.m3.1.1.2.cmml">𝖭𝖢</mi><mn id="S1.F2.8.3.m3.1.1.3" xref="S1.F2.8.3.m3.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S1.F2.8.3.m3.1c"><apply id="S1.F2.8.3.m3.1.1.cmml" xref="S1.F2.8.3.m3.1.1"><csymbol cd="ambiguous" id="S1.F2.8.3.m3.1.1.1.cmml" xref="S1.F2.8.3.m3.1.1">superscript</csymbol><ci id="S1.F2.8.3.m3.1.1.2.cmml" xref="S1.F2.8.3.m3.1.1.2">𝖭𝖢</ci><cn type="integer" id="S1.F2.8.3.m3.1.1.3.cmml" xref="S1.F2.8.3.m3.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.8.3.m3.1d">\mathsf{NC}^{1}</annotation></semantics></math>-complete problems like <math id="S1.F2.9.4.m4.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S1.F2.9.4.m4.1b"><msub id="S1.F2.9.4.m4.1.1" xref="S1.F2.9.4.m4.1.1.cmml"><mi id="S1.F2.9.4.m4.1.1.2" xref="S1.F2.9.4.m4.1.1.2.cmml">S</mi><mn id="S1.F2.9.4.m4.1.1.3" xref="S1.F2.9.4.m4.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S1.F2.9.4.m4.1c"><apply id="S1.F2.9.4.m4.1.1.cmml" xref="S1.F2.9.4.m4.1.1"><csymbol cd="ambiguous" id="S1.F2.9.4.m4.1.1.1.cmml" xref="S1.F2.9.4.m4.1.1">subscript</csymbol><ci id="S1.F2.9.4.m4.1.1.2.cmml" xref="S1.F2.9.4.m4.1.1.2">𝑆</ci><cn type="integer" id="S1.F2.9.4.m4.1.1.3.cmml" xref="S1.F2.9.4.m4.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.9.4.m4.1d">S_{5}</annotation></semantics></math>, which <em id="S1.F2.10.5.1" class="ltx_emph ltx_font_italic">can</em> be straightforwardly expressed by RNNs. The figure assumes the widely held conjecture <math id="S1.F2.10.5.m5.1" class="ltx_Math" alttext="\mathsf{TC}^{0}\neq\mathsf{NC}^{1}" display="inline"><semantics id="S1.F2.10.5.m5.1b"><mrow id="S1.F2.10.5.m5.1.1" xref="S1.F2.10.5.m5.1.1.cmml"><msup id="S1.F2.10.5.m5.1.1.2" xref="S1.F2.10.5.m5.1.1.2.cmml"><mi id="S1.F2.10.5.m5.1.1.2.2" xref="S1.F2.10.5.m5.1.1.2.2.cmml">𝖳𝖢</mi><mn id="S1.F2.10.5.m5.1.1.2.3" xref="S1.F2.10.5.m5.1.1.2.3.cmml">0</mn></msup><mo id="S1.F2.10.5.m5.1.1.1" xref="S1.F2.10.5.m5.1.1.1.cmml">≠</mo><msup id="S1.F2.10.5.m5.1.1.3" xref="S1.F2.10.5.m5.1.1.3.cmml"><mi id="S1.F2.10.5.m5.1.1.3.2" xref="S1.F2.10.5.m5.1.1.3.2.cmml">𝖭𝖢</mi><mn id="S1.F2.10.5.m5.1.1.3.3" xref="S1.F2.10.5.m5.1.1.3.3.cmml">1</mn></msup></mrow><annotation-xml encoding="MathML-Content" id="S1.F2.10.5.m5.1c"><apply id="S1.F2.10.5.m5.1.1.cmml" xref="S1.F2.10.5.m5.1.1"><neq id="S1.F2.10.5.m5.1.1.1.cmml" xref="S1.F2.10.5.m5.1.1.1"></neq><apply id="S1.F2.10.5.m5.1.1.2.cmml" xref="S1.F2.10.5.m5.1.1.2"><csymbol cd="ambiguous" id="S1.F2.10.5.m5.1.1.2.1.cmml" xref="S1.F2.10.5.m5.1.1.2">superscript</csymbol><ci id="S1.F2.10.5.m5.1.1.2.2.cmml" xref="S1.F2.10.5.m5.1.1.2.2">𝖳𝖢</ci><cn type="integer" id="S1.F2.10.5.m5.1.1.2.3.cmml" xref="S1.F2.10.5.m5.1.1.2.3">0</cn></apply><apply id="S1.F2.10.5.m5.1.1.3.cmml" xref="S1.F2.10.5.m5.1.1.3"><csymbol cd="ambiguous" id="S1.F2.10.5.m5.1.1.3.1.cmml" xref="S1.F2.10.5.m5.1.1.3">superscript</csymbol><ci id="S1.F2.10.5.m5.1.1.3.2.cmml" xref="S1.F2.10.5.m5.1.1.3.2">𝖭𝖢</ci><cn type="integer" id="S1.F2.10.5.m5.1.1.3.3.cmml" xref="S1.F2.10.5.m5.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.10.5.m5.1d">\mathsf{TC}^{0}\neq\mathsf{NC}^{1}</annotation></semantics></math>.</span></figcaption>
</figure>
<div id="S1.p4" class="ltx_para">
<p id="S1.p4.1" class="ltx_p">At first glance, our results may appear to contradict <cite class="ltx_cite ltx_citemacro_citet">Gu et al. (<a href="#bib.bib9" title="" class="ltx_ref">2021</a>)</cite>’s claim that linear SSMs can simulate general recurrent models, which can express permutation composition. But the contradiction is resolved by a difference in assumptions: <cite class="ltx_cite ltx_citemacro_citet">Gu et al. (<a href="#bib.bib9" title="" class="ltx_ref">2021</a>)</cite> relied on <em id="S1.p4.1.1" class="ltx_emph ltx_font_italic">infinite depth</em>
(number of layers) to show that SSMs could simulate RNNs. We, on the other hand, analyze the realistic setting with a bounded number of layers, under which we find that SSMs cannot simulate the recurrent state of an RNN and, in fact, suffer from similar limitations as transformers for state tracking.
</p>
</div>
<div id="S1.p5" class="ltx_para">
<p id="S1.p5.1" class="ltx_p">Our empirical investigation shows that, in practice, both SSMs with the Mamba architecture <cite class="ltx_cite ltx_citemacro_citep">(Gu &amp; Dao, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>)</cite> and transformers do <em id="S1.p5.1.1" class="ltx_emph ltx_font_italic">not</em> learn to solve the permutation composition state-tracking problem with a fixed number of layers, while simple RNNs can do so with just one layer. This provides empirical support for our theoretical separation in expressive power for state tracking between SSMs and true recurrent models. We also find that both transformers and SSMs struggle compared to RNNs on state-tracking problems less complex than permutation composition where it is not known whether they can express a solution.
Thus, in practice, SSMs may struggle not just on the hardest state-tracking problems like permutation composition but also on easier variants.</p>
</div>
<div id="S1.p6" class="ltx_para">
<p id="S1.p6.1" class="ltx_p">Finally, we propose two minimal extensions of linear SSMs that increase their expressive power for state tracking, allowing them to solve permutation composition. These extensions, however, may come with a cost: they may negatively impact parallelism as well as learning dynamics. We view it as an interesting open question whether it is possible to develop SSM-like models with greater expressivity for state tracking that also have strong parallelizability and learning dynamics, or whether these different goals are fundamentally at odds, as <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite> suggest.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Background</h2>

<div id="S2.p1" class="ltx_para">
<p id="S2.p1.1" class="ltx_p">We first present the SSM architectures we will analyze (<a href="#S2.SS1" title="2.1 Architecture of State-Space Models ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">2.1</span></a>).
Our analysis of the state tracking capabilities of SSMs borrows deeply from the circuit complexity and algebraic formal language theory literature. We thus review how circuit complexity can be used to analyze the power of neural networks (<a href="#S2.SS3" title="2.3 Limits of Transformers via Circuit Complexity ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">2.3</span></a>) and how state-tracking problems can be captured algebraically and analyzed within the circuit complexity framework (<a href="#S3.SS1" title="3.1 State Tracking as a Monoid Word Problem ‣ 3 State Tracking ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">3.1</span></a>).</p>
</div>
<section id="S2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.1 </span>Architecture of State-Space Models</h3>

<div id="S2.SS1.p1" class="ltx_para">
<p id="S2.SS1.p1.1" class="ltx_p">SSMs are a neural network architecture for processing sequences similar in design to RNNs or linear dynamical systems.
SSMs have been suggested to have two potential advantages compared to transformers owing to their recurrent formulation: faster inference and, possibly, the ability to better express inherently sequential or stateful problems <cite class="ltx_cite ltx_citemacro_citep">(Gu et al., <a href="#bib.bib9" title="" class="ltx_ref">2021</a>, <a href="#bib.bib11" title="" class="ltx_ref">2022b</a>)</cite>.
Several architectural variants of SSMs have been proposed, including S4 <cite class="ltx_cite ltx_citemacro_citep">(Gu et al., <a href="#bib.bib10" title="" class="ltx_ref">2022a</a>)</cite> and Mamba <cite class="ltx_cite ltx_citemacro_citep">(Gu &amp; Dao, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>)</cite>.
Recently, SSMs have been shown to achieve strong empirical performance compared to transformers in certain settings, particularly those involving a long context <cite class="ltx_cite ltx_citemacro_citep">(Gu &amp; Dao, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>; Wang et al., <a href="#bib.bib31" title="" class="ltx_ref">2024</a>)</cite>.</p>
</div>
<div id="S2.SS1.p2" class="ltx_para">
<p id="S2.SS1.p2.1" class="ltx_p">SSMs consist of <em id="S2.SS1.p2.1.1" class="ltx_emph ltx_font_italic">state-space layers</em>, which can be thought of as simplified RNN layers. We consider two variants of the state-space layer: the linear SSM layer (of which S4 is a special case; <cite class="ltx_cite ltx_citemacro_citep">Gu et al., <a href="#bib.bib10" title="" class="ltx_ref">2022a</a></cite>) and the extended S6 layer used by Mamba <cite class="ltx_cite ltx_citemacro_citep">(Gu &amp; Dao, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
<div id="S2.Thmtheorem1" class="ltx_theorem ltx_theorem_definition">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S2.Thmtheorem1.1.1.1" class="ltx_text ltx_font_bold">Definition 2.1</span></span><span id="S2.Thmtheorem1.2.2" class="ltx_text ltx_font_bold"> </span>(Linear SSM layer; e.g. S4)<span id="S2.Thmtheorem1.3.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S2.Thmtheorem1.p1" class="ltx_para">
<p id="S2.Thmtheorem1.p1.5" class="ltx_p">Given a sequence of embeddings or previous states <math id="S2.Thmtheorem1.p1.1.m1.3" class="ltx_Math" alttext="x_{1},\ldots,x_{n}\in\mathbb{R}^{m}" display="inline"><semantics id="S2.Thmtheorem1.p1.1.m1.3a"><mrow id="S2.Thmtheorem1.p1.1.m1.3.3" xref="S2.Thmtheorem1.p1.1.m1.3.3.cmml"><mrow id="S2.Thmtheorem1.p1.1.m1.3.3.2.2" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.3.cmml"><msub id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.cmml"><mi id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.2" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.2.cmml">x</mi><mn id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.3" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.3.cmml">1</mn></msub><mo id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.3" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.Thmtheorem1.p1.1.m1.1.1" xref="S2.Thmtheorem1.p1.1.m1.1.1.cmml">…</mi><mo id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.4" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.3.cmml">,</mo><msub id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.cmml"><mi id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.2" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.2.cmml">x</mi><mi id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.3" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.3.cmml">n</mi></msub></mrow><mo id="S2.Thmtheorem1.p1.1.m1.3.3.3" xref="S2.Thmtheorem1.p1.1.m1.3.3.3.cmml">∈</mo><msup id="S2.Thmtheorem1.p1.1.m1.3.3.4" xref="S2.Thmtheorem1.p1.1.m1.3.3.4.cmml"><mi id="S2.Thmtheorem1.p1.1.m1.3.3.4.2" xref="S2.Thmtheorem1.p1.1.m1.3.3.4.2.cmml">ℝ</mi><mi id="S2.Thmtheorem1.p1.1.m1.3.3.4.3" xref="S2.Thmtheorem1.p1.1.m1.3.3.4.3.cmml">m</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.1.m1.3b"><apply id="S2.Thmtheorem1.p1.1.m1.3.3.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3"><in id="S2.Thmtheorem1.p1.1.m1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.3"></in><list id="S2.Thmtheorem1.p1.1.m1.3.3.2.3.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2"><apply id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.cmml" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.1.cmml" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1">subscript</csymbol><ci id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.2.cmml" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.2">𝑥</ci><cn type="integer" id="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.3.cmml" xref="S2.Thmtheorem1.p1.1.m1.2.2.1.1.1.3">1</cn></apply><ci id="S2.Thmtheorem1.p1.1.m1.1.1.cmml" xref="S2.Thmtheorem1.p1.1.m1.1.1">…</ci><apply id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.1.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2">subscript</csymbol><ci id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.2.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.2">𝑥</ci><ci id="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.3.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.2.2.2.3">𝑛</ci></apply></list><apply id="S2.Thmtheorem1.p1.1.m1.3.3.4.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.4"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.1.m1.3.3.4.1.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.4">superscript</csymbol><ci id="S2.Thmtheorem1.p1.1.m1.3.3.4.2.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.4.2">ℝ</ci><ci id="S2.Thmtheorem1.p1.1.m1.3.3.4.3.cmml" xref="S2.Thmtheorem1.p1.1.m1.3.3.4.3">𝑚</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.1.m1.3c">x_{1},\ldots,x_{n}\in\mathbb{R}^{m}</annotation></semantics></math>, the <em id="S2.Thmtheorem1.p1.5.1" class="ltx_emph ltx_font_italic">recurrent form</em> of a linear SSM layer defines a new sequence of states <math id="S2.Thmtheorem1.p1.2.m2.3" class="ltx_Math" alttext="h_{1},\ldots,h_{n}\in\mathbb{R}^{d}" display="inline"><semantics id="S2.Thmtheorem1.p1.2.m2.3a"><mrow id="S2.Thmtheorem1.p1.2.m2.3.3" xref="S2.Thmtheorem1.p1.2.m2.3.3.cmml"><mrow id="S2.Thmtheorem1.p1.2.m2.3.3.2.2" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.3.cmml"><msub id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.cmml"><mi id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.2" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.2.cmml">h</mi><mn id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.3" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.3.cmml">1</mn></msub><mo id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.3" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.Thmtheorem1.p1.2.m2.1.1" xref="S2.Thmtheorem1.p1.2.m2.1.1.cmml">…</mi><mo id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.4" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.3.cmml">,</mo><msub id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.cmml"><mi id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.2" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.2.cmml">h</mi><mi id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.3" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.3.cmml">n</mi></msub></mrow><mo id="S2.Thmtheorem1.p1.2.m2.3.3.3" xref="S2.Thmtheorem1.p1.2.m2.3.3.3.cmml">∈</mo><msup id="S2.Thmtheorem1.p1.2.m2.3.3.4" xref="S2.Thmtheorem1.p1.2.m2.3.3.4.cmml"><mi id="S2.Thmtheorem1.p1.2.m2.3.3.4.2" xref="S2.Thmtheorem1.p1.2.m2.3.3.4.2.cmml">ℝ</mi><mi id="S2.Thmtheorem1.p1.2.m2.3.3.4.3" xref="S2.Thmtheorem1.p1.2.m2.3.3.4.3.cmml">d</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.2.m2.3b"><apply id="S2.Thmtheorem1.p1.2.m2.3.3.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3"><in id="S2.Thmtheorem1.p1.2.m2.3.3.3.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.3"></in><list id="S2.Thmtheorem1.p1.2.m2.3.3.2.3.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2"><apply id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.cmml" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.1.cmml" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1">subscript</csymbol><ci id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.2.cmml" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.2">ℎ</ci><cn type="integer" id="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.3.cmml" xref="S2.Thmtheorem1.p1.2.m2.2.2.1.1.1.3">1</cn></apply><ci id="S2.Thmtheorem1.p1.2.m2.1.1.cmml" xref="S2.Thmtheorem1.p1.2.m2.1.1">…</ci><apply id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.1.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2">subscript</csymbol><ci id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.2.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.2">ℎ</ci><ci id="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.3.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.2.2.2.3">𝑛</ci></apply></list><apply id="S2.Thmtheorem1.p1.2.m2.3.3.4.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.4"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.2.m2.3.3.4.1.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.4">superscript</csymbol><ci id="S2.Thmtheorem1.p1.2.m2.3.3.4.2.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.4.2">ℝ</ci><ci id="S2.Thmtheorem1.p1.2.m2.3.3.4.3.cmml" xref="S2.Thmtheorem1.p1.2.m2.3.3.4.3">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.2.m2.3c">h_{1},\ldots,h_{n}\in\mathbb{R}^{d}</annotation></semantics></math> using learned parameter matrices <math id="S2.Thmtheorem1.p1.3.m3.1" class="ltx_Math" alttext="A\in\mathbb{R}^{d\times d}" display="inline"><semantics id="S2.Thmtheorem1.p1.3.m3.1a"><mrow id="S2.Thmtheorem1.p1.3.m3.1.1" xref="S2.Thmtheorem1.p1.3.m3.1.1.cmml"><mi id="S2.Thmtheorem1.p1.3.m3.1.1.2" xref="S2.Thmtheorem1.p1.3.m3.1.1.2.cmml">A</mi><mo id="S2.Thmtheorem1.p1.3.m3.1.1.1" xref="S2.Thmtheorem1.p1.3.m3.1.1.1.cmml">∈</mo><msup id="S2.Thmtheorem1.p1.3.m3.1.1.3" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.cmml"><mi id="S2.Thmtheorem1.p1.3.m3.1.1.3.2" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.2.cmml">ℝ</mi><mrow id="S2.Thmtheorem1.p1.3.m3.1.1.3.3" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.cmml"><mi id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.2" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.2.cmml">d</mi><mo lspace="0.222em" rspace="0.222em" id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.1" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.1.cmml">×</mo><mi id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.3" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.3.cmml">d</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.3.m3.1b"><apply id="S2.Thmtheorem1.p1.3.m3.1.1.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1"><in id="S2.Thmtheorem1.p1.3.m3.1.1.1.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.1"></in><ci id="S2.Thmtheorem1.p1.3.m3.1.1.2.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.2">𝐴</ci><apply id="S2.Thmtheorem1.p1.3.m3.1.1.3.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.3.m3.1.1.3.1.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3">superscript</csymbol><ci id="S2.Thmtheorem1.p1.3.m3.1.1.3.2.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.2">ℝ</ci><apply id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3"><times id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.1.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.1"></times><ci id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.2.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.2">𝑑</ci><ci id="S2.Thmtheorem1.p1.3.m3.1.1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.3.m3.1.1.3.3.3">𝑑</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.3.m3.1c">A\in\mathbb{R}^{d\times d}</annotation></semantics></math> and <math id="S2.Thmtheorem1.p1.4.m4.1" class="ltx_Math" alttext="B\in\mathbb{R}^{d\times m}" display="inline"><semantics id="S2.Thmtheorem1.p1.4.m4.1a"><mrow id="S2.Thmtheorem1.p1.4.m4.1.1" xref="S2.Thmtheorem1.p1.4.m4.1.1.cmml"><mi id="S2.Thmtheorem1.p1.4.m4.1.1.2" xref="S2.Thmtheorem1.p1.4.m4.1.1.2.cmml">B</mi><mo id="S2.Thmtheorem1.p1.4.m4.1.1.1" xref="S2.Thmtheorem1.p1.4.m4.1.1.1.cmml">∈</mo><msup id="S2.Thmtheorem1.p1.4.m4.1.1.3" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.cmml"><mi id="S2.Thmtheorem1.p1.4.m4.1.1.3.2" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.2.cmml">ℝ</mi><mrow id="S2.Thmtheorem1.p1.4.m4.1.1.3.3" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.cmml"><mi id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.2" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.2.cmml">d</mi><mo lspace="0.222em" rspace="0.222em" id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.1" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.1.cmml">×</mo><mi id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.3" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.3.cmml">m</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.4.m4.1b"><apply id="S2.Thmtheorem1.p1.4.m4.1.1.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1"><in id="S2.Thmtheorem1.p1.4.m4.1.1.1.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.1"></in><ci id="S2.Thmtheorem1.p1.4.m4.1.1.2.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.2">𝐵</ci><apply id="S2.Thmtheorem1.p1.4.m4.1.1.3.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.4.m4.1.1.3.1.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3">superscript</csymbol><ci id="S2.Thmtheorem1.p1.4.m4.1.1.3.2.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.2">ℝ</ci><apply id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3"><times id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.1.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.1"></times><ci id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.2.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.2">𝑑</ci><ci id="S2.Thmtheorem1.p1.4.m4.1.1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.4.m4.1.1.3.3.3">𝑚</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.4.m4.1c">B\in\mathbb{R}^{d\times m}</annotation></semantics></math>. For each <math id="S2.Thmtheorem1.p1.5.m5.1" class="ltx_Math" alttext="1\leq i\leq n" display="inline"><semantics id="S2.Thmtheorem1.p1.5.m5.1a"><mrow id="S2.Thmtheorem1.p1.5.m5.1.1" xref="S2.Thmtheorem1.p1.5.m5.1.1.cmml"><mn id="S2.Thmtheorem1.p1.5.m5.1.1.2" xref="S2.Thmtheorem1.p1.5.m5.1.1.2.cmml">1</mn><mo id="S2.Thmtheorem1.p1.5.m5.1.1.3" xref="S2.Thmtheorem1.p1.5.m5.1.1.3.cmml">≤</mo><mi id="S2.Thmtheorem1.p1.5.m5.1.1.4" xref="S2.Thmtheorem1.p1.5.m5.1.1.4.cmml">i</mi><mo id="S2.Thmtheorem1.p1.5.m5.1.1.5" xref="S2.Thmtheorem1.p1.5.m5.1.1.5.cmml">≤</mo><mi id="S2.Thmtheorem1.p1.5.m5.1.1.6" xref="S2.Thmtheorem1.p1.5.m5.1.1.6.cmml">n</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.5.m5.1b"><apply id="S2.Thmtheorem1.p1.5.m5.1.1.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1"><and id="S2.Thmtheorem1.p1.5.m5.1.1a.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1"></and><apply id="S2.Thmtheorem1.p1.5.m5.1.1b.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1"><leq id="S2.Thmtheorem1.p1.5.m5.1.1.3.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1.3"></leq><cn type="integer" id="S2.Thmtheorem1.p1.5.m5.1.1.2.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1.2">1</cn><ci id="S2.Thmtheorem1.p1.5.m5.1.1.4.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1.4">𝑖</ci></apply><apply id="S2.Thmtheorem1.p1.5.m5.1.1c.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1"><leq id="S2.Thmtheorem1.p1.5.m5.1.1.5.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1.5"></leq><share href="#S2.Thmtheorem1.p1.5.m5.1.1.4.cmml" id="S2.Thmtheorem1.p1.5.m5.1.1d.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1"></share><ci id="S2.Thmtheorem1.p1.5.m5.1.1.6.cmml" xref="S2.Thmtheorem1.p1.5.m5.1.1.6">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.5.m5.1c">1\leq i\leq n</annotation></semantics></math>,</p>
<table id="S2.E1" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E1.m1.1" class="ltx_Math" alttext="h_{i}=Ah_{i-1}+Bx_{i}." display="block"><semantics id="S2.E1.m1.1a"><mrow id="S2.E1.m1.1.1.1" xref="S2.E1.m1.1.1.1.1.cmml"><mrow id="S2.E1.m1.1.1.1.1" xref="S2.E1.m1.1.1.1.1.cmml"><msub id="S2.E1.m1.1.1.1.1.2" xref="S2.E1.m1.1.1.1.1.2.cmml"><mi id="S2.E1.m1.1.1.1.1.2.2" xref="S2.E1.m1.1.1.1.1.2.2.cmml">h</mi><mi id="S2.E1.m1.1.1.1.1.2.3" xref="S2.E1.m1.1.1.1.1.2.3.cmml">i</mi></msub><mo id="S2.E1.m1.1.1.1.1.1" xref="S2.E1.m1.1.1.1.1.1.cmml">=</mo><mrow id="S2.E1.m1.1.1.1.1.3" xref="S2.E1.m1.1.1.1.1.3.cmml"><mrow id="S2.E1.m1.1.1.1.1.3.2" xref="S2.E1.m1.1.1.1.1.3.2.cmml"><mi id="S2.E1.m1.1.1.1.1.3.2.2" xref="S2.E1.m1.1.1.1.1.3.2.2.cmml">A</mi><mo lspace="0em" rspace="0em" id="S2.E1.m1.1.1.1.1.3.2.1" xref="S2.E1.m1.1.1.1.1.3.2.1.cmml">​</mo><msub id="S2.E1.m1.1.1.1.1.3.2.3" xref="S2.E1.m1.1.1.1.1.3.2.3.cmml"><mi id="S2.E1.m1.1.1.1.1.3.2.3.2" xref="S2.E1.m1.1.1.1.1.3.2.3.2.cmml">h</mi><mrow id="S2.E1.m1.1.1.1.1.3.2.3.3" xref="S2.E1.m1.1.1.1.1.3.2.3.3.cmml"><mi id="S2.E1.m1.1.1.1.1.3.2.3.3.2" xref="S2.E1.m1.1.1.1.1.3.2.3.3.2.cmml">i</mi><mo id="S2.E1.m1.1.1.1.1.3.2.3.3.1" xref="S2.E1.m1.1.1.1.1.3.2.3.3.1.cmml">−</mo><mn id="S2.E1.m1.1.1.1.1.3.2.3.3.3" xref="S2.E1.m1.1.1.1.1.3.2.3.3.3.cmml">1</mn></mrow></msub></mrow><mo id="S2.E1.m1.1.1.1.1.3.1" xref="S2.E1.m1.1.1.1.1.3.1.cmml">+</mo><mrow id="S2.E1.m1.1.1.1.1.3.3" xref="S2.E1.m1.1.1.1.1.3.3.cmml"><mi id="S2.E1.m1.1.1.1.1.3.3.2" xref="S2.E1.m1.1.1.1.1.3.3.2.cmml">B</mi><mo lspace="0em" rspace="0em" id="S2.E1.m1.1.1.1.1.3.3.1" xref="S2.E1.m1.1.1.1.1.3.3.1.cmml">​</mo><msub id="S2.E1.m1.1.1.1.1.3.3.3" xref="S2.E1.m1.1.1.1.1.3.3.3.cmml"><mi id="S2.E1.m1.1.1.1.1.3.3.3.2" xref="S2.E1.m1.1.1.1.1.3.3.3.2.cmml">x</mi><mi id="S2.E1.m1.1.1.1.1.3.3.3.3" xref="S2.E1.m1.1.1.1.1.3.3.3.3.cmml">i</mi></msub></mrow></mrow></mrow><mo lspace="0em" id="S2.E1.m1.1.1.1.2" xref="S2.E1.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.E1.m1.1b"><apply id="S2.E1.m1.1.1.1.1.cmml" xref="S2.E1.m1.1.1.1"><eq id="S2.E1.m1.1.1.1.1.1.cmml" xref="S2.E1.m1.1.1.1.1.1"></eq><apply id="S2.E1.m1.1.1.1.1.2.cmml" xref="S2.E1.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.E1.m1.1.1.1.1.2.1.cmml" xref="S2.E1.m1.1.1.1.1.2">subscript</csymbol><ci id="S2.E1.m1.1.1.1.1.2.2.cmml" xref="S2.E1.m1.1.1.1.1.2.2">ℎ</ci><ci id="S2.E1.m1.1.1.1.1.2.3.cmml" xref="S2.E1.m1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S2.E1.m1.1.1.1.1.3.cmml" xref="S2.E1.m1.1.1.1.1.3"><plus id="S2.E1.m1.1.1.1.1.3.1.cmml" xref="S2.E1.m1.1.1.1.1.3.1"></plus><apply id="S2.E1.m1.1.1.1.1.3.2.cmml" xref="S2.E1.m1.1.1.1.1.3.2"><times id="S2.E1.m1.1.1.1.1.3.2.1.cmml" xref="S2.E1.m1.1.1.1.1.3.2.1"></times><ci id="S2.E1.m1.1.1.1.1.3.2.2.cmml" xref="S2.E1.m1.1.1.1.1.3.2.2">𝐴</ci><apply id="S2.E1.m1.1.1.1.1.3.2.3.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3"><csymbol cd="ambiguous" id="S2.E1.m1.1.1.1.1.3.2.3.1.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3">subscript</csymbol><ci id="S2.E1.m1.1.1.1.1.3.2.3.2.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3.2">ℎ</ci><apply id="S2.E1.m1.1.1.1.1.3.2.3.3.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3.3"><minus id="S2.E1.m1.1.1.1.1.3.2.3.3.1.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3.3.1"></minus><ci id="S2.E1.m1.1.1.1.1.3.2.3.3.2.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3.3.2">𝑖</ci><cn type="integer" id="S2.E1.m1.1.1.1.1.3.2.3.3.3.cmml" xref="S2.E1.m1.1.1.1.1.3.2.3.3.3">1</cn></apply></apply></apply><apply id="S2.E1.m1.1.1.1.1.3.3.cmml" xref="S2.E1.m1.1.1.1.1.3.3"><times id="S2.E1.m1.1.1.1.1.3.3.1.cmml" xref="S2.E1.m1.1.1.1.1.3.3.1"></times><ci id="S2.E1.m1.1.1.1.1.3.3.2.cmml" xref="S2.E1.m1.1.1.1.1.3.3.2">𝐵</ci><apply id="S2.E1.m1.1.1.1.1.3.3.3.cmml" xref="S2.E1.m1.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.E1.m1.1.1.1.1.3.3.3.1.cmml" xref="S2.E1.m1.1.1.1.1.3.3.3">subscript</csymbol><ci id="S2.E1.m1.1.1.1.1.3.3.3.2.cmml" xref="S2.E1.m1.1.1.1.1.3.3.3.2">𝑥</ci><ci id="S2.E1.m1.1.1.1.1.3.3.3.3.cmml" xref="S2.E1.m1.1.1.1.1.3.3.3.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E1.m1.1c">h_{i}=Ah_{i-1}+Bx_{i}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(1)</span></td>
</tr></tbody>
</table>
<p id="S2.Thmtheorem1.p1.6" class="ltx_p">The <em id="S2.Thmtheorem1.p1.6.1" class="ltx_emph ltx_font_italic">convolutional form</em> of the linear SSM layer defines the same<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>The two forms express the same function over <math id="footnote2.m1.1" class="ltx_Math" alttext="\mathbb{R}" display="inline"><semantics id="footnote2.m1.1b"><mi id="footnote2.m1.1.1" xref="footnote2.m1.1.1.cmml">ℝ</mi><annotation-xml encoding="MathML-Content" id="footnote2.m1.1c"><ci id="footnote2.m1.1.1.cmml" xref="footnote2.m1.1.1">ℝ</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote2.m1.1d">\mathbb{R}</annotation></semantics></math> or any other distributive datatype. Over floating points (<a href="#A1" title="Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">A</span></a>), they are not guaranteed to be the same, but we must assume the error is negligible for them to be well-defined and usable in practice.</span></span></span> <math id="S2.Thmtheorem1.p1.6.m1.3" class="ltx_Math" alttext="h_{1},\ldots,h_{n}" display="inline"><semantics id="S2.Thmtheorem1.p1.6.m1.3a"><mrow id="S2.Thmtheorem1.p1.6.m1.3.3.2" xref="S2.Thmtheorem1.p1.6.m1.3.3.3.cmml"><msub id="S2.Thmtheorem1.p1.6.m1.2.2.1.1" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1.cmml"><mi id="S2.Thmtheorem1.p1.6.m1.2.2.1.1.2" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1.2.cmml">h</mi><mn id="S2.Thmtheorem1.p1.6.m1.2.2.1.1.3" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1.3.cmml">1</mn></msub><mo id="S2.Thmtheorem1.p1.6.m1.3.3.2.3" xref="S2.Thmtheorem1.p1.6.m1.3.3.3.cmml">,</mo><mi mathvariant="normal" id="S2.Thmtheorem1.p1.6.m1.1.1" xref="S2.Thmtheorem1.p1.6.m1.1.1.cmml">…</mi><mo id="S2.Thmtheorem1.p1.6.m1.3.3.2.4" xref="S2.Thmtheorem1.p1.6.m1.3.3.3.cmml">,</mo><msub id="S2.Thmtheorem1.p1.6.m1.3.3.2.2" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2.cmml"><mi id="S2.Thmtheorem1.p1.6.m1.3.3.2.2.2" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2.2.cmml">h</mi><mi id="S2.Thmtheorem1.p1.6.m1.3.3.2.2.3" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2.3.cmml">n</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.6.m1.3b"><list id="S2.Thmtheorem1.p1.6.m1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.6.m1.3.3.2"><apply id="S2.Thmtheorem1.p1.6.m1.2.2.1.1.cmml" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.6.m1.2.2.1.1.1.cmml" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1">subscript</csymbol><ci id="S2.Thmtheorem1.p1.6.m1.2.2.1.1.2.cmml" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1.2">ℎ</ci><cn type="integer" id="S2.Thmtheorem1.p1.6.m1.2.2.1.1.3.cmml" xref="S2.Thmtheorem1.p1.6.m1.2.2.1.1.3">1</cn></apply><ci id="S2.Thmtheorem1.p1.6.m1.1.1.cmml" xref="S2.Thmtheorem1.p1.6.m1.1.1">…</ci><apply id="S2.Thmtheorem1.p1.6.m1.3.3.2.2.cmml" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.6.m1.3.3.2.2.1.cmml" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2">subscript</csymbol><ci id="S2.Thmtheorem1.p1.6.m1.3.3.2.2.2.cmml" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2.2">ℎ</ci><ci id="S2.Thmtheorem1.p1.6.m1.3.3.2.2.3.cmml" xref="S2.Thmtheorem1.p1.6.m1.3.3.2.2.3">𝑛</ci></apply></list></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.6.m1.3c">h_{1},\ldots,h_{n}</annotation></semantics></math> but is unrolled as a summation of terms:</p>
<table id="S2.E2" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E2.m1.1" class="ltx_Math" alttext="h_{i}=\sum_{j=1}^{i}A^{i-j}Bx_{j}." display="block"><semantics id="S2.E2.m1.1a"><mrow id="S2.E2.m1.1.1.1" xref="S2.E2.m1.1.1.1.1.cmml"><mrow id="S2.E2.m1.1.1.1.1" xref="S2.E2.m1.1.1.1.1.cmml"><msub id="S2.E2.m1.1.1.1.1.2" xref="S2.E2.m1.1.1.1.1.2.cmml"><mi id="S2.E2.m1.1.1.1.1.2.2" xref="S2.E2.m1.1.1.1.1.2.2.cmml">h</mi><mi id="S2.E2.m1.1.1.1.1.2.3" xref="S2.E2.m1.1.1.1.1.2.3.cmml">i</mi></msub><mo rspace="0.111em" id="S2.E2.m1.1.1.1.1.1" xref="S2.E2.m1.1.1.1.1.1.cmml">=</mo><mrow id="S2.E2.m1.1.1.1.1.3" xref="S2.E2.m1.1.1.1.1.3.cmml"><munderover id="S2.E2.m1.1.1.1.1.3.1" xref="S2.E2.m1.1.1.1.1.3.1.cmml"><mo movablelimits="false" id="S2.E2.m1.1.1.1.1.3.1.2.2" xref="S2.E2.m1.1.1.1.1.3.1.2.2.cmml">∑</mo><mrow id="S2.E2.m1.1.1.1.1.3.1.2.3" xref="S2.E2.m1.1.1.1.1.3.1.2.3.cmml"><mi id="S2.E2.m1.1.1.1.1.3.1.2.3.2" xref="S2.E2.m1.1.1.1.1.3.1.2.3.2.cmml">j</mi><mo id="S2.E2.m1.1.1.1.1.3.1.2.3.1" xref="S2.E2.m1.1.1.1.1.3.1.2.3.1.cmml">=</mo><mn id="S2.E2.m1.1.1.1.1.3.1.2.3.3" xref="S2.E2.m1.1.1.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="S2.E2.m1.1.1.1.1.3.1.3" xref="S2.E2.m1.1.1.1.1.3.1.3.cmml">i</mi></munderover><mrow id="S2.E2.m1.1.1.1.1.3.2" xref="S2.E2.m1.1.1.1.1.3.2.cmml"><msup id="S2.E2.m1.1.1.1.1.3.2.2" xref="S2.E2.m1.1.1.1.1.3.2.2.cmml"><mi id="S2.E2.m1.1.1.1.1.3.2.2.2" xref="S2.E2.m1.1.1.1.1.3.2.2.2.cmml">A</mi><mrow id="S2.E2.m1.1.1.1.1.3.2.2.3" xref="S2.E2.m1.1.1.1.1.3.2.2.3.cmml"><mi id="S2.E2.m1.1.1.1.1.3.2.2.3.2" xref="S2.E2.m1.1.1.1.1.3.2.2.3.2.cmml">i</mi><mo id="S2.E2.m1.1.1.1.1.3.2.2.3.1" xref="S2.E2.m1.1.1.1.1.3.2.2.3.1.cmml">−</mo><mi id="S2.E2.m1.1.1.1.1.3.2.2.3.3" xref="S2.E2.m1.1.1.1.1.3.2.2.3.3.cmml">j</mi></mrow></msup><mo lspace="0em" rspace="0em" id="S2.E2.m1.1.1.1.1.3.2.1" xref="S2.E2.m1.1.1.1.1.3.2.1.cmml">​</mo><mi id="S2.E2.m1.1.1.1.1.3.2.3" xref="S2.E2.m1.1.1.1.1.3.2.3.cmml">B</mi><mo lspace="0em" rspace="0em" id="S2.E2.m1.1.1.1.1.3.2.1a" xref="S2.E2.m1.1.1.1.1.3.2.1.cmml">​</mo><msub id="S2.E2.m1.1.1.1.1.3.2.4" xref="S2.E2.m1.1.1.1.1.3.2.4.cmml"><mi id="S2.E2.m1.1.1.1.1.3.2.4.2" xref="S2.E2.m1.1.1.1.1.3.2.4.2.cmml">x</mi><mi id="S2.E2.m1.1.1.1.1.3.2.4.3" xref="S2.E2.m1.1.1.1.1.3.2.4.3.cmml">j</mi></msub></mrow></mrow></mrow><mo lspace="0em" id="S2.E2.m1.1.1.1.2" xref="S2.E2.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.E2.m1.1b"><apply id="S2.E2.m1.1.1.1.1.cmml" xref="S2.E2.m1.1.1.1"><eq id="S2.E2.m1.1.1.1.1.1.cmml" xref="S2.E2.m1.1.1.1.1.1"></eq><apply id="S2.E2.m1.1.1.1.1.2.cmml" xref="S2.E2.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.E2.m1.1.1.1.1.2.1.cmml" xref="S2.E2.m1.1.1.1.1.2">subscript</csymbol><ci id="S2.E2.m1.1.1.1.1.2.2.cmml" xref="S2.E2.m1.1.1.1.1.2.2">ℎ</ci><ci id="S2.E2.m1.1.1.1.1.2.3.cmml" xref="S2.E2.m1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S2.E2.m1.1.1.1.1.3.cmml" xref="S2.E2.m1.1.1.1.1.3"><apply id="S2.E2.m1.1.1.1.1.3.1.cmml" xref="S2.E2.m1.1.1.1.1.3.1"><csymbol cd="ambiguous" id="S2.E2.m1.1.1.1.1.3.1.1.cmml" xref="S2.E2.m1.1.1.1.1.3.1">superscript</csymbol><apply id="S2.E2.m1.1.1.1.1.3.1.2.cmml" xref="S2.E2.m1.1.1.1.1.3.1"><csymbol cd="ambiguous" id="S2.E2.m1.1.1.1.1.3.1.2.1.cmml" xref="S2.E2.m1.1.1.1.1.3.1">subscript</csymbol><sum id="S2.E2.m1.1.1.1.1.3.1.2.2.cmml" xref="S2.E2.m1.1.1.1.1.3.1.2.2"></sum><apply id="S2.E2.m1.1.1.1.1.3.1.2.3.cmml" xref="S2.E2.m1.1.1.1.1.3.1.2.3"><eq id="S2.E2.m1.1.1.1.1.3.1.2.3.1.cmml" xref="S2.E2.m1.1.1.1.1.3.1.2.3.1"></eq><ci id="S2.E2.m1.1.1.1.1.3.1.2.3.2.cmml" xref="S2.E2.m1.1.1.1.1.3.1.2.3.2">𝑗</ci><cn type="integer" id="S2.E2.m1.1.1.1.1.3.1.2.3.3.cmml" xref="S2.E2.m1.1.1.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="S2.E2.m1.1.1.1.1.3.1.3.cmml" xref="S2.E2.m1.1.1.1.1.3.1.3">𝑖</ci></apply><apply id="S2.E2.m1.1.1.1.1.3.2.cmml" xref="S2.E2.m1.1.1.1.1.3.2"><times id="S2.E2.m1.1.1.1.1.3.2.1.cmml" xref="S2.E2.m1.1.1.1.1.3.2.1"></times><apply id="S2.E2.m1.1.1.1.1.3.2.2.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2"><csymbol cd="ambiguous" id="S2.E2.m1.1.1.1.1.3.2.2.1.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2">superscript</csymbol><ci id="S2.E2.m1.1.1.1.1.3.2.2.2.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2.2">𝐴</ci><apply id="S2.E2.m1.1.1.1.1.3.2.2.3.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2.3"><minus id="S2.E2.m1.1.1.1.1.3.2.2.3.1.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2.3.1"></minus><ci id="S2.E2.m1.1.1.1.1.3.2.2.3.2.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2.3.2">𝑖</ci><ci id="S2.E2.m1.1.1.1.1.3.2.2.3.3.cmml" xref="S2.E2.m1.1.1.1.1.3.2.2.3.3">𝑗</ci></apply></apply><ci id="S2.E2.m1.1.1.1.1.3.2.3.cmml" xref="S2.E2.m1.1.1.1.1.3.2.3">𝐵</ci><apply id="S2.E2.m1.1.1.1.1.3.2.4.cmml" xref="S2.E2.m1.1.1.1.1.3.2.4"><csymbol cd="ambiguous" id="S2.E2.m1.1.1.1.1.3.2.4.1.cmml" xref="S2.E2.m1.1.1.1.1.3.2.4">subscript</csymbol><ci id="S2.E2.m1.1.1.1.1.3.2.4.2.cmml" xref="S2.E2.m1.1.1.1.1.3.2.4.2">𝑥</ci><ci id="S2.E2.m1.1.1.1.1.3.2.4.3.cmml" xref="S2.E2.m1.1.1.1.1.3.2.4.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E2.m1.1c">h_{i}=\sum_{j=1}^{i}A^{i-j}Bx_{j}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(2)</span></td>
</tr></tbody>
</table>
<p id="S2.Thmtheorem1.p1.9" class="ltx_p">The layer outputs <math id="S2.Thmtheorem1.p1.7.m1.1" class="ltx_Math" alttext="y_{i}=Ch_{i}+Dx_{i}" display="inline"><semantics id="S2.Thmtheorem1.p1.7.m1.1a"><mrow id="S2.Thmtheorem1.p1.7.m1.1.1" xref="S2.Thmtheorem1.p1.7.m1.1.1.cmml"><msub id="S2.Thmtheorem1.p1.7.m1.1.1.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.2.cmml"><mi id="S2.Thmtheorem1.p1.7.m1.1.1.2.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.2.2.cmml">y</mi><mi id="S2.Thmtheorem1.p1.7.m1.1.1.2.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.2.3.cmml">i</mi></msub><mo id="S2.Thmtheorem1.p1.7.m1.1.1.1" xref="S2.Thmtheorem1.p1.7.m1.1.1.1.cmml">=</mo><mrow id="S2.Thmtheorem1.p1.7.m1.1.1.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.cmml"><mrow id="S2.Thmtheorem1.p1.7.m1.1.1.3.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.cmml"><mi id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.2.cmml">C</mi><mo lspace="0em" rspace="0em" id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.1" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.1.cmml">​</mo><msub id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.cmml"><mi id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.2.cmml">h</mi><mi id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.3.cmml">i</mi></msub></mrow><mo id="S2.Thmtheorem1.p1.7.m1.1.1.3.1" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.1.cmml">+</mo><mrow id="S2.Thmtheorem1.p1.7.m1.1.1.3.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.cmml"><mi id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.2.cmml">D</mi><mo lspace="0em" rspace="0em" id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.1" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.1.cmml">​</mo><msub id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.cmml"><mi id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.2" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.2.cmml">x</mi><mi id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.3" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.3.cmml">i</mi></msub></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.7.m1.1b"><apply id="S2.Thmtheorem1.p1.7.m1.1.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1"><eq id="S2.Thmtheorem1.p1.7.m1.1.1.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.1"></eq><apply id="S2.Thmtheorem1.p1.7.m1.1.1.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.2"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.7.m1.1.1.2.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.2">subscript</csymbol><ci id="S2.Thmtheorem1.p1.7.m1.1.1.2.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.2.2">𝑦</ci><ci id="S2.Thmtheorem1.p1.7.m1.1.1.2.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.2.3">𝑖</ci></apply><apply id="S2.Thmtheorem1.p1.7.m1.1.1.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3"><plus id="S2.Thmtheorem1.p1.7.m1.1.1.3.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.1"></plus><apply id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2"><times id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.1"></times><ci id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.2">𝐶</ci><apply id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3">subscript</csymbol><ci id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.2">ℎ</ci><ci id="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.2.3.3">𝑖</ci></apply></apply><apply id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3"><times id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.1"></times><ci id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.2">𝐷</ci><apply id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.1.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3">subscript</csymbol><ci id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.2.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.2">𝑥</ci><ci id="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.3.cmml" xref="S2.Thmtheorem1.p1.7.m1.1.1.3.3.3.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.7.m1.1c">y_{i}=Ch_{i}+Dx_{i}</annotation></semantics></math> where <math id="S2.Thmtheorem1.p1.8.m2.1" class="ltx_Math" alttext="C\in\mathbb{R}^{d\times m}" display="inline"><semantics id="S2.Thmtheorem1.p1.8.m2.1a"><mrow id="S2.Thmtheorem1.p1.8.m2.1.1" xref="S2.Thmtheorem1.p1.8.m2.1.1.cmml"><mi id="S2.Thmtheorem1.p1.8.m2.1.1.2" xref="S2.Thmtheorem1.p1.8.m2.1.1.2.cmml">C</mi><mo id="S2.Thmtheorem1.p1.8.m2.1.1.1" xref="S2.Thmtheorem1.p1.8.m2.1.1.1.cmml">∈</mo><msup id="S2.Thmtheorem1.p1.8.m2.1.1.3" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.cmml"><mi id="S2.Thmtheorem1.p1.8.m2.1.1.3.2" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.2.cmml">ℝ</mi><mrow id="S2.Thmtheorem1.p1.8.m2.1.1.3.3" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.cmml"><mi id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.2" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.2.cmml">d</mi><mo lspace="0.222em" rspace="0.222em" id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.1" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.1.cmml">×</mo><mi id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.3" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.3.cmml">m</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.8.m2.1b"><apply id="S2.Thmtheorem1.p1.8.m2.1.1.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1"><in id="S2.Thmtheorem1.p1.8.m2.1.1.1.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.1"></in><ci id="S2.Thmtheorem1.p1.8.m2.1.1.2.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.2">𝐶</ci><apply id="S2.Thmtheorem1.p1.8.m2.1.1.3.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.8.m2.1.1.3.1.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3">superscript</csymbol><ci id="S2.Thmtheorem1.p1.8.m2.1.1.3.2.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.2">ℝ</ci><apply id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3"><times id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.1.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.1"></times><ci id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.2.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.2">𝑑</ci><ci id="S2.Thmtheorem1.p1.8.m2.1.1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.8.m2.1.1.3.3.3">𝑚</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.8.m2.1c">C\in\mathbb{R}^{d\times m}</annotation></semantics></math> and <math id="S2.Thmtheorem1.p1.9.m3.1" class="ltx_Math" alttext="D\in\mathbb{R}^{m\times m}" display="inline"><semantics id="S2.Thmtheorem1.p1.9.m3.1a"><mrow id="S2.Thmtheorem1.p1.9.m3.1.1" xref="S2.Thmtheorem1.p1.9.m3.1.1.cmml"><mi id="S2.Thmtheorem1.p1.9.m3.1.1.2" xref="S2.Thmtheorem1.p1.9.m3.1.1.2.cmml">D</mi><mo id="S2.Thmtheorem1.p1.9.m3.1.1.1" xref="S2.Thmtheorem1.p1.9.m3.1.1.1.cmml">∈</mo><msup id="S2.Thmtheorem1.p1.9.m3.1.1.3" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.cmml"><mi id="S2.Thmtheorem1.p1.9.m3.1.1.3.2" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.2.cmml">ℝ</mi><mrow id="S2.Thmtheorem1.p1.9.m3.1.1.3.3" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.cmml"><mi id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.2" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.2.cmml">m</mi><mo lspace="0.222em" rspace="0.222em" id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.1" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.1.cmml">×</mo><mi id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.3" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.3.cmml">m</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem1.p1.9.m3.1b"><apply id="S2.Thmtheorem1.p1.9.m3.1.1.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1"><in id="S2.Thmtheorem1.p1.9.m3.1.1.1.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.1"></in><ci id="S2.Thmtheorem1.p1.9.m3.1.1.2.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.2">𝐷</ci><apply id="S2.Thmtheorem1.p1.9.m3.1.1.3.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3"><csymbol cd="ambiguous" id="S2.Thmtheorem1.p1.9.m3.1.1.3.1.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3">superscript</csymbol><ci id="S2.Thmtheorem1.p1.9.m3.1.1.3.2.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.2">ℝ</ci><apply id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3"><times id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.1.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.1"></times><ci id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.2.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.2">𝑚</ci><ci id="S2.Thmtheorem1.p1.9.m3.1.1.3.3.3.cmml" xref="S2.Thmtheorem1.p1.9.m3.1.1.3.3.3">𝑚</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem1.p1.9.m3.1c">D\in\mathbb{R}^{m\times m}</annotation></semantics></math>.
</p>
</div>
</div>
<div id="S2.SS1.p3" class="ltx_para">
<p id="S2.SS1.p3.1" class="ltx_p">S4 chooses a specific parameterization of a linear layer.
A full S4 model is a cascade of such layers and feedforward layers, analogous to how transformers alternate multihead-self-attention layers with feedforward layers.</p>
</div>
<div id="S2.SS1.p4" class="ltx_para">
<p id="S2.SS1.p4.1" class="ltx_p">The S6 layer used by Mamba <cite class="ltx_cite ltx_citemacro_citep">(Gu &amp; Dao, <a href="#bib.bib8" title="" class="ltx_ref">2023</a>)</cite> generalizes a linear SSM layer by adding a <em id="S2.SS1.p4.1.1" class="ltx_emph ltx_font_italic">selection mechanism</em> inspired by the dynamic gating in LSTMs <cite class="ltx_cite ltx_citemacro_citep">(Hochreiter &amp; Schmidhuber, <a href="#bib.bib16" title="" class="ltx_ref">1997</a>)</cite> and GRUs <cite class="ltx_cite ltx_citemacro_citep">(Cho et al., <a href="#bib.bib5" title="" class="ltx_ref">2014</a>)</cite>.</p>
</div>
<div id="S2.Thmtheorem2" class="ltx_theorem ltx_theorem_definition">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S2.Thmtheorem2.1.1.1" class="ltx_text ltx_font_bold">Definition 2.2</span></span><span id="S2.Thmtheorem2.2.2" class="ltx_text ltx_font_bold"> </span>(S6 layer)<span id="S2.Thmtheorem2.3.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S2.Thmtheorem2.p1" class="ltx_para">
<p id="S2.Thmtheorem2.p1.5" class="ltx_p">An S6 layer is parameterized by a <em id="S2.Thmtheorem2.p1.5.1" class="ltx_emph ltx_font_italic">diagonal</em> matrix <math id="S2.Thmtheorem2.p1.1.m1.1" class="ltx_Math" alttext="A\in\mathbb{R}^{d\times d}" display="inline"><semantics id="S2.Thmtheorem2.p1.1.m1.1a"><mrow id="S2.Thmtheorem2.p1.1.m1.1.1" xref="S2.Thmtheorem2.p1.1.m1.1.1.cmml"><mi id="S2.Thmtheorem2.p1.1.m1.1.1.2" xref="S2.Thmtheorem2.p1.1.m1.1.1.2.cmml">A</mi><mo id="S2.Thmtheorem2.p1.1.m1.1.1.1" xref="S2.Thmtheorem2.p1.1.m1.1.1.1.cmml">∈</mo><msup id="S2.Thmtheorem2.p1.1.m1.1.1.3" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.cmml"><mi id="S2.Thmtheorem2.p1.1.m1.1.1.3.2" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.2.cmml">ℝ</mi><mrow id="S2.Thmtheorem2.p1.1.m1.1.1.3.3" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.cmml"><mi id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.2" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.2.cmml">d</mi><mo lspace="0.222em" rspace="0.222em" id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.1" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.1.cmml">×</mo><mi id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.3" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.3.cmml">d</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.1.m1.1b"><apply id="S2.Thmtheorem2.p1.1.m1.1.1.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1"><in id="S2.Thmtheorem2.p1.1.m1.1.1.1.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.1"></in><ci id="S2.Thmtheorem2.p1.1.m1.1.1.2.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.2">𝐴</ci><apply id="S2.Thmtheorem2.p1.1.m1.1.1.3.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.1.m1.1.1.3.1.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3">superscript</csymbol><ci id="S2.Thmtheorem2.p1.1.m1.1.1.3.2.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.2">ℝ</ci><apply id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3"><times id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.1.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.1"></times><ci id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.2.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.2">𝑑</ci><ci id="S2.Thmtheorem2.p1.1.m1.1.1.3.3.3.cmml" xref="S2.Thmtheorem2.p1.1.m1.1.1.3.3.3">𝑑</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.1.m1.1c">A\in\mathbb{R}^{d\times d}</annotation></semantics></math>, a vector <math id="S2.Thmtheorem2.p1.2.m2.1" class="ltx_Math" alttext="\delta\in\mathbb{R}^{d}" display="inline"><semantics id="S2.Thmtheorem2.p1.2.m2.1a"><mrow id="S2.Thmtheorem2.p1.2.m2.1.1" xref="S2.Thmtheorem2.p1.2.m2.1.1.cmml"><mi id="S2.Thmtheorem2.p1.2.m2.1.1.2" xref="S2.Thmtheorem2.p1.2.m2.1.1.2.cmml">δ</mi><mo id="S2.Thmtheorem2.p1.2.m2.1.1.1" xref="S2.Thmtheorem2.p1.2.m2.1.1.1.cmml">∈</mo><msup id="S2.Thmtheorem2.p1.2.m2.1.1.3" xref="S2.Thmtheorem2.p1.2.m2.1.1.3.cmml"><mi id="S2.Thmtheorem2.p1.2.m2.1.1.3.2" xref="S2.Thmtheorem2.p1.2.m2.1.1.3.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.2.m2.1.1.3.3" xref="S2.Thmtheorem2.p1.2.m2.1.1.3.3.cmml">d</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.2.m2.1b"><apply id="S2.Thmtheorem2.p1.2.m2.1.1.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1"><in id="S2.Thmtheorem2.p1.2.m2.1.1.1.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1.1"></in><ci id="S2.Thmtheorem2.p1.2.m2.1.1.2.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1.2">𝛿</ci><apply id="S2.Thmtheorem2.p1.2.m2.1.1.3.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1.3"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.2.m2.1.1.3.1.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1.3">superscript</csymbol><ci id="S2.Thmtheorem2.p1.2.m2.1.1.3.2.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1.3.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.2.m2.1.1.3.3.cmml" xref="S2.Thmtheorem2.p1.2.m2.1.1.3.3">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.2.m2.1c">\delta\in\mathbb{R}^{d}</annotation></semantics></math>, and affine projections <math id="S2.Thmtheorem2.p1.3.m3.1" class="ltx_Math" alttext="s_{\delta}:\mathbb{R}^{m}\to\mathbb{R}^{d},s_{B}:\mathbb{R}^{m}\to\mathbb{R}^{d}," display="inline"><semantics id="S2.Thmtheorem2.p1.3.m3.1a"><mrow id="S2.Thmtheorem2.p1.3.m3.1.1.1" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.cmml"><mrow id="S2.Thmtheorem2.p1.3.m3.1.1.1.1" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.cmml"><msub id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.cmml"><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.2.cmml">s</mi><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.3.cmml">δ</mi></msub><mo lspace="0.278em" rspace="0.278em" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.5" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.5.cmml">:</mo><mrow id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.cmml"><msup id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.cmml"><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.3.cmml">m</mi></msup><mo stretchy="false" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.3.cmml">→</mo><mrow id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.3.cmml"><msup id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.cmml"><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.3.cmml">d</mi></msup><mo id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.3.cmml">,</mo><msub id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.cmml"><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.2.cmml">s</mi><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.3.cmml">B</mi></msub></mrow></mrow><mo lspace="0.278em" rspace="0.278em" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.6" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.6.cmml">:</mo><mrow id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.cmml"><msup id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.cmml"><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.3.cmml">m</mi></msup><mo stretchy="false" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.1" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.1.cmml">→</mo><msup id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.cmml"><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.3" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.3.cmml">d</mi></msup></mrow></mrow><mo id="S2.Thmtheorem2.p1.3.m3.1.1.1.2" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.3.m3.1b"><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1"><and id="S2.Thmtheorem2.p1.3.m3.1.1.1.1a.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1"></and><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1b.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1"><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.5.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.5">:</ci><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4">subscript</csymbol><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.2">𝑠</ci><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.4.3">𝛿</ci></apply><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2"><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.3">→</ci><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4">superscript</csymbol><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.4.3">𝑚</ci></apply><list id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2"><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1">superscript</csymbol><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.1.1.1.1.3">𝑑</ci></apply><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.2">𝑠</ci><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.2.2.2.3">𝐵</ci></apply></list></apply></apply><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1c.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1"><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.6.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.6">:</ci><share href="#S2.Thmtheorem2.p1.3.m3.1.1.1.1.2.cmml" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1d.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1"></share><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7"><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.1">→</ci><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2">superscript</csymbol><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.2.3">𝑚</ci></apply><apply id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.1.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3">superscript</csymbol><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.2.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.3.cmml" xref="S2.Thmtheorem2.p1.3.m3.1.1.1.1.7.3.3">𝑑</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.3.m3.1c">s_{\delta}:\mathbb{R}^{m}\to\mathbb{R}^{d},s_{B}:\mathbb{R}^{m}\to\mathbb{R}^{d},</annotation></semantics></math> and <math id="S2.Thmtheorem2.p1.4.m4.1" class="ltx_Math" alttext="s_{C}:\mathbb{R}^{d}\to\mathbb{R}^{m}" display="inline"><semantics id="S2.Thmtheorem2.p1.4.m4.1a"><mrow id="S2.Thmtheorem2.p1.4.m4.1.1" xref="S2.Thmtheorem2.p1.4.m4.1.1.cmml"><msub id="S2.Thmtheorem2.p1.4.m4.1.1.2" xref="S2.Thmtheorem2.p1.4.m4.1.1.2.cmml"><mi id="S2.Thmtheorem2.p1.4.m4.1.1.2.2" xref="S2.Thmtheorem2.p1.4.m4.1.1.2.2.cmml">s</mi><mi id="S2.Thmtheorem2.p1.4.m4.1.1.2.3" xref="S2.Thmtheorem2.p1.4.m4.1.1.2.3.cmml">C</mi></msub><mo lspace="0.278em" rspace="0.278em" id="S2.Thmtheorem2.p1.4.m4.1.1.1" xref="S2.Thmtheorem2.p1.4.m4.1.1.1.cmml">:</mo><mrow id="S2.Thmtheorem2.p1.4.m4.1.1.3" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.cmml"><msup id="S2.Thmtheorem2.p1.4.m4.1.1.3.2" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2.cmml"><mi id="S2.Thmtheorem2.p1.4.m4.1.1.3.2.2" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.4.m4.1.1.3.2.3" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2.3.cmml">d</mi></msup><mo stretchy="false" id="S2.Thmtheorem2.p1.4.m4.1.1.3.1" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.1.cmml">→</mo><msup id="S2.Thmtheorem2.p1.4.m4.1.1.3.3" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3.cmml"><mi id="S2.Thmtheorem2.p1.4.m4.1.1.3.3.2" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3.2.cmml">ℝ</mi><mi id="S2.Thmtheorem2.p1.4.m4.1.1.3.3.3" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3.3.cmml">m</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.4.m4.1b"><apply id="S2.Thmtheorem2.p1.4.m4.1.1.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1"><ci id="S2.Thmtheorem2.p1.4.m4.1.1.1.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.1">:</ci><apply id="S2.Thmtheorem2.p1.4.m4.1.1.2.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.2"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.4.m4.1.1.2.1.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.2">subscript</csymbol><ci id="S2.Thmtheorem2.p1.4.m4.1.1.2.2.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.2.2">𝑠</ci><ci id="S2.Thmtheorem2.p1.4.m4.1.1.2.3.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.2.3">𝐶</ci></apply><apply id="S2.Thmtheorem2.p1.4.m4.1.1.3.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3"><ci id="S2.Thmtheorem2.p1.4.m4.1.1.3.1.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.1">→</ci><apply id="S2.Thmtheorem2.p1.4.m4.1.1.3.2.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.4.m4.1.1.3.2.1.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2">superscript</csymbol><ci id="S2.Thmtheorem2.p1.4.m4.1.1.3.2.2.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.4.m4.1.1.3.2.3.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.2.3">𝑑</ci></apply><apply id="S2.Thmtheorem2.p1.4.m4.1.1.3.3.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.4.m4.1.1.3.3.1.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3">superscript</csymbol><ci id="S2.Thmtheorem2.p1.4.m4.1.1.3.3.2.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3.2">ℝ</ci><ci id="S2.Thmtheorem2.p1.4.m4.1.1.3.3.3.cmml" xref="S2.Thmtheorem2.p1.4.m4.1.1.3.3.3">𝑚</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.4.m4.1c">s_{C}:\mathbb{R}^{d}\to\mathbb{R}^{m}</annotation></semantics></math>.
Let <math id="S2.Thmtheorem2.p1.5.m5.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="S2.Thmtheorem2.p1.5.m5.1a"><mi id="S2.Thmtheorem2.p1.5.m5.1.1" xref="S2.Thmtheorem2.p1.5.m5.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.5.m5.1b"><ci id="S2.Thmtheorem2.p1.5.m5.1.1.cmml" xref="S2.Thmtheorem2.p1.5.m5.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.5.m5.1c">\tau</annotation></semantics></math> be softplus.
The S6 layer is:</p>
<table id="A2.EGx1" class="ltx_equationgroup ltx_eqn_align ltx_eqn_table">

<tbody id="S2.E3"><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S2.E3.m1.1" class="ltx_Math" alttext="\displaystyle h_{i}" display="inline"><semantics id="S2.E3.m1.1a"><msub id="S2.E3.m1.1.1" xref="S2.E3.m1.1.1.cmml"><mi id="S2.E3.m1.1.1.2" xref="S2.E3.m1.1.1.2.cmml">h</mi><mi id="S2.E3.m1.1.1.3" xref="S2.E3.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.E3.m1.1b"><apply id="S2.E3.m1.1.1.cmml" xref="S2.E3.m1.1.1"><csymbol cd="ambiguous" id="S2.E3.m1.1.1.1.cmml" xref="S2.E3.m1.1.1">subscript</csymbol><ci id="S2.E3.m1.1.1.2.cmml" xref="S2.E3.m1.1.1.2">ℎ</ci><ci id="S2.E3.m1.1.1.3.cmml" xref="S2.E3.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E3.m1.1c">\displaystyle h_{i}</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="S2.E3.m2.2" class="ltx_Math" alttext="\displaystyle=\exp(\delta_{i}A)h_{i-1}+B_{i}x_{i}," display="inline"><semantics id="S2.E3.m2.2a"><mrow id="S2.E3.m2.2.2.1" xref="S2.E3.m2.2.2.1.1.cmml"><mrow id="S2.E3.m2.2.2.1.1" xref="S2.E3.m2.2.2.1.1.cmml"><mi id="S2.E3.m2.2.2.1.1.3" xref="S2.E3.m2.2.2.1.1.3.cmml"></mi><mo id="S2.E3.m2.2.2.1.1.2" xref="S2.E3.m2.2.2.1.1.2.cmml">=</mo><mrow id="S2.E3.m2.2.2.1.1.1" xref="S2.E3.m2.2.2.1.1.1.cmml"><mrow id="S2.E3.m2.2.2.1.1.1.1" xref="S2.E3.m2.2.2.1.1.1.1.cmml"><mrow id="S2.E3.m2.2.2.1.1.1.1.1.1" xref="S2.E3.m2.2.2.1.1.1.1.1.2.cmml"><mi id="S2.E3.m2.1.1" xref="S2.E3.m2.1.1.cmml">exp</mi><mo id="S2.E3.m2.2.2.1.1.1.1.1.1a" xref="S2.E3.m2.2.2.1.1.1.1.1.2.cmml">⁡</mo><mrow id="S2.E3.m2.2.2.1.1.1.1.1.1.1" xref="S2.E3.m2.2.2.1.1.1.1.1.2.cmml"><mo stretchy="false" id="S2.E3.m2.2.2.1.1.1.1.1.1.1.2" xref="S2.E3.m2.2.2.1.1.1.1.1.2.cmml">(</mo><mrow id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.cmml"><msub id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.cmml"><mi id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.2" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.2.cmml">δ</mi><mi id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.3" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.1" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.1.cmml">​</mo><mi id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.3" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.3.cmml">A</mi></mrow><mo stretchy="false" id="S2.E3.m2.2.2.1.1.1.1.1.1.1.3" xref="S2.E3.m2.2.2.1.1.1.1.1.2.cmml">)</mo></mrow></mrow><mo lspace="0em" rspace="0em" id="S2.E3.m2.2.2.1.1.1.1.2" xref="S2.E3.m2.2.2.1.1.1.1.2.cmml">​</mo><msub id="S2.E3.m2.2.2.1.1.1.1.3" xref="S2.E3.m2.2.2.1.1.1.1.3.cmml"><mi id="S2.E3.m2.2.2.1.1.1.1.3.2" xref="S2.E3.m2.2.2.1.1.1.1.3.2.cmml">h</mi><mrow id="S2.E3.m2.2.2.1.1.1.1.3.3" xref="S2.E3.m2.2.2.1.1.1.1.3.3.cmml"><mi id="S2.E3.m2.2.2.1.1.1.1.3.3.2" xref="S2.E3.m2.2.2.1.1.1.1.3.3.2.cmml">i</mi><mo id="S2.E3.m2.2.2.1.1.1.1.3.3.1" xref="S2.E3.m2.2.2.1.1.1.1.3.3.1.cmml">−</mo><mn id="S2.E3.m2.2.2.1.1.1.1.3.3.3" xref="S2.E3.m2.2.2.1.1.1.1.3.3.3.cmml">1</mn></mrow></msub></mrow><mo id="S2.E3.m2.2.2.1.1.1.2" xref="S2.E3.m2.2.2.1.1.1.2.cmml">+</mo><mrow id="S2.E3.m2.2.2.1.1.1.3" xref="S2.E3.m2.2.2.1.1.1.3.cmml"><msub id="S2.E3.m2.2.2.1.1.1.3.2" xref="S2.E3.m2.2.2.1.1.1.3.2.cmml"><mi id="S2.E3.m2.2.2.1.1.1.3.2.2" xref="S2.E3.m2.2.2.1.1.1.3.2.2.cmml">B</mi><mi id="S2.E3.m2.2.2.1.1.1.3.2.3" xref="S2.E3.m2.2.2.1.1.1.3.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.E3.m2.2.2.1.1.1.3.1" xref="S2.E3.m2.2.2.1.1.1.3.1.cmml">​</mo><msub id="S2.E3.m2.2.2.1.1.1.3.3" xref="S2.E3.m2.2.2.1.1.1.3.3.cmml"><mi id="S2.E3.m2.2.2.1.1.1.3.3.2" xref="S2.E3.m2.2.2.1.1.1.3.3.2.cmml">x</mi><mi id="S2.E3.m2.2.2.1.1.1.3.3.3" xref="S2.E3.m2.2.2.1.1.1.3.3.3.cmml">i</mi></msub></mrow></mrow></mrow><mo id="S2.E3.m2.2.2.1.2" xref="S2.E3.m2.2.2.1.1.cmml">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.E3.m2.2b"><apply id="S2.E3.m2.2.2.1.1.cmml" xref="S2.E3.m2.2.2.1"><eq id="S2.E3.m2.2.2.1.1.2.cmml" xref="S2.E3.m2.2.2.1.1.2"></eq><csymbol cd="latexml" id="S2.E3.m2.2.2.1.1.3.cmml" xref="S2.E3.m2.2.2.1.1.3">absent</csymbol><apply id="S2.E3.m2.2.2.1.1.1.cmml" xref="S2.E3.m2.2.2.1.1.1"><plus id="S2.E3.m2.2.2.1.1.1.2.cmml" xref="S2.E3.m2.2.2.1.1.1.2"></plus><apply id="S2.E3.m2.2.2.1.1.1.1.cmml" xref="S2.E3.m2.2.2.1.1.1.1"><times id="S2.E3.m2.2.2.1.1.1.1.2.cmml" xref="S2.E3.m2.2.2.1.1.1.1.2"></times><apply id="S2.E3.m2.2.2.1.1.1.1.1.2.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1"><exp id="S2.E3.m2.1.1.cmml" xref="S2.E3.m2.1.1"></exp><apply id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1"><times id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.1"></times><apply id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.1.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2">subscript</csymbol><ci id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.2.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.2">𝛿</ci><ci id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.3.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.2.3">𝑖</ci></apply><ci id="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.3.cmml" xref="S2.E3.m2.2.2.1.1.1.1.1.1.1.1.3">𝐴</ci></apply></apply><apply id="S2.E3.m2.2.2.1.1.1.1.3.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3"><csymbol cd="ambiguous" id="S2.E3.m2.2.2.1.1.1.1.3.1.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3">subscript</csymbol><ci id="S2.E3.m2.2.2.1.1.1.1.3.2.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3.2">ℎ</ci><apply id="S2.E3.m2.2.2.1.1.1.1.3.3.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3.3"><minus id="S2.E3.m2.2.2.1.1.1.1.3.3.1.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3.3.1"></minus><ci id="S2.E3.m2.2.2.1.1.1.1.3.3.2.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3.3.2">𝑖</ci><cn type="integer" id="S2.E3.m2.2.2.1.1.1.1.3.3.3.cmml" xref="S2.E3.m2.2.2.1.1.1.1.3.3.3">1</cn></apply></apply></apply><apply id="S2.E3.m2.2.2.1.1.1.3.cmml" xref="S2.E3.m2.2.2.1.1.1.3"><times id="S2.E3.m2.2.2.1.1.1.3.1.cmml" xref="S2.E3.m2.2.2.1.1.1.3.1"></times><apply id="S2.E3.m2.2.2.1.1.1.3.2.cmml" xref="S2.E3.m2.2.2.1.1.1.3.2"><csymbol cd="ambiguous" id="S2.E3.m2.2.2.1.1.1.3.2.1.cmml" xref="S2.E3.m2.2.2.1.1.1.3.2">subscript</csymbol><ci id="S2.E3.m2.2.2.1.1.1.3.2.2.cmml" xref="S2.E3.m2.2.2.1.1.1.3.2.2">𝐵</ci><ci id="S2.E3.m2.2.2.1.1.1.3.2.3.cmml" xref="S2.E3.m2.2.2.1.1.1.3.2.3">𝑖</ci></apply><apply id="S2.E3.m2.2.2.1.1.1.3.3.cmml" xref="S2.E3.m2.2.2.1.1.1.3.3"><csymbol cd="ambiguous" id="S2.E3.m2.2.2.1.1.1.3.3.1.cmml" xref="S2.E3.m2.2.2.1.1.1.3.3">subscript</csymbol><ci id="S2.E3.m2.2.2.1.1.1.3.3.2.cmml" xref="S2.E3.m2.2.2.1.1.1.3.3.2">𝑥</ci><ci id="S2.E3.m2.2.2.1.1.1.3.3.3.cmml" xref="S2.E3.m2.2.2.1.1.1.3.3.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E3.m2.2c">\displaystyle=\exp(\delta_{i}A)h_{i-1}+B_{i}x_{i},</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(3)</span></td>
</tr></tbody>
<tbody id="S2.Ex1"><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S2.Ex1.m1.1" class="ltx_Math" alttext="\displaystyle\textrm{where}\;\delta_{i}" display="inline"><semantics id="S2.Ex1.m1.1a"><mrow id="S2.Ex1.m1.1.1" xref="S2.Ex1.m1.1.1.cmml"><mtext id="S2.Ex1.m1.1.1.2" xref="S2.Ex1.m1.1.1.2a.cmml">where</mtext><mo lspace="0.280em" rspace="0em" id="S2.Ex1.m1.1.1.1" xref="S2.Ex1.m1.1.1.1.cmml">​</mo><msub id="S2.Ex1.m1.1.1.3" xref="S2.Ex1.m1.1.1.3.cmml"><mi id="S2.Ex1.m1.1.1.3.2" xref="S2.Ex1.m1.1.1.3.2.cmml">δ</mi><mi id="S2.Ex1.m1.1.1.3.3" xref="S2.Ex1.m1.1.1.3.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex1.m1.1b"><apply id="S2.Ex1.m1.1.1.cmml" xref="S2.Ex1.m1.1.1"><times id="S2.Ex1.m1.1.1.1.cmml" xref="S2.Ex1.m1.1.1.1"></times><ci id="S2.Ex1.m1.1.1.2a.cmml" xref="S2.Ex1.m1.1.1.2"><mtext id="S2.Ex1.m1.1.1.2.cmml" xref="S2.Ex1.m1.1.1.2">where</mtext></ci><apply id="S2.Ex1.m1.1.1.3.cmml" xref="S2.Ex1.m1.1.1.3"><csymbol cd="ambiguous" id="S2.Ex1.m1.1.1.3.1.cmml" xref="S2.Ex1.m1.1.1.3">subscript</csymbol><ci id="S2.Ex1.m1.1.1.3.2.cmml" xref="S2.Ex1.m1.1.1.3.2">𝛿</ci><ci id="S2.Ex1.m1.1.1.3.3.cmml" xref="S2.Ex1.m1.1.1.3.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex1.m1.1c">\displaystyle\textrm{where}\;\delta_{i}</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="S2.Ex1.m2.2" class="ltx_Math" alttext="\displaystyle=\tau(\delta+s_{\delta}(x))\;\textrm{and}" display="inline"><semantics id="S2.Ex1.m2.2a"><mrow id="S2.Ex1.m2.2.2" xref="S2.Ex1.m2.2.2.cmml"><mi id="S2.Ex1.m2.2.2.3" xref="S2.Ex1.m2.2.2.3.cmml"></mi><mo id="S2.Ex1.m2.2.2.2" xref="S2.Ex1.m2.2.2.2.cmml">=</mo><mrow id="S2.Ex1.m2.2.2.1" xref="S2.Ex1.m2.2.2.1.cmml"><mi id="S2.Ex1.m2.2.2.1.3" xref="S2.Ex1.m2.2.2.1.3.cmml">τ</mi><mo lspace="0em" rspace="0em" id="S2.Ex1.m2.2.2.1.2" xref="S2.Ex1.m2.2.2.1.2.cmml">​</mo><mrow id="S2.Ex1.m2.2.2.1.1.1" xref="S2.Ex1.m2.2.2.1.1.1.1.cmml"><mo stretchy="false" id="S2.Ex1.m2.2.2.1.1.1.2" xref="S2.Ex1.m2.2.2.1.1.1.1.cmml">(</mo><mrow id="S2.Ex1.m2.2.2.1.1.1.1" xref="S2.Ex1.m2.2.2.1.1.1.1.cmml"><mi id="S2.Ex1.m2.2.2.1.1.1.1.2" xref="S2.Ex1.m2.2.2.1.1.1.1.2.cmml">δ</mi><mo id="S2.Ex1.m2.2.2.1.1.1.1.1" xref="S2.Ex1.m2.2.2.1.1.1.1.1.cmml">+</mo><mrow id="S2.Ex1.m2.2.2.1.1.1.1.3" xref="S2.Ex1.m2.2.2.1.1.1.1.3.cmml"><msub id="S2.Ex1.m2.2.2.1.1.1.1.3.2" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2.cmml"><mi id="S2.Ex1.m2.2.2.1.1.1.1.3.2.2" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2.2.cmml">s</mi><mi id="S2.Ex1.m2.2.2.1.1.1.1.3.2.3" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2.3.cmml">δ</mi></msub><mo lspace="0em" rspace="0em" id="S2.Ex1.m2.2.2.1.1.1.1.3.1" xref="S2.Ex1.m2.2.2.1.1.1.1.3.1.cmml">​</mo><mrow id="S2.Ex1.m2.2.2.1.1.1.1.3.3.2" xref="S2.Ex1.m2.2.2.1.1.1.1.3.cmml"><mo stretchy="false" id="S2.Ex1.m2.2.2.1.1.1.1.3.3.2.1" xref="S2.Ex1.m2.2.2.1.1.1.1.3.cmml">(</mo><mi id="S2.Ex1.m2.1.1" xref="S2.Ex1.m2.1.1.cmml">x</mi><mo stretchy="false" id="S2.Ex1.m2.2.2.1.1.1.1.3.3.2.2" xref="S2.Ex1.m2.2.2.1.1.1.1.3.cmml">)</mo></mrow></mrow></mrow><mo stretchy="false" id="S2.Ex1.m2.2.2.1.1.1.3" xref="S2.Ex1.m2.2.2.1.1.1.1.cmml">)</mo></mrow><mo lspace="0.280em" rspace="0em" id="S2.Ex1.m2.2.2.1.2a" xref="S2.Ex1.m2.2.2.1.2.cmml">​</mo><mtext id="S2.Ex1.m2.2.2.1.4" xref="S2.Ex1.m2.2.2.1.4a.cmml">and</mtext></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex1.m2.2b"><apply id="S2.Ex1.m2.2.2.cmml" xref="S2.Ex1.m2.2.2"><eq id="S2.Ex1.m2.2.2.2.cmml" xref="S2.Ex1.m2.2.2.2"></eq><csymbol cd="latexml" id="S2.Ex1.m2.2.2.3.cmml" xref="S2.Ex1.m2.2.2.3">absent</csymbol><apply id="S2.Ex1.m2.2.2.1.cmml" xref="S2.Ex1.m2.2.2.1"><times id="S2.Ex1.m2.2.2.1.2.cmml" xref="S2.Ex1.m2.2.2.1.2"></times><ci id="S2.Ex1.m2.2.2.1.3.cmml" xref="S2.Ex1.m2.2.2.1.3">𝜏</ci><apply id="S2.Ex1.m2.2.2.1.1.1.1.cmml" xref="S2.Ex1.m2.2.2.1.1.1"><plus id="S2.Ex1.m2.2.2.1.1.1.1.1.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.1"></plus><ci id="S2.Ex1.m2.2.2.1.1.1.1.2.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.2">𝛿</ci><apply id="S2.Ex1.m2.2.2.1.1.1.1.3.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.3"><times id="S2.Ex1.m2.2.2.1.1.1.1.3.1.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.3.1"></times><apply id="S2.Ex1.m2.2.2.1.1.1.1.3.2.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2"><csymbol cd="ambiguous" id="S2.Ex1.m2.2.2.1.1.1.1.3.2.1.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2">subscript</csymbol><ci id="S2.Ex1.m2.2.2.1.1.1.1.3.2.2.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2.2">𝑠</ci><ci id="S2.Ex1.m2.2.2.1.1.1.1.3.2.3.cmml" xref="S2.Ex1.m2.2.2.1.1.1.1.3.2.3">𝛿</ci></apply><ci id="S2.Ex1.m2.1.1.cmml" xref="S2.Ex1.m2.1.1">𝑥</ci></apply></apply><ci id="S2.Ex1.m2.2.2.1.4a.cmml" xref="S2.Ex1.m2.2.2.1.4"><mtext id="S2.Ex1.m2.2.2.1.4.cmml" xref="S2.Ex1.m2.2.2.1.4">and</mtext></ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex1.m2.2c">\displaystyle=\tau(\delta+s_{\delta}(x))\;\textrm{and}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
<tbody id="S2.Ex2"><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_td ltx_align_right ltx_eqn_cell"><math id="S2.Ex2.m1.1" class="ltx_Math" alttext="\displaystyle\;B_{i}" display="inline"><semantics id="S2.Ex2.m1.1a"><msub id="S2.Ex2.m1.1.1" xref="S2.Ex2.m1.1.1.cmml"><mi id="S2.Ex2.m1.1.1.2" xref="S2.Ex2.m1.1.1.2.cmml">B</mi><mi id="S2.Ex2.m1.1.1.3" xref="S2.Ex2.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.Ex2.m1.1b"><apply id="S2.Ex2.m1.1.1.cmml" xref="S2.Ex2.m1.1.1"><csymbol cd="ambiguous" id="S2.Ex2.m1.1.1.1.cmml" xref="S2.Ex2.m1.1.1">subscript</csymbol><ci id="S2.Ex2.m1.1.1.2.cmml" xref="S2.Ex2.m1.1.1.2">𝐵</ci><ci id="S2.Ex2.m1.1.1.3.cmml" xref="S2.Ex2.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex2.m1.1c">\displaystyle\;B_{i}</annotation></semantics></math></td>
<td class="ltx_td ltx_align_left ltx_eqn_cell"><math id="S2.Ex2.m2.2" class="ltx_Math" alttext="\displaystyle=s_{B}(x)." display="inline"><semantics id="S2.Ex2.m2.2a"><mrow id="S2.Ex2.m2.2.2.1" xref="S2.Ex2.m2.2.2.1.1.cmml"><mrow id="S2.Ex2.m2.2.2.1.1" xref="S2.Ex2.m2.2.2.1.1.cmml"><mi id="S2.Ex2.m2.2.2.1.1.2" xref="S2.Ex2.m2.2.2.1.1.2.cmml"></mi><mo id="S2.Ex2.m2.2.2.1.1.1" xref="S2.Ex2.m2.2.2.1.1.1.cmml">=</mo><mrow id="S2.Ex2.m2.2.2.1.1.3" xref="S2.Ex2.m2.2.2.1.1.3.cmml"><msub id="S2.Ex2.m2.2.2.1.1.3.2" xref="S2.Ex2.m2.2.2.1.1.3.2.cmml"><mi id="S2.Ex2.m2.2.2.1.1.3.2.2" xref="S2.Ex2.m2.2.2.1.1.3.2.2.cmml">s</mi><mi id="S2.Ex2.m2.2.2.1.1.3.2.3" xref="S2.Ex2.m2.2.2.1.1.3.2.3.cmml">B</mi></msub><mo lspace="0em" rspace="0em" id="S2.Ex2.m2.2.2.1.1.3.1" xref="S2.Ex2.m2.2.2.1.1.3.1.cmml">​</mo><mrow id="S2.Ex2.m2.2.2.1.1.3.3.2" xref="S2.Ex2.m2.2.2.1.1.3.cmml"><mo stretchy="false" id="S2.Ex2.m2.2.2.1.1.3.3.2.1" xref="S2.Ex2.m2.2.2.1.1.3.cmml">(</mo><mi id="S2.Ex2.m2.1.1" xref="S2.Ex2.m2.1.1.cmml">x</mi><mo stretchy="false" id="S2.Ex2.m2.2.2.1.1.3.3.2.2" xref="S2.Ex2.m2.2.2.1.1.3.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="S2.Ex2.m2.2.2.1.2" xref="S2.Ex2.m2.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex2.m2.2b"><apply id="S2.Ex2.m2.2.2.1.1.cmml" xref="S2.Ex2.m2.2.2.1"><eq id="S2.Ex2.m2.2.2.1.1.1.cmml" xref="S2.Ex2.m2.2.2.1.1.1"></eq><csymbol cd="latexml" id="S2.Ex2.m2.2.2.1.1.2.cmml" xref="S2.Ex2.m2.2.2.1.1.2">absent</csymbol><apply id="S2.Ex2.m2.2.2.1.1.3.cmml" xref="S2.Ex2.m2.2.2.1.1.3"><times id="S2.Ex2.m2.2.2.1.1.3.1.cmml" xref="S2.Ex2.m2.2.2.1.1.3.1"></times><apply id="S2.Ex2.m2.2.2.1.1.3.2.cmml" xref="S2.Ex2.m2.2.2.1.1.3.2"><csymbol cd="ambiguous" id="S2.Ex2.m2.2.2.1.1.3.2.1.cmml" xref="S2.Ex2.m2.2.2.1.1.3.2">subscript</csymbol><ci id="S2.Ex2.m2.2.2.1.1.3.2.2.cmml" xref="S2.Ex2.m2.2.2.1.1.3.2.2">𝑠</ci><ci id="S2.Ex2.m2.2.2.1.1.3.2.3.cmml" xref="S2.Ex2.m2.2.2.1.1.3.2.3">𝐵</ci></apply><ci id="S2.Ex2.m2.1.1.cmml" xref="S2.Ex2.m2.1.1">𝑥</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex2.m2.2c">\displaystyle=s_{B}(x).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S2.Thmtheorem2.p1.11" class="ltx_p">This implies the convolutional form:</p>
<table id="S2.E4" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E4.m1.2" class="ltx_Math" alttext="h_{i}=\sum_{j=1}^{i}\left(\prod_{k=j+1}^{i}\exp(\delta_{k}A)\right)B_{j}x_{j}." display="block"><semantics id="S2.E4.m1.2a"><mrow id="S2.E4.m1.2.2.1" xref="S2.E4.m1.2.2.1.1.cmml"><mrow id="S2.E4.m1.2.2.1.1" xref="S2.E4.m1.2.2.1.1.cmml"><msub id="S2.E4.m1.2.2.1.1.3" xref="S2.E4.m1.2.2.1.1.3.cmml"><mi id="S2.E4.m1.2.2.1.1.3.2" xref="S2.E4.m1.2.2.1.1.3.2.cmml">h</mi><mi id="S2.E4.m1.2.2.1.1.3.3" xref="S2.E4.m1.2.2.1.1.3.3.cmml">i</mi></msub><mo rspace="0.111em" id="S2.E4.m1.2.2.1.1.2" xref="S2.E4.m1.2.2.1.1.2.cmml">=</mo><mrow id="S2.E4.m1.2.2.1.1.1" xref="S2.E4.m1.2.2.1.1.1.cmml"><munderover id="S2.E4.m1.2.2.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.2.cmml"><mo movablelimits="false" rspace="0em" id="S2.E4.m1.2.2.1.1.1.2.2.2" xref="S2.E4.m1.2.2.1.1.1.2.2.2.cmml">∑</mo><mrow id="S2.E4.m1.2.2.1.1.1.2.2.3" xref="S2.E4.m1.2.2.1.1.1.2.2.3.cmml"><mi id="S2.E4.m1.2.2.1.1.1.2.2.3.2" xref="S2.E4.m1.2.2.1.1.1.2.2.3.2.cmml">j</mi><mo id="S2.E4.m1.2.2.1.1.1.2.2.3.1" xref="S2.E4.m1.2.2.1.1.1.2.2.3.1.cmml">=</mo><mn id="S2.E4.m1.2.2.1.1.1.2.2.3.3" xref="S2.E4.m1.2.2.1.1.1.2.2.3.3.cmml">1</mn></mrow><mi id="S2.E4.m1.2.2.1.1.1.2.3" xref="S2.E4.m1.2.2.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S2.E4.m1.2.2.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.cmml"><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml"><mo id="S2.E4.m1.2.2.1.1.1.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml">(</mo><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml"><munderover id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.cmml"><mo lspace="0em" movablelimits="false" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.2.cmml">∏</mo><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.2.cmml">k</mi><mo id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.1.cmml">=</mo><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.2.cmml">j</mi><mo id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.1.cmml">+</mo><mn id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.3.cmml">1</mn></mrow></mrow><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.2.cmml"><mi id="S2.E4.m1.1.1" xref="S2.E4.m1.1.1.cmml">exp</mi><mo id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1a" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.2.cmml">⁡</mo><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.2.cmml"><mo stretchy="false" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.2.cmml">(</mo><mrow id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.cmml"><msub id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.2.cmml">δ</mi><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.3.cmml">k</mi></msub><mo lspace="0em" rspace="0em" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.1.cmml">​</mo><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.3.cmml">A</mi></mrow><mo stretchy="false" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.2.cmml">)</mo></mrow></mrow></mrow><mo id="S2.E4.m1.2.2.1.1.1.1.1.1.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml">)</mo></mrow><mo lspace="0em" rspace="0em" id="S2.E4.m1.2.2.1.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.2.cmml">​</mo><msub id="S2.E4.m1.2.2.1.1.1.1.3" xref="S2.E4.m1.2.2.1.1.1.1.3.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.3.2" xref="S2.E4.m1.2.2.1.1.1.1.3.2.cmml">B</mi><mi id="S2.E4.m1.2.2.1.1.1.1.3.3" xref="S2.E4.m1.2.2.1.1.1.1.3.3.cmml">j</mi></msub><mo lspace="0em" rspace="0em" id="S2.E4.m1.2.2.1.1.1.1.2a" xref="S2.E4.m1.2.2.1.1.1.1.2.cmml">​</mo><msub id="S2.E4.m1.2.2.1.1.1.1.4" xref="S2.E4.m1.2.2.1.1.1.1.4.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.4.2" xref="S2.E4.m1.2.2.1.1.1.1.4.2.cmml">x</mi><mi id="S2.E4.m1.2.2.1.1.1.1.4.3" xref="S2.E4.m1.2.2.1.1.1.1.4.3.cmml">j</mi></msub></mrow></mrow></mrow><mo lspace="0em" id="S2.E4.m1.2.2.1.2" xref="S2.E4.m1.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.E4.m1.2b"><apply id="S2.E4.m1.2.2.1.1.cmml" xref="S2.E4.m1.2.2.1"><eq id="S2.E4.m1.2.2.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.2"></eq><apply id="S2.E4.m1.2.2.1.1.3.cmml" xref="S2.E4.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.3.1.cmml" xref="S2.E4.m1.2.2.1.1.3">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.3.2.cmml" xref="S2.E4.m1.2.2.1.1.3.2">ℎ</ci><ci id="S2.E4.m1.2.2.1.1.3.3.cmml" xref="S2.E4.m1.2.2.1.1.3.3">𝑖</ci></apply><apply id="S2.E4.m1.2.2.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1"><apply id="S2.E4.m1.2.2.1.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.1.2"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.2.1.cmml" xref="S2.E4.m1.2.2.1.1.1.2">superscript</csymbol><apply id="S2.E4.m1.2.2.1.1.1.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.2"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.2.2.1.cmml" xref="S2.E4.m1.2.2.1.1.1.2">subscript</csymbol><sum id="S2.E4.m1.2.2.1.1.1.2.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.2.2.2"></sum><apply id="S2.E4.m1.2.2.1.1.1.2.2.3.cmml" xref="S2.E4.m1.2.2.1.1.1.2.2.3"><eq id="S2.E4.m1.2.2.1.1.1.2.2.3.1.cmml" xref="S2.E4.m1.2.2.1.1.1.2.2.3.1"></eq><ci id="S2.E4.m1.2.2.1.1.1.2.2.3.2.cmml" xref="S2.E4.m1.2.2.1.1.1.2.2.3.2">𝑗</ci><cn type="integer" id="S2.E4.m1.2.2.1.1.1.2.2.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.2.2.3.3">1</cn></apply></apply><ci id="S2.E4.m1.2.2.1.1.1.2.3.cmml" xref="S2.E4.m1.2.2.1.1.1.2.3">𝑖</ci></apply><apply id="S2.E4.m1.2.2.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1"><times id="S2.E4.m1.2.2.1.1.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.2"></times><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1"><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2">superscript</csymbol><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.2">product</csymbol><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3"><eq id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.1"></eq><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.2">𝑘</ci><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3"><plus id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.1"></plus><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.2">𝑗</ci><cn type="integer" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.2.3.3.3">1</cn></apply></apply></apply><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1"><exp id="S2.E4.m1.1.1.cmml" xref="S2.E4.m1.1.1"></exp><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1"><times id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.1"></times><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.2">𝛿</ci><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.2.3">𝑘</ci></apply><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.3">𝐴</ci></apply></apply></apply><apply id="S2.E4.m1.2.2.1.1.1.1.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.3.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.3.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.2">𝐵</ci><ci id="S2.E4.m1.2.2.1.1.1.1.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3">𝑗</ci></apply><apply id="S2.E4.m1.2.2.1.1.1.1.4.cmml" xref="S2.E4.m1.2.2.1.1.1.1.4"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.4.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.4">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.4.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.4.2">𝑥</ci><ci id="S2.E4.m1.2.2.1.1.1.1.4.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.4.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E4.m1.2c">h_{i}=\sum_{j=1}^{i}\left(\prod_{k=j+1}^{i}\exp(\delta_{k}A)\right)B_{j}x_{j}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(4)</span></td>
</tr></tbody>
</table>
<p id="S2.Thmtheorem2.p1.10" class="ltx_p">Finally, to compute <math id="S2.Thmtheorem2.p1.6.m1.1" class="ltx_Math" alttext="y_{i}" display="inline"><semantics id="S2.Thmtheorem2.p1.6.m1.1a"><msub id="S2.Thmtheorem2.p1.6.m1.1.1" xref="S2.Thmtheorem2.p1.6.m1.1.1.cmml"><mi id="S2.Thmtheorem2.p1.6.m1.1.1.2" xref="S2.Thmtheorem2.p1.6.m1.1.1.2.cmml">y</mi><mi id="S2.Thmtheorem2.p1.6.m1.1.1.3" xref="S2.Thmtheorem2.p1.6.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.6.m1.1b"><apply id="S2.Thmtheorem2.p1.6.m1.1.1.cmml" xref="S2.Thmtheorem2.p1.6.m1.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.6.m1.1.1.1.cmml" xref="S2.Thmtheorem2.p1.6.m1.1.1">subscript</csymbol><ci id="S2.Thmtheorem2.p1.6.m1.1.1.2.cmml" xref="S2.Thmtheorem2.p1.6.m1.1.1.2">𝑦</ci><ci id="S2.Thmtheorem2.p1.6.m1.1.1.3.cmml" xref="S2.Thmtheorem2.p1.6.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.6.m1.1c">y_{i}</annotation></semantics></math>, <math id="S2.Thmtheorem2.p1.7.m2.1" class="ltx_Math" alttext="C" display="inline"><semantics id="S2.Thmtheorem2.p1.7.m2.1a"><mi id="S2.Thmtheorem2.p1.7.m2.1.1" xref="S2.Thmtheorem2.p1.7.m2.1.1.cmml">C</mi><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.7.m2.1b"><ci id="S2.Thmtheorem2.p1.7.m2.1.1.cmml" xref="S2.Thmtheorem2.p1.7.m2.1.1">𝐶</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.7.m2.1c">C</annotation></semantics></math> is made input dependent <math id="S2.Thmtheorem2.p1.8.m3.1" class="ltx_Math" alttext="C_{i}" display="inline"><semantics id="S2.Thmtheorem2.p1.8.m3.1a"><msub id="S2.Thmtheorem2.p1.8.m3.1.1" xref="S2.Thmtheorem2.p1.8.m3.1.1.cmml"><mi id="S2.Thmtheorem2.p1.8.m3.1.1.2" xref="S2.Thmtheorem2.p1.8.m3.1.1.2.cmml">C</mi><mi id="S2.Thmtheorem2.p1.8.m3.1.1.3" xref="S2.Thmtheorem2.p1.8.m3.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.8.m3.1b"><apply id="S2.Thmtheorem2.p1.8.m3.1.1.cmml" xref="S2.Thmtheorem2.p1.8.m3.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.8.m3.1.1.1.cmml" xref="S2.Thmtheorem2.p1.8.m3.1.1">subscript</csymbol><ci id="S2.Thmtheorem2.p1.8.m3.1.1.2.cmml" xref="S2.Thmtheorem2.p1.8.m3.1.1.2">𝐶</ci><ci id="S2.Thmtheorem2.p1.8.m3.1.1.3.cmml" xref="S2.Thmtheorem2.p1.8.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.8.m3.1c">C_{i}</annotation></semantics></math> and computed via a projection in the same manner as <math id="S2.Thmtheorem2.p1.9.m4.1" class="ltx_Math" alttext="B_{i}" display="inline"><semantics id="S2.Thmtheorem2.p1.9.m4.1a"><msub id="S2.Thmtheorem2.p1.9.m4.1.1" xref="S2.Thmtheorem2.p1.9.m4.1.1.cmml"><mi id="S2.Thmtheorem2.p1.9.m4.1.1.2" xref="S2.Thmtheorem2.p1.9.m4.1.1.2.cmml">B</mi><mi id="S2.Thmtheorem2.p1.9.m4.1.1.3" xref="S2.Thmtheorem2.p1.9.m4.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.9.m4.1b"><apply id="S2.Thmtheorem2.p1.9.m4.1.1.cmml" xref="S2.Thmtheorem2.p1.9.m4.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.9.m4.1.1.1.cmml" xref="S2.Thmtheorem2.p1.9.m4.1.1">subscript</csymbol><ci id="S2.Thmtheorem2.p1.9.m4.1.1.2.cmml" xref="S2.Thmtheorem2.p1.9.m4.1.1.2">𝐵</ci><ci id="S2.Thmtheorem2.p1.9.m4.1.1.3.cmml" xref="S2.Thmtheorem2.p1.9.m4.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.9.m4.1c">B_{i}</annotation></semantics></math>. The layer output is then <math id="S2.Thmtheorem2.p1.10.m5.1" class="ltx_Math" alttext="y_{i}=C_{i}h_{i}+x_{i}" display="inline"><semantics id="S2.Thmtheorem2.p1.10.m5.1a"><mrow id="S2.Thmtheorem2.p1.10.m5.1.1" xref="S2.Thmtheorem2.p1.10.m5.1.1.cmml"><msub id="S2.Thmtheorem2.p1.10.m5.1.1.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.2.cmml"><mi id="S2.Thmtheorem2.p1.10.m5.1.1.2.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.2.2.cmml">y</mi><mi id="S2.Thmtheorem2.p1.10.m5.1.1.2.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.2.3.cmml">i</mi></msub><mo id="S2.Thmtheorem2.p1.10.m5.1.1.1" xref="S2.Thmtheorem2.p1.10.m5.1.1.1.cmml">=</mo><mrow id="S2.Thmtheorem2.p1.10.m5.1.1.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.cmml"><mrow id="S2.Thmtheorem2.p1.10.m5.1.1.3.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.cmml"><msub id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.cmml"><mi id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.2.cmml">C</mi><mi id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.1" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.1.cmml">​</mo><msub id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.cmml"><mi id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.2.cmml">h</mi><mi id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.3.cmml">i</mi></msub></mrow><mo id="S2.Thmtheorem2.p1.10.m5.1.1.3.1" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.1.cmml">+</mo><msub id="S2.Thmtheorem2.p1.10.m5.1.1.3.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3.cmml"><mi id="S2.Thmtheorem2.p1.10.m5.1.1.3.3.2" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3.2.cmml">x</mi><mi id="S2.Thmtheorem2.p1.10.m5.1.1.3.3.3" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3.3.cmml">i</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem2.p1.10.m5.1b"><apply id="S2.Thmtheorem2.p1.10.m5.1.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1"><eq id="S2.Thmtheorem2.p1.10.m5.1.1.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.1"></eq><apply id="S2.Thmtheorem2.p1.10.m5.1.1.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.2"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.10.m5.1.1.2.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.2">subscript</csymbol><ci id="S2.Thmtheorem2.p1.10.m5.1.1.2.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.2.2">𝑦</ci><ci id="S2.Thmtheorem2.p1.10.m5.1.1.2.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.2.3">𝑖</ci></apply><apply id="S2.Thmtheorem2.p1.10.m5.1.1.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3"><plus id="S2.Thmtheorem2.p1.10.m5.1.1.3.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.1"></plus><apply id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2"><times id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.1"></times><apply id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2">subscript</csymbol><ci id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.2">𝐶</ci><ci id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.2.3">𝑖</ci></apply><apply id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3">subscript</csymbol><ci id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.2">ℎ</ci><ci id="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.2.3.3">𝑖</ci></apply></apply><apply id="S2.Thmtheorem2.p1.10.m5.1.1.3.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3"><csymbol cd="ambiguous" id="S2.Thmtheorem2.p1.10.m5.1.1.3.3.1.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3">subscript</csymbol><ci id="S2.Thmtheorem2.p1.10.m5.1.1.3.3.2.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3.2">𝑥</ci><ci id="S2.Thmtheorem2.p1.10.m5.1.1.3.3.3.cmml" xref="S2.Thmtheorem2.p1.10.m5.1.1.3.3.3">𝑖</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem2.p1.10.m5.1c">y_{i}=C_{i}h_{i}+x_{i}</annotation></semantics></math>.</p>
</div>
</div>
<div id="S2.SS1.p5" class="ltx_para">
<p id="S2.SS1.p5.1" class="ltx_p">In practice, a crucial detail for training SSMs is the initialization of the weight matrices.
Our main results (<a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorems</span> <span class="ltx_text ltx_ref_tag">4.1</span></a> and <a href="#S4.Thmtheorem2" title="Theorem 4.2. ‣ 4.2 Simulating S6 with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.2</span></a>) will apply for any linear SSM (including S4) as well as S6, independent of the specific values of its weights.
In contrast to S4 and S6, H3 <cite class="ltx_cite ltx_citemacro_citep">(Fu et al., <a href="#bib.bib7" title="" class="ltx_ref">2023</a>)</cite> is not a true SSM because the context is not represented by a single vector. Rather, its architecture resembles a transformer with SSM components. Analyzing H3 is beyond our focus, but we believe our ideas could be extended to H3 in future work.</p>
</div>
</section>
<section id="S2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.2 </span>Numeric Datatype</h3>

<div id="S2.SS2.p1" class="ltx_para">
<p id="S2.SS2.p1.8" class="ltx_p">Circuit-complexity analysis of neural networks depends to some degree on low-level details about arithmetic and the underlying datatype <math id="S2.SS2.p1.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p1.1.m1.1a"><mi id="S2.SS2.p1.1.m1.1.1" xref="S2.SS2.p1.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.1.m1.1b"><ci id="S2.SS2.p1.1.m1.1.1.cmml" xref="S2.SS2.p1.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.1.m1.1c">\mathbb{D}</annotation></semantics></math> used in the network’s computation graph. We can think of <math id="S2.SS2.p1.2.m2.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p1.2.m2.1a"><mi id="S2.SS2.p1.2.m2.1.1" xref="S2.SS2.p1.2.m2.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.2.m2.1b"><ci id="S2.SS2.p1.2.m2.1.1.cmml" xref="S2.SS2.p1.2.m2.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.2.m2.1c">\mathbb{D}</annotation></semantics></math> as parameterized by the number of bits available to represent a number in <math id="S2.SS2.p1.3.m3.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p1.3.m3.1a"><mi id="S2.SS2.p1.3.m3.1.1" xref="S2.SS2.p1.3.m3.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.3.m3.1b"><ci id="S2.SS2.p1.3.m3.1.1.cmml" xref="S2.SS2.p1.3.m3.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.3.m3.1c">\mathbb{D}</annotation></semantics></math>. For instance, non-negative integers in <math id="S2.SS2.p1.4.m4.2" class="ltx_Math" alttext="[0,2^{p}]" display="inline"><semantics id="S2.SS2.p1.4.m4.2a"><mrow id="S2.SS2.p1.4.m4.2.2.1" xref="S2.SS2.p1.4.m4.2.2.2.cmml"><mo stretchy="false" id="S2.SS2.p1.4.m4.2.2.1.2" xref="S2.SS2.p1.4.m4.2.2.2.cmml">[</mo><mn id="S2.SS2.p1.4.m4.1.1" xref="S2.SS2.p1.4.m4.1.1.cmml">0</mn><mo id="S2.SS2.p1.4.m4.2.2.1.3" xref="S2.SS2.p1.4.m4.2.2.2.cmml">,</mo><msup id="S2.SS2.p1.4.m4.2.2.1.1" xref="S2.SS2.p1.4.m4.2.2.1.1.cmml"><mn id="S2.SS2.p1.4.m4.2.2.1.1.2" xref="S2.SS2.p1.4.m4.2.2.1.1.2.cmml">2</mn><mi id="S2.SS2.p1.4.m4.2.2.1.1.3" xref="S2.SS2.p1.4.m4.2.2.1.1.3.cmml">p</mi></msup><mo stretchy="false" id="S2.SS2.p1.4.m4.2.2.1.4" xref="S2.SS2.p1.4.m4.2.2.2.cmml">]</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.4.m4.2b"><interval closure="closed" id="S2.SS2.p1.4.m4.2.2.2.cmml" xref="S2.SS2.p1.4.m4.2.2.1"><cn type="integer" id="S2.SS2.p1.4.m4.1.1.cmml" xref="S2.SS2.p1.4.m4.1.1">0</cn><apply id="S2.SS2.p1.4.m4.2.2.1.1.cmml" xref="S2.SS2.p1.4.m4.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS2.p1.4.m4.2.2.1.1.1.cmml" xref="S2.SS2.p1.4.m4.2.2.1.1">superscript</csymbol><cn type="integer" id="S2.SS2.p1.4.m4.2.2.1.1.2.cmml" xref="S2.SS2.p1.4.m4.2.2.1.1.2">2</cn><ci id="S2.SS2.p1.4.m4.2.2.1.1.3.cmml" xref="S2.SS2.p1.4.m4.2.2.1.1.3">𝑝</ci></apply></interval></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.4.m4.2c">[0,2^{p}]</annotation></semantics></math> use <math id="S2.SS2.p1.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS2.p1.5.m5.1a"><mi id="S2.SS2.p1.5.m5.1.1" xref="S2.SS2.p1.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.5.m5.1b"><ci id="S2.SS2.p1.5.m5.1.1.cmml" xref="S2.SS2.p1.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.5.m5.1c">p</annotation></semantics></math> bits, signed integers in <math id="S2.SS2.p1.6.m6.2" class="ltx_Math" alttext="[-2^{p},2^{p}]" display="inline"><semantics id="S2.SS2.p1.6.m6.2a"><mrow id="S2.SS2.p1.6.m6.2.2.2" xref="S2.SS2.p1.6.m6.2.2.3.cmml"><mo stretchy="false" id="S2.SS2.p1.6.m6.2.2.2.3" xref="S2.SS2.p1.6.m6.2.2.3.cmml">[</mo><mrow id="S2.SS2.p1.6.m6.1.1.1.1" xref="S2.SS2.p1.6.m6.1.1.1.1.cmml"><mo id="S2.SS2.p1.6.m6.1.1.1.1a" xref="S2.SS2.p1.6.m6.1.1.1.1.cmml">−</mo><msup id="S2.SS2.p1.6.m6.1.1.1.1.2" xref="S2.SS2.p1.6.m6.1.1.1.1.2.cmml"><mn id="S2.SS2.p1.6.m6.1.1.1.1.2.2" xref="S2.SS2.p1.6.m6.1.1.1.1.2.2.cmml">2</mn><mi id="S2.SS2.p1.6.m6.1.1.1.1.2.3" xref="S2.SS2.p1.6.m6.1.1.1.1.2.3.cmml">p</mi></msup></mrow><mo id="S2.SS2.p1.6.m6.2.2.2.4" xref="S2.SS2.p1.6.m6.2.2.3.cmml">,</mo><msup id="S2.SS2.p1.6.m6.2.2.2.2" xref="S2.SS2.p1.6.m6.2.2.2.2.cmml"><mn id="S2.SS2.p1.6.m6.2.2.2.2.2" xref="S2.SS2.p1.6.m6.2.2.2.2.2.cmml">2</mn><mi id="S2.SS2.p1.6.m6.2.2.2.2.3" xref="S2.SS2.p1.6.m6.2.2.2.2.3.cmml">p</mi></msup><mo stretchy="false" id="S2.SS2.p1.6.m6.2.2.2.5" xref="S2.SS2.p1.6.m6.2.2.3.cmml">]</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.6.m6.2b"><interval closure="closed" id="S2.SS2.p1.6.m6.2.2.3.cmml" xref="S2.SS2.p1.6.m6.2.2.2"><apply id="S2.SS2.p1.6.m6.1.1.1.1.cmml" xref="S2.SS2.p1.6.m6.1.1.1.1"><minus id="S2.SS2.p1.6.m6.1.1.1.1.1.cmml" xref="S2.SS2.p1.6.m6.1.1.1.1"></minus><apply id="S2.SS2.p1.6.m6.1.1.1.1.2.cmml" xref="S2.SS2.p1.6.m6.1.1.1.1.2"><csymbol cd="ambiguous" id="S2.SS2.p1.6.m6.1.1.1.1.2.1.cmml" xref="S2.SS2.p1.6.m6.1.1.1.1.2">superscript</csymbol><cn type="integer" id="S2.SS2.p1.6.m6.1.1.1.1.2.2.cmml" xref="S2.SS2.p1.6.m6.1.1.1.1.2.2">2</cn><ci id="S2.SS2.p1.6.m6.1.1.1.1.2.3.cmml" xref="S2.SS2.p1.6.m6.1.1.1.1.2.3">𝑝</ci></apply></apply><apply id="S2.SS2.p1.6.m6.2.2.2.2.cmml" xref="S2.SS2.p1.6.m6.2.2.2.2"><csymbol cd="ambiguous" id="S2.SS2.p1.6.m6.2.2.2.2.1.cmml" xref="S2.SS2.p1.6.m6.2.2.2.2">superscript</csymbol><cn type="integer" id="S2.SS2.p1.6.m6.2.2.2.2.2.cmml" xref="S2.SS2.p1.6.m6.2.2.2.2.2">2</cn><ci id="S2.SS2.p1.6.m6.2.2.2.2.3.cmml" xref="S2.SS2.p1.6.m6.2.2.2.2.3">𝑝</ci></apply></interval></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.6.m6.2c">[-2^{p},2^{p}]</annotation></semantics></math> use <math id="S2.SS2.p1.7.m7.1" class="ltx_Math" alttext="p+1" display="inline"><semantics id="S2.SS2.p1.7.m7.1a"><mrow id="S2.SS2.p1.7.m7.1.1" xref="S2.SS2.p1.7.m7.1.1.cmml"><mi id="S2.SS2.p1.7.m7.1.1.2" xref="S2.SS2.p1.7.m7.1.1.2.cmml">p</mi><mo id="S2.SS2.p1.7.m7.1.1.1" xref="S2.SS2.p1.7.m7.1.1.1.cmml">+</mo><mn id="S2.SS2.p1.7.m7.1.1.3" xref="S2.SS2.p1.7.m7.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.7.m7.1b"><apply id="S2.SS2.p1.7.m7.1.1.cmml" xref="S2.SS2.p1.7.m7.1.1"><plus id="S2.SS2.p1.7.m7.1.1.1.cmml" xref="S2.SS2.p1.7.m7.1.1.1"></plus><ci id="S2.SS2.p1.7.m7.1.1.2.cmml" xref="S2.SS2.p1.7.m7.1.1.2">𝑝</ci><cn type="integer" id="S2.SS2.p1.7.m7.1.1.3.cmml" xref="S2.SS2.p1.7.m7.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.7.m7.1c">p+1</annotation></semantics></math> bits, FP16 uses <math id="S2.SS2.p1.8.m8.1" class="ltx_Math" alttext="16" display="inline"><semantics id="S2.SS2.p1.8.m8.1a"><mn id="S2.SS2.p1.8.m8.1.1" xref="S2.SS2.p1.8.m8.1.1.cmml">16</mn><annotation-xml encoding="MathML-Content" id="S2.SS2.p1.8.m8.1b"><cn type="integer" id="S2.SS2.p1.8.m8.1.1.cmml" xref="S2.SS2.p1.8.m8.1.1">16</cn></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p1.8.m8.1c">16</annotation></semantics></math> bits, etc.</p>
</div>
<div id="S2.SS2.p2" class="ltx_para">
<p id="S2.SS2.p2.3" class="ltx_p">Our main results (<a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorems</span> <span class="ltx_text ltx_ref_tag">4.1</span></a> and <a href="#S4.Thmtheorem2" title="Theorem 4.2. ‣ 4.2 Simulating S6 with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.2</span></a>) will go through for any datatype <math id="S2.SS2.p2.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p2.1.m1.1a"><mi id="S2.SS2.p2.1.m1.1.1" xref="S2.SS2.p2.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.1.m1.1b"><ci id="S2.SS2.p2.1.m1.1.1.cmml" xref="S2.SS2.p2.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.1.m1.1c">\mathbb{D}</annotation></semantics></math> for which the following 3 operations are efficiently parallel-computable, i.e., are in the complexity class <math id="S2.SS2.p2.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.SS2.p2.2.m2.1a"><mi id="S2.SS2.p2.2.m2.1.1" xref="S2.SS2.p2.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.2.m2.1b"><ci id="S2.SS2.p2.2.m2.1.1.cmml" xref="S2.SS2.p2.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.SS2.p2.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS2.p2.3.m3.1a"><msup id="S2.SS2.p2.3.m3.1.1" xref="S2.SS2.p2.3.m3.1.1.cmml"><mi id="S2.SS2.p2.3.m3.1.1.2" xref="S2.SS2.p2.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS2.p2.3.m3.1.1.3" xref="S2.SS2.p2.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.3.m3.1b"><apply id="S2.SS2.p2.3.m3.1.1.cmml" xref="S2.SS2.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S2.SS2.p2.3.m3.1.1.1.cmml" xref="S2.SS2.p2.3.m3.1.1">superscript</csymbol><ci id="S2.SS2.p2.3.m3.1.1.2.cmml" xref="S2.SS2.p2.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS2.p2.3.m3.1.1.3.cmml" xref="S2.SS2.p2.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math> (to be defined shortly in <a href="#S2.SS3" title="2.3 Limits of Transformers via Circuit Complexity ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">2.3</span></a>):</p>
<ol id="S2.I1" class="ltx_enumerate">
<li id="S2.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S2.I1.i1.p1" class="ltx_para">
<p id="S2.I1.i1.p1.2" class="ltx_p">Iterated addition, i.e., summing <math id="S2.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S2.I1.i1.p1.1.m1.1a"><mi id="S2.I1.i1.p1.1.m1.1.1" xref="S2.I1.i1.p1.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i1.p1.1.m1.1b"><ci id="S2.I1.i1.p1.1.m1.1.1.cmml" xref="S2.I1.i1.p1.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i1.p1.1.m1.1c">n</annotation></semantics></math> numbers in <math id="S2.I1.i1.p1.2.m2.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.I1.i1.p1.2.m2.1a"><mi id="S2.I1.i1.p1.2.m2.1.1" xref="S2.I1.i1.p1.2.m2.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i1.p1.2.m2.1b"><ci id="S2.I1.i1.p1.2.m2.1.1.cmml" xref="S2.I1.i1.p1.2.m2.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i1.p1.2.m2.1c">\mathbb{D}</annotation></semantics></math></p>
</div>
</li>
<li id="S2.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S2.I1.i2.p1" class="ltx_para">
<p id="S2.I1.i2.p1.2" class="ltx_p">Iterated product, i.e., multiplying <math id="S2.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S2.I1.i2.p1.1.m1.1a"><mi id="S2.I1.i2.p1.1.m1.1.1" xref="S2.I1.i2.p1.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.1.m1.1b"><ci id="S2.I1.i2.p1.1.m1.1.1.cmml" xref="S2.I1.i2.p1.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.1.m1.1c">n</annotation></semantics></math> numbers in <math id="S2.I1.i2.p1.2.m2.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.I1.i2.p1.2.m2.1a"><mi id="S2.I1.i2.p1.2.m2.1.1" xref="S2.I1.i2.p1.2.m2.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i2.p1.2.m2.1b"><ci id="S2.I1.i2.p1.2.m2.1.1.cmml" xref="S2.I1.i2.p1.2.m2.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i2.p1.2.m2.1c">\mathbb{D}</annotation></semantics></math></p>
</div>
</li>
<li id="S2.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="S2.I1.i3.p1" class="ltx_para">
<p id="S2.I1.i3.p1.3" class="ltx_p">Matrix powering, i.e., computing the <math id="S2.I1.i3.p1.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S2.I1.i3.p1.1.m1.1a"><mi id="S2.I1.i3.p1.1.m1.1.1" xref="S2.I1.i3.p1.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i3.p1.1.m1.1b"><ci id="S2.I1.i3.p1.1.m1.1.1.cmml" xref="S2.I1.i3.p1.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i3.p1.1.m1.1c">n</annotation></semantics></math>-th power of a fixed-size <math id="S2.I1.i3.p1.2.m2.1" class="ltx_Math" alttext="k\times k" display="inline"><semantics id="S2.I1.i3.p1.2.m2.1a"><mrow id="S2.I1.i3.p1.2.m2.1.1" xref="S2.I1.i3.p1.2.m2.1.1.cmml"><mi id="S2.I1.i3.p1.2.m2.1.1.2" xref="S2.I1.i3.p1.2.m2.1.1.2.cmml">k</mi><mo lspace="0.222em" rspace="0.222em" id="S2.I1.i3.p1.2.m2.1.1.1" xref="S2.I1.i3.p1.2.m2.1.1.1.cmml">×</mo><mi id="S2.I1.i3.p1.2.m2.1.1.3" xref="S2.I1.i3.p1.2.m2.1.1.3.cmml">k</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.I1.i3.p1.2.m2.1b"><apply id="S2.I1.i3.p1.2.m2.1.1.cmml" xref="S2.I1.i3.p1.2.m2.1.1"><times id="S2.I1.i3.p1.2.m2.1.1.1.cmml" xref="S2.I1.i3.p1.2.m2.1.1.1"></times><ci id="S2.I1.i3.p1.2.m2.1.1.2.cmml" xref="S2.I1.i3.p1.2.m2.1.1.2">𝑘</ci><ci id="S2.I1.i3.p1.2.m2.1.1.3.cmml" xref="S2.I1.i3.p1.2.m2.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i3.p1.2.m2.1c">k\times k</annotation></semantics></math> matrix over <math id="S2.I1.i3.p1.3.m3.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.I1.i3.p1.3.m3.1a"><mi id="S2.I1.i3.p1.3.m3.1.1" xref="S2.I1.i3.p1.3.m3.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.I1.i3.p1.3.m3.1b"><ci id="S2.I1.i3.p1.3.m3.1.1.cmml" xref="S2.I1.i3.p1.3.m3.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.I1.i3.p1.3.m3.1c">\mathbb{D}</annotation></semantics></math></p>
</div>
</li>
</ol>
</div>
<div id="S2.SS2.p3" class="ltx_para">
<p id="S2.SS2.p3.8" class="ltx_p">When <math id="S2.SS2.p3.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p3.1.m1.1a"><mi id="S2.SS2.p3.1.m1.1.1" xref="S2.SS2.p3.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.1.m1.1b"><ci id="S2.SS2.p3.1.m1.1.1.cmml" xref="S2.SS2.p3.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.1.m1.1c">\mathbb{D}</annotation></semantics></math> is any finite-precision datatype, i.e., has a fixed number of bits available (e.g., 16 or 64), then these operations are easily seen to be in <math id="S2.SS2.p3.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.SS2.p3.2.m2.1a"><mi id="S2.SS2.p3.2.m2.1.1" xref="S2.SS2.p3.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.2.m2.1b"><ci id="S2.SS2.p3.2.m2.1.1.cmml" xref="S2.SS2.p3.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.SS2.p3.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS2.p3.3.m3.1a"><msup id="S2.SS2.p3.3.m3.1.1" xref="S2.SS2.p3.3.m3.1.1.cmml"><mi id="S2.SS2.p3.3.m3.1.1.2" xref="S2.SS2.p3.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS2.p3.3.m3.1.1.3" xref="S2.SS2.p3.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.3.m3.1b"><apply id="S2.SS2.p3.3.m3.1.1.cmml" xref="S2.SS2.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S2.SS2.p3.3.m3.1.1.1.cmml" xref="S2.SS2.p3.3.m3.1.1">superscript</csymbol><ci id="S2.SS2.p3.3.m3.1.1.2.cmml" xref="S2.SS2.p3.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS2.p3.3.m3.1.1.3.cmml" xref="S2.SS2.p3.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math>. As <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib26" title="" class="ltx_ref">2023b</a>)</cite> argue, however, finite-precision datatypes severely limit the expressivity of neural architectures from a formal perspective (e.g., finite-precision transformers cannot represent uniform attention), motivating the use of parameterized datatypes that can (approximately) represent any number with a sufficiently large parameter. Interestingly, when <math id="S2.SS2.p3.4.m4.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p3.4.m4.1a"><mi id="S2.SS2.p3.4.m4.1.1" xref="S2.SS2.p3.4.m4.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.4.m4.1b"><ci id="S2.SS2.p3.4.m4.1.1.cmml" xref="S2.SS2.p3.4.m4.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.4.m4.1c">\mathbb{D}</annotation></semantics></math> is the datatype of <math id="S2.SS2.p3.5.m5.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S2.SS2.p3.5.m5.1a"><mi id="S2.SS2.p3.5.m5.1.1" xref="S2.SS2.p3.5.m5.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.5.m5.1b"><ci id="S2.SS2.p3.5.m5.1.1.cmml" xref="S2.SS2.p3.5.m5.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.5.m5.1c">n</annotation></semantics></math>-bit integers, all of the above operations are known to be in <math id="S2.SS2.p3.6.m6.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.SS2.p3.6.m6.1a"><mi id="S2.SS2.p3.6.m6.1.1" xref="S2.SS2.p3.6.m6.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.6.m6.1b"><ci id="S2.SS2.p3.6.m6.1.1.cmml" xref="S2.SS2.p3.6.m6.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.6.m6.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.SS2.p3.7.m7.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS2.p3.7.m7.1a"><msup id="S2.SS2.p3.7.m7.1.1" xref="S2.SS2.p3.7.m7.1.1.cmml"><mi id="S2.SS2.p3.7.m7.1.1.2" xref="S2.SS2.p3.7.m7.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS2.p3.7.m7.1.1.3" xref="S2.SS2.p3.7.m7.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.7.m7.1b"><apply id="S2.SS2.p3.7.m7.1.1.cmml" xref="S2.SS2.p3.7.m7.1.1"><csymbol cd="ambiguous" id="S2.SS2.p3.7.m7.1.1.1.cmml" xref="S2.SS2.p3.7.m7.1.1">superscript</csymbol><ci id="S2.SS2.p3.7.m7.1.1.2.cmml" xref="S2.SS2.p3.7.m7.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS2.p3.7.m7.1.1.3.cmml" xref="S2.SS2.p3.7.m7.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.7.m7.1c">\mathsf{TC}^{0}</annotation></semantics></math> <cite class="ltx_cite ltx_citemacro_citep">(Hesse, <a href="#bib.bib14" title="" class="ltx_ref">2001</a>; Mereghetti &amp; Palano, <a href="#bib.bib23" title="" class="ltx_ref">2000</a>)</cite>. Realistically, however, neural model implementations use floating point numbers with much fewer than <math id="S2.SS2.p3.8.m8.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S2.SS2.p3.8.m8.1a"><mi id="S2.SS2.p3.8.m8.1.1" xref="S2.SS2.p3.8.m8.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p3.8.m8.1b"><ci id="S2.SS2.p3.8.m8.1.1.cmml" xref="S2.SS2.p3.8.m8.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p3.8.m8.1c">n</annotation></semantics></math> bits.</p>
</div>
<div id="S2.SS2.p4" class="ltx_para">
<p id="S2.SS2.p4.8" class="ltx_p">Concretely, we imagine working with the <span id="S2.SS2.p4.8.1" class="ltx_text ltx_font_bold">log-precision floating point</span> model used by <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib26" title="" class="ltx_ref">2023b</a>)</cite> to analyze transformers, which we will show satisfy all these properties. For some fixed constant <math id="S2.SS2.p4.1.m1.1" class="ltx_Math" alttext="c\in\mathbb{Z}^{+}" display="inline"><semantics id="S2.SS2.p4.1.m1.1a"><mrow id="S2.SS2.p4.1.m1.1.1" xref="S2.SS2.p4.1.m1.1.1.cmml"><mi id="S2.SS2.p4.1.m1.1.1.2" xref="S2.SS2.p4.1.m1.1.1.2.cmml">c</mi><mo id="S2.SS2.p4.1.m1.1.1.1" xref="S2.SS2.p4.1.m1.1.1.1.cmml">∈</mo><msup id="S2.SS2.p4.1.m1.1.1.3" xref="S2.SS2.p4.1.m1.1.1.3.cmml"><mi id="S2.SS2.p4.1.m1.1.1.3.2" xref="S2.SS2.p4.1.m1.1.1.3.2.cmml">ℤ</mi><mo id="S2.SS2.p4.1.m1.1.1.3.3" xref="S2.SS2.p4.1.m1.1.1.3.3.cmml">+</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.1.m1.1b"><apply id="S2.SS2.p4.1.m1.1.1.cmml" xref="S2.SS2.p4.1.m1.1.1"><in id="S2.SS2.p4.1.m1.1.1.1.cmml" xref="S2.SS2.p4.1.m1.1.1.1"></in><ci id="S2.SS2.p4.1.m1.1.1.2.cmml" xref="S2.SS2.p4.1.m1.1.1.2">𝑐</ci><apply id="S2.SS2.p4.1.m1.1.1.3.cmml" xref="S2.SS2.p4.1.m1.1.1.3"><csymbol cd="ambiguous" id="S2.SS2.p4.1.m1.1.1.3.1.cmml" xref="S2.SS2.p4.1.m1.1.1.3">superscript</csymbol><ci id="S2.SS2.p4.1.m1.1.1.3.2.cmml" xref="S2.SS2.p4.1.m1.1.1.3.2">ℤ</ci><plus id="S2.SS2.p4.1.m1.1.1.3.3.cmml" xref="S2.SS2.p4.1.m1.1.1.3.3"></plus></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.1.m1.1c">c\in\mathbb{Z}^{+}</annotation></semantics></math>, a <math id="S2.SS2.p4.2.m2.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="S2.SS2.p4.2.m2.1a"><mrow id="S2.SS2.p4.2.m2.1.1" xref="S2.SS2.p4.2.m2.1.1.cmml"><mi id="S2.SS2.p4.2.m2.1.1.2" xref="S2.SS2.p4.2.m2.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="S2.SS2.p4.2.m2.1.1.1" xref="S2.SS2.p4.2.m2.1.1.1.cmml">​</mo><mrow id="S2.SS2.p4.2.m2.1.1.3" xref="S2.SS2.p4.2.m2.1.1.3.cmml"><mi id="S2.SS2.p4.2.m2.1.1.3.1" xref="S2.SS2.p4.2.m2.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="S2.SS2.p4.2.m2.1.1.3a" xref="S2.SS2.p4.2.m2.1.1.3.cmml">⁡</mo><mi id="S2.SS2.p4.2.m2.1.1.3.2" xref="S2.SS2.p4.2.m2.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.2.m2.1b"><apply id="S2.SS2.p4.2.m2.1.1.cmml" xref="S2.SS2.p4.2.m2.1.1"><times id="S2.SS2.p4.2.m2.1.1.1.cmml" xref="S2.SS2.p4.2.m2.1.1.1"></times><ci id="S2.SS2.p4.2.m2.1.1.2.cmml" xref="S2.SS2.p4.2.m2.1.1.2">𝑐</ci><apply id="S2.SS2.p4.2.m2.1.1.3.cmml" xref="S2.SS2.p4.2.m2.1.1.3"><log id="S2.SS2.p4.2.m2.1.1.3.1.cmml" xref="S2.SS2.p4.2.m2.1.1.3.1"></log><ci id="S2.SS2.p4.2.m2.1.1.3.2.cmml" xref="S2.SS2.p4.2.m2.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.2.m2.1c">c\log n</annotation></semantics></math> precision float is a tuple <math id="S2.SS2.p4.3.m3.2" class="ltx_Math" alttext="\langle m,e\rangle" display="inline"><semantics id="S2.SS2.p4.3.m3.2a"><mrow id="S2.SS2.p4.3.m3.2.3.2" xref="S2.SS2.p4.3.m3.2.3.1.cmml"><mo stretchy="false" id="S2.SS2.p4.3.m3.2.3.2.1" xref="S2.SS2.p4.3.m3.2.3.1.cmml">⟨</mo><mi id="S2.SS2.p4.3.m3.1.1" xref="S2.SS2.p4.3.m3.1.1.cmml">m</mi><mo id="S2.SS2.p4.3.m3.2.3.2.2" xref="S2.SS2.p4.3.m3.2.3.1.cmml">,</mo><mi id="S2.SS2.p4.3.m3.2.2" xref="S2.SS2.p4.3.m3.2.2.cmml">e</mi><mo stretchy="false" id="S2.SS2.p4.3.m3.2.3.2.3" xref="S2.SS2.p4.3.m3.2.3.1.cmml">⟩</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.3.m3.2b"><list id="S2.SS2.p4.3.m3.2.3.1.cmml" xref="S2.SS2.p4.3.m3.2.3.2"><ci id="S2.SS2.p4.3.m3.1.1.cmml" xref="S2.SS2.p4.3.m3.1.1">𝑚</ci><ci id="S2.SS2.p4.3.m3.2.2.cmml" xref="S2.SS2.p4.3.m3.2.2">𝑒</ci></list></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.3.m3.2c">\langle m,e\rangle</annotation></semantics></math> where <math id="S2.SS2.p4.4.m4.2" class="ltx_Math" alttext="m,e" display="inline"><semantics id="S2.SS2.p4.4.m4.2a"><mrow id="S2.SS2.p4.4.m4.2.3.2" xref="S2.SS2.p4.4.m4.2.3.1.cmml"><mi id="S2.SS2.p4.4.m4.1.1" xref="S2.SS2.p4.4.m4.1.1.cmml">m</mi><mo id="S2.SS2.p4.4.m4.2.3.2.1" xref="S2.SS2.p4.4.m4.2.3.1.cmml">,</mo><mi id="S2.SS2.p4.4.m4.2.2" xref="S2.SS2.p4.4.m4.2.2.cmml">e</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.4.m4.2b"><list id="S2.SS2.p4.4.m4.2.3.1.cmml" xref="S2.SS2.p4.4.m4.2.3.2"><ci id="S2.SS2.p4.4.m4.1.1.cmml" xref="S2.SS2.p4.4.m4.1.1">𝑚</ci><ci id="S2.SS2.p4.4.m4.2.2.cmml" xref="S2.SS2.p4.4.m4.2.2">𝑒</ci></list></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.4.m4.2c">m,e</annotation></semantics></math> are signed integers together taking <math id="S2.SS2.p4.5.m5.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="S2.SS2.p4.5.m5.1a"><mrow id="S2.SS2.p4.5.m5.1.1" xref="S2.SS2.p4.5.m5.1.1.cmml"><mi id="S2.SS2.p4.5.m5.1.1.2" xref="S2.SS2.p4.5.m5.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="S2.SS2.p4.5.m5.1.1.1" xref="S2.SS2.p4.5.m5.1.1.1.cmml">​</mo><mrow id="S2.SS2.p4.5.m5.1.1.3" xref="S2.SS2.p4.5.m5.1.1.3.cmml"><mi id="S2.SS2.p4.5.m5.1.1.3.1" xref="S2.SS2.p4.5.m5.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="S2.SS2.p4.5.m5.1.1.3a" xref="S2.SS2.p4.5.m5.1.1.3.cmml">⁡</mo><mi id="S2.SS2.p4.5.m5.1.1.3.2" xref="S2.SS2.p4.5.m5.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.5.m5.1b"><apply id="S2.SS2.p4.5.m5.1.1.cmml" xref="S2.SS2.p4.5.m5.1.1"><times id="S2.SS2.p4.5.m5.1.1.1.cmml" xref="S2.SS2.p4.5.m5.1.1.1"></times><ci id="S2.SS2.p4.5.m5.1.1.2.cmml" xref="S2.SS2.p4.5.m5.1.1.2">𝑐</ci><apply id="S2.SS2.p4.5.m5.1.1.3.cmml" xref="S2.SS2.p4.5.m5.1.1.3"><log id="S2.SS2.p4.5.m5.1.1.3.1.cmml" xref="S2.SS2.p4.5.m5.1.1.3.1"></log><ci id="S2.SS2.p4.5.m5.1.1.3.2.cmml" xref="S2.SS2.p4.5.m5.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.5.m5.1c">c\log n</annotation></semantics></math> bits. Using <math id="S2.SS2.p4.6.m6.1" class="ltx_Math" alttext="|x|" display="inline"><semantics id="S2.SS2.p4.6.m6.1a"><mrow id="S2.SS2.p4.6.m6.1.2.2" xref="S2.SS2.p4.6.m6.1.2.1.cmml"><mo stretchy="false" id="S2.SS2.p4.6.m6.1.2.2.1" xref="S2.SS2.p4.6.m6.1.2.1.1.cmml">|</mo><mi id="S2.SS2.p4.6.m6.1.1" xref="S2.SS2.p4.6.m6.1.1.cmml">x</mi><mo stretchy="false" id="S2.SS2.p4.6.m6.1.2.2.2" xref="S2.SS2.p4.6.m6.1.2.1.1.cmml">|</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.6.m6.1b"><apply id="S2.SS2.p4.6.m6.1.2.1.cmml" xref="S2.SS2.p4.6.m6.1.2.2"><abs id="S2.SS2.p4.6.m6.1.2.1.1.cmml" xref="S2.SS2.p4.6.m6.1.2.2.1"></abs><ci id="S2.SS2.p4.6.m6.1.1.cmml" xref="S2.SS2.p4.6.m6.1.1">𝑥</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.6.m6.1c">|x|</annotation></semantics></math> to mean the number of bits used to represent integer <math id="S2.SS2.p4.7.m7.1" class="ltx_Math" alttext="x" display="inline"><semantics id="S2.SS2.p4.7.m7.1a"><mi id="S2.SS2.p4.7.m7.1.1" xref="S2.SS2.p4.7.m7.1.1.cmml">x</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.7.m7.1b"><ci id="S2.SS2.p4.7.m7.1.1.cmml" xref="S2.SS2.p4.7.m7.1.1">𝑥</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.7.m7.1c">x</annotation></semantics></math>, this float represents the value <math id="S2.SS2.p4.8.m8.1" class="ltx_Math" alttext="m\cdot 2^{e-|m|+1}" display="inline"><semantics id="S2.SS2.p4.8.m8.1a"><mrow id="S2.SS2.p4.8.m8.1.2" xref="S2.SS2.p4.8.m8.1.2.cmml"><mi id="S2.SS2.p4.8.m8.1.2.2" xref="S2.SS2.p4.8.m8.1.2.2.cmml">m</mi><mo lspace="0.222em" rspace="0.222em" id="S2.SS2.p4.8.m8.1.2.1" xref="S2.SS2.p4.8.m8.1.2.1.cmml">⋅</mo><msup id="S2.SS2.p4.8.m8.1.2.3" xref="S2.SS2.p4.8.m8.1.2.3.cmml"><mn id="S2.SS2.p4.8.m8.1.2.3.2" xref="S2.SS2.p4.8.m8.1.2.3.2.cmml">2</mn><mrow id="S2.SS2.p4.8.m8.1.1.1" xref="S2.SS2.p4.8.m8.1.1.1.cmml"><mrow id="S2.SS2.p4.8.m8.1.1.1.3" xref="S2.SS2.p4.8.m8.1.1.1.3.cmml"><mi id="S2.SS2.p4.8.m8.1.1.1.3.2" xref="S2.SS2.p4.8.m8.1.1.1.3.2.cmml">e</mi><mo id="S2.SS2.p4.8.m8.1.1.1.3.1" xref="S2.SS2.p4.8.m8.1.1.1.3.1.cmml">−</mo><mrow id="S2.SS2.p4.8.m8.1.1.1.3.3.2" xref="S2.SS2.p4.8.m8.1.1.1.3.3.1.cmml"><mo stretchy="false" id="S2.SS2.p4.8.m8.1.1.1.3.3.2.1" xref="S2.SS2.p4.8.m8.1.1.1.3.3.1.1.cmml">|</mo><mi id="S2.SS2.p4.8.m8.1.1.1.1" xref="S2.SS2.p4.8.m8.1.1.1.1.cmml">m</mi><mo stretchy="false" id="S2.SS2.p4.8.m8.1.1.1.3.3.2.2" xref="S2.SS2.p4.8.m8.1.1.1.3.3.1.1.cmml">|</mo></mrow></mrow><mo id="S2.SS2.p4.8.m8.1.1.1.2" xref="S2.SS2.p4.8.m8.1.1.1.2.cmml">+</mo><mn id="S2.SS2.p4.8.m8.1.1.1.4" xref="S2.SS2.p4.8.m8.1.1.1.4.cmml">1</mn></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p4.8.m8.1b"><apply id="S2.SS2.p4.8.m8.1.2.cmml" xref="S2.SS2.p4.8.m8.1.2"><ci id="S2.SS2.p4.8.m8.1.2.1.cmml" xref="S2.SS2.p4.8.m8.1.2.1">⋅</ci><ci id="S2.SS2.p4.8.m8.1.2.2.cmml" xref="S2.SS2.p4.8.m8.1.2.2">𝑚</ci><apply id="S2.SS2.p4.8.m8.1.2.3.cmml" xref="S2.SS2.p4.8.m8.1.2.3"><csymbol cd="ambiguous" id="S2.SS2.p4.8.m8.1.2.3.1.cmml" xref="S2.SS2.p4.8.m8.1.2.3">superscript</csymbol><cn type="integer" id="S2.SS2.p4.8.m8.1.2.3.2.cmml" xref="S2.SS2.p4.8.m8.1.2.3.2">2</cn><apply id="S2.SS2.p4.8.m8.1.1.1.cmml" xref="S2.SS2.p4.8.m8.1.1.1"><plus id="S2.SS2.p4.8.m8.1.1.1.2.cmml" xref="S2.SS2.p4.8.m8.1.1.1.2"></plus><apply id="S2.SS2.p4.8.m8.1.1.1.3.cmml" xref="S2.SS2.p4.8.m8.1.1.1.3"><minus id="S2.SS2.p4.8.m8.1.1.1.3.1.cmml" xref="S2.SS2.p4.8.m8.1.1.1.3.1"></minus><ci id="S2.SS2.p4.8.m8.1.1.1.3.2.cmml" xref="S2.SS2.p4.8.m8.1.1.1.3.2">𝑒</ci><apply id="S2.SS2.p4.8.m8.1.1.1.3.3.1.cmml" xref="S2.SS2.p4.8.m8.1.1.1.3.3.2"><abs id="S2.SS2.p4.8.m8.1.1.1.3.3.1.1.cmml" xref="S2.SS2.p4.8.m8.1.1.1.3.3.2.1"></abs><ci id="S2.SS2.p4.8.m8.1.1.1.1.cmml" xref="S2.SS2.p4.8.m8.1.1.1.1">𝑚</ci></apply></apply><cn type="integer" id="S2.SS2.p4.8.m8.1.1.1.4.cmml" xref="S2.SS2.p4.8.m8.1.1.1.4">1</cn></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p4.8.m8.1c">m\cdot 2^{e-|m|+1}</annotation></semantics></math>.</p>
</div>
<div id="S2.SS2.p5" class="ltx_para">
<p id="S2.SS2.p5.9" class="ltx_p">Unlike for integers, arithmetic operations over log-precision floats are not closed. That is, the product <math id="S2.SS2.p5.1.m1.1" class="ltx_Math" alttext="\phi_{1}\times\phi_{2}" display="inline"><semantics id="S2.SS2.p5.1.m1.1a"><mrow id="S2.SS2.p5.1.m1.1.1" xref="S2.SS2.p5.1.m1.1.1.cmml"><msub id="S2.SS2.p5.1.m1.1.1.2" xref="S2.SS2.p5.1.m1.1.1.2.cmml"><mi id="S2.SS2.p5.1.m1.1.1.2.2" xref="S2.SS2.p5.1.m1.1.1.2.2.cmml">ϕ</mi><mn id="S2.SS2.p5.1.m1.1.1.2.3" xref="S2.SS2.p5.1.m1.1.1.2.3.cmml">1</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S2.SS2.p5.1.m1.1.1.1" xref="S2.SS2.p5.1.m1.1.1.1.cmml">×</mo><msub id="S2.SS2.p5.1.m1.1.1.3" xref="S2.SS2.p5.1.m1.1.1.3.cmml"><mi id="S2.SS2.p5.1.m1.1.1.3.2" xref="S2.SS2.p5.1.m1.1.1.3.2.cmml">ϕ</mi><mn id="S2.SS2.p5.1.m1.1.1.3.3" xref="S2.SS2.p5.1.m1.1.1.3.3.cmml">2</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.1.m1.1b"><apply id="S2.SS2.p5.1.m1.1.1.cmml" xref="S2.SS2.p5.1.m1.1.1"><times id="S2.SS2.p5.1.m1.1.1.1.cmml" xref="S2.SS2.p5.1.m1.1.1.1"></times><apply id="S2.SS2.p5.1.m1.1.1.2.cmml" xref="S2.SS2.p5.1.m1.1.1.2"><csymbol cd="ambiguous" id="S2.SS2.p5.1.m1.1.1.2.1.cmml" xref="S2.SS2.p5.1.m1.1.1.2">subscript</csymbol><ci id="S2.SS2.p5.1.m1.1.1.2.2.cmml" xref="S2.SS2.p5.1.m1.1.1.2.2">italic-ϕ</ci><cn type="integer" id="S2.SS2.p5.1.m1.1.1.2.3.cmml" xref="S2.SS2.p5.1.m1.1.1.2.3">1</cn></apply><apply id="S2.SS2.p5.1.m1.1.1.3.cmml" xref="S2.SS2.p5.1.m1.1.1.3"><csymbol cd="ambiguous" id="S2.SS2.p5.1.m1.1.1.3.1.cmml" xref="S2.SS2.p5.1.m1.1.1.3">subscript</csymbol><ci id="S2.SS2.p5.1.m1.1.1.3.2.cmml" xref="S2.SS2.p5.1.m1.1.1.3.2">italic-ϕ</ci><cn type="integer" id="S2.SS2.p5.1.m1.1.1.3.3.cmml" xref="S2.SS2.p5.1.m1.1.1.3.3">2</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.1.m1.1c">\phi_{1}\times\phi_{2}</annotation></semantics></math> of two <math id="S2.SS2.p5.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS2.p5.2.m2.1a"><mi id="S2.SS2.p5.2.m2.1.1" xref="S2.SS2.p5.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.2.m2.1b"><ci id="S2.SS2.p5.2.m2.1.1.cmml" xref="S2.SS2.p5.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.2.m2.1c">p</annotation></semantics></math>-precision floats is a well-defined number but may not be exactly representable as a <math id="S2.SS2.p5.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS2.p5.3.m3.1a"><mi id="S2.SS2.p5.3.m3.1.1" xref="S2.SS2.p5.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.3.m3.1b"><ci id="S2.SS2.p5.3.m3.1.1.cmml" xref="S2.SS2.p5.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.3.m3.1c">p</annotation></semantics></math>-precision float.
It is thus necessary to define approximate versions of these operations when formalizing log-precision floating-point arithmetic.
To this end, <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite> define a natural notion of approximate iterated addition over log-precision floats and show that it is computable in <math id="S2.SS2.p5.4.m4.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.SS2.p5.4.m4.1a"><mi id="S2.SS2.p5.4.m4.1.1" xref="S2.SS2.p5.4.m4.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.4.m4.1b"><ci id="S2.SS2.p5.4.m4.1.1.cmml" xref="S2.SS2.p5.4.m4.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.4.m4.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.SS2.p5.5.m5.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS2.p5.5.m5.1a"><msup id="S2.SS2.p5.5.m5.1.1" xref="S2.SS2.p5.5.m5.1.1.cmml"><mi id="S2.SS2.p5.5.m5.1.1.2" xref="S2.SS2.p5.5.m5.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS2.p5.5.m5.1.1.3" xref="S2.SS2.p5.5.m5.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.5.m5.1b"><apply id="S2.SS2.p5.5.m5.1.1.cmml" xref="S2.SS2.p5.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS2.p5.5.m5.1.1.1.cmml" xref="S2.SS2.p5.5.m5.1.1">superscript</csymbol><ci id="S2.SS2.p5.5.m5.1.1.2.cmml" xref="S2.SS2.p5.5.m5.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS2.p5.5.m5.1.1.3.cmml" xref="S2.SS2.p5.5.m5.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.5.m5.1c">\mathsf{TC}^{0}</annotation></semantics></math>.
Following <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>’s definition of iterated addition for floats, we define iterated multiplication and matrix powering over any datatype <math id="S2.SS2.p5.6.m6.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p5.6.m6.1a"><mi id="S2.SS2.p5.6.m6.1.1" xref="S2.SS2.p5.6.m6.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.6.m6.1b"><ci id="S2.SS2.p5.6.m6.1.1.cmml" xref="S2.SS2.p5.6.m6.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.6.m6.1c">\mathbb{D}</annotation></semantics></math> as the result of treating the numbers as reals, performing exact arithmetic, and casting the exact output <math id="S2.SS2.p5.7.m7.1" class="ltx_Math" alttext="\phi" display="inline"><semantics id="S2.SS2.p5.7.m7.1a"><mi id="S2.SS2.p5.7.m7.1.1" xref="S2.SS2.p5.7.m7.1.1.cmml">ϕ</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.7.m7.1b"><ci id="S2.SS2.p5.7.m7.1.1.cmml" xref="S2.SS2.p5.7.m7.1.1">italic-ϕ</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.7.m7.1c">\phi</annotation></semantics></math> back to <math id="S2.SS2.p5.8.m8.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="S2.SS2.p5.8.m8.1a"><mi id="S2.SS2.p5.8.m8.1.1" xref="S2.SS2.p5.8.m8.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.8.m8.1b"><ci id="S2.SS2.p5.8.m8.1.1.cmml" xref="S2.SS2.p5.8.m8.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.8.m8.1c">\mathbb{D}</annotation></semantics></math>, denoted <math id="S2.SS2.p5.9.m9.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{D}}(\phi)" display="inline"><semantics id="S2.SS2.p5.9.m9.1a"><mrow id="S2.SS2.p5.9.m9.1.2" xref="S2.SS2.p5.9.m9.1.2.cmml"><msub id="S2.SS2.p5.9.m9.1.2.2" xref="S2.SS2.p5.9.m9.1.2.2.cmml"><mi id="S2.SS2.p5.9.m9.1.2.2.2" xref="S2.SS2.p5.9.m9.1.2.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="S2.SS2.p5.9.m9.1.2.2.3" xref="S2.SS2.p5.9.m9.1.2.2.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="S2.SS2.p5.9.m9.1.2.1" xref="S2.SS2.p5.9.m9.1.2.1.cmml">​</mo><mrow id="S2.SS2.p5.9.m9.1.2.3.2" xref="S2.SS2.p5.9.m9.1.2.cmml"><mo stretchy="false" id="S2.SS2.p5.9.m9.1.2.3.2.1" xref="S2.SS2.p5.9.m9.1.2.cmml">(</mo><mi id="S2.SS2.p5.9.m9.1.1" xref="S2.SS2.p5.9.m9.1.1.cmml">ϕ</mi><mo stretchy="false" id="S2.SS2.p5.9.m9.1.2.3.2.2" xref="S2.SS2.p5.9.m9.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p5.9.m9.1b"><apply id="S2.SS2.p5.9.m9.1.2.cmml" xref="S2.SS2.p5.9.m9.1.2"><times id="S2.SS2.p5.9.m9.1.2.1.cmml" xref="S2.SS2.p5.9.m9.1.2.1"></times><apply id="S2.SS2.p5.9.m9.1.2.2.cmml" xref="S2.SS2.p5.9.m9.1.2.2"><csymbol cd="ambiguous" id="S2.SS2.p5.9.m9.1.2.2.1.cmml" xref="S2.SS2.p5.9.m9.1.2.2">subscript</csymbol><ci id="S2.SS2.p5.9.m9.1.2.2.2.cmml" xref="S2.SS2.p5.9.m9.1.2.2.2">𝖼𝖺𝗌𝗍</ci><ci id="S2.SS2.p5.9.m9.1.2.2.3.cmml" xref="S2.SS2.p5.9.m9.1.2.2.3">𝔻</ci></apply><ci id="S2.SS2.p5.9.m9.1.1.cmml" xref="S2.SS2.p5.9.m9.1.1">italic-ϕ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p5.9.m9.1c">\mathsf{cast}_{\mathbb{D}}(\phi)</annotation></semantics></math>.</p>
</div>
<div id="S2.SS2.p6" class="ltx_para">
<p id="S2.SS2.p6.2" class="ltx_p">In <a href="#A1" title="Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">A</span></a>, we extend the arguments of <cite class="ltx_cite ltx_citemacro_citet">Hesse (<a href="#bib.bib14" title="" class="ltx_ref">2001</a>)</cite> and <cite class="ltx_cite ltx_citemacro_citet">Mereghetti &amp; Palano (<a href="#bib.bib23" title="" class="ltx_ref">2000</a>)</cite> for integers to show that iterated product and matrix powering over log-precision floats are also computable in <math id="S2.SS2.p6.1.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.SS2.p6.1.m1.1a"><mi id="S2.SS2.p6.1.m1.1.1" xref="S2.SS2.p6.1.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p6.1.m1.1b"><ci id="S2.SS2.p6.1.m1.1.1.cmml" xref="S2.SS2.p6.1.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p6.1.m1.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.SS2.p6.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS2.p6.2.m2.1a"><msup id="S2.SS2.p6.2.m2.1.1" xref="S2.SS2.p6.2.m2.1.1.cmml"><mi id="S2.SS2.p6.2.m2.1.1.2" xref="S2.SS2.p6.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS2.p6.2.m2.1.1.3" xref="S2.SS2.p6.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS2.p6.2.m2.1b"><apply id="S2.SS2.p6.2.m2.1.1.cmml" xref="S2.SS2.p6.2.m2.1.1"><csymbol cd="ambiguous" id="S2.SS2.p6.2.m2.1.1.1.cmml" xref="S2.SS2.p6.2.m2.1.1">superscript</csymbol><ci id="S2.SS2.p6.2.m2.1.1.2.cmml" xref="S2.SS2.p6.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS2.p6.2.m2.1.1.3.cmml" xref="S2.SS2.p6.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p6.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math> (cf. <a href="#S2.Thmtheorem3" title="Lemma 2.3 (Iterated float product). ‣ 2.2 Numeric Datatype ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemmas</span> <span class="ltx_text ltx_ref_tag">2.3</span></a> and <a href="#S2.Thmtheorem4" title="Lemma 2.4 (Float matrix power). ‣ 2.2 Numeric Datatype ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.4</span></a> in <a href="#A1" title="Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">A</span></a>).</p>
</div>
<div id="S2.Thmtheorem3" class="ltx_theorem ltx_theorem_lemma">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S2.Thmtheorem3.1.1.1" class="ltx_text ltx_font_bold">Lemma 2.3</span></span><span id="S2.Thmtheorem3.2.2" class="ltx_text ltx_font_bold"> </span>(Iterated float product)<span id="S2.Thmtheorem3.3.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S2.Thmtheorem3.p1" class="ltx_para">
<p id="S2.Thmtheorem3.p1.6" class="ltx_p"><span id="S2.Thmtheorem3.p1.6.6" class="ltx_text ltx_font_italic">Let <math id="S2.Thmtheorem3.p1.1.1.m1.3" class="ltx_Math" alttext="\phi_{1},\ldots,\phi_{z}" display="inline"><semantics id="S2.Thmtheorem3.p1.1.1.m1.3a"><mrow id="S2.Thmtheorem3.p1.1.1.m1.3.3.2" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.3.cmml"><msub id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.cmml"><mi id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.2" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.2.cmml">ϕ</mi><mn id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.3" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.3.cmml">1</mn></msub><mo id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.3" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.3.cmml">,</mo><mi mathvariant="normal" id="S2.Thmtheorem3.p1.1.1.m1.1.1" xref="S2.Thmtheorem3.p1.1.1.m1.1.1.cmml">…</mi><mo id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.4" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.3.cmml">,</mo><msub id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.cmml"><mi id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.2" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.2.cmml">ϕ</mi><mi id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.3" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.3.cmml">z</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem3.p1.1.1.m1.3b"><list id="S2.Thmtheorem3.p1.1.1.m1.3.3.3.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2"><apply id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.1.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1">subscript</csymbol><ci id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.2.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.2">italic-ϕ</ci><cn type="integer" id="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.3.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.2.2.1.1.3">1</cn></apply><ci id="S2.Thmtheorem3.p1.1.1.m1.1.1.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.1.1">…</ci><apply id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.1.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2">subscript</csymbol><ci id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.2.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.2">italic-ϕ</ci><ci id="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.3.cmml" xref="S2.Thmtheorem3.p1.1.1.m1.3.3.2.2.3">𝑧</ci></apply></list></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem3.p1.1.1.m1.3c">\phi_{1},\ldots,\phi_{z}</annotation></semantics></math> be <math id="S2.Thmtheorem3.p1.2.2.m2.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="S2.Thmtheorem3.p1.2.2.m2.1a"><mrow id="S2.Thmtheorem3.p1.2.2.m2.1.1" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.cmml"><mi id="S2.Thmtheorem3.p1.2.2.m2.1.1.2" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="S2.Thmtheorem3.p1.2.2.m2.1.1.1" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.1.cmml">​</mo><mrow id="S2.Thmtheorem3.p1.2.2.m2.1.1.3" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3.cmml"><mi id="S2.Thmtheorem3.p1.2.2.m2.1.1.3.1" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="S2.Thmtheorem3.p1.2.2.m2.1.1.3a" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3.cmml">⁡</mo><mi id="S2.Thmtheorem3.p1.2.2.m2.1.1.3.2" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem3.p1.2.2.m2.1b"><apply id="S2.Thmtheorem3.p1.2.2.m2.1.1.cmml" xref="S2.Thmtheorem3.p1.2.2.m2.1.1"><times id="S2.Thmtheorem3.p1.2.2.m2.1.1.1.cmml" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.1"></times><ci id="S2.Thmtheorem3.p1.2.2.m2.1.1.2.cmml" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.2">𝑐</ci><apply id="S2.Thmtheorem3.p1.2.2.m2.1.1.3.cmml" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3"><log id="S2.Thmtheorem3.p1.2.2.m2.1.1.3.1.cmml" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3.1"></log><ci id="S2.Thmtheorem3.p1.2.2.m2.1.1.3.2.cmml" xref="S2.Thmtheorem3.p1.2.2.m2.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem3.p1.2.2.m2.1c">c\log n</annotation></semantics></math> precision floats and <math id="S2.Thmtheorem3.p1.3.3.m3.1" class="ltx_Math" alttext="z\leq n" display="inline"><semantics id="S2.Thmtheorem3.p1.3.3.m3.1a"><mrow id="S2.Thmtheorem3.p1.3.3.m3.1.1" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.cmml"><mi id="S2.Thmtheorem3.p1.3.3.m3.1.1.2" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.2.cmml">z</mi><mo id="S2.Thmtheorem3.p1.3.3.m3.1.1.1" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.1.cmml">≤</mo><mi id="S2.Thmtheorem3.p1.3.3.m3.1.1.3" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.3.cmml">n</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem3.p1.3.3.m3.1b"><apply id="S2.Thmtheorem3.p1.3.3.m3.1.1.cmml" xref="S2.Thmtheorem3.p1.3.3.m3.1.1"><leq id="S2.Thmtheorem3.p1.3.3.m3.1.1.1.cmml" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.1"></leq><ci id="S2.Thmtheorem3.p1.3.3.m3.1.1.2.cmml" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.2">𝑧</ci><ci id="S2.Thmtheorem3.p1.3.3.m3.1.1.3.cmml" xref="S2.Thmtheorem3.p1.3.3.m3.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem3.p1.3.3.m3.1c">z\leq n</annotation></semantics></math>. Then the iterated float product <math id="S2.Thmtheorem3.p1.4.4.m4.1" class="ltx_Math" alttext="\bigotimes_{i=1}^{z}\phi_{i}" display="inline"><semantics id="S2.Thmtheorem3.p1.4.4.m4.1a"><mrow id="S2.Thmtheorem3.p1.4.4.m4.1.1" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.cmml"><msubsup id="S2.Thmtheorem3.p1.4.4.m4.1.1.1" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.cmml"><mo id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.2" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.2.cmml">⨂</mo><mrow id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.cmml"><mi id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.2" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.2.cmml">i</mi><mo id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.1" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.1.cmml">=</mo><mn id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.3" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.3" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.3.cmml">z</mi></msubsup><msub id="S2.Thmtheorem3.p1.4.4.m4.1.1.2" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2.cmml"><mi id="S2.Thmtheorem3.p1.4.4.m4.1.1.2.2" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2.2.cmml">ϕ</mi><mi id="S2.Thmtheorem3.p1.4.4.m4.1.1.2.3" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem3.p1.4.4.m4.1b"><apply id="S2.Thmtheorem3.p1.4.4.m4.1.1.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1"><apply id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.1.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1">superscript</csymbol><apply id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.1.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1">subscript</csymbol><csymbol cd="latexml" id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.2.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.2">tensor-product</csymbol><apply id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3"><eq id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.1.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.1"></eq><ci id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.2.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.2">𝑖</ci><cn type="integer" id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.3.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.2.3.3">1</cn></apply></apply><ci id="S2.Thmtheorem3.p1.4.4.m4.1.1.1.3.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.1.3">𝑧</ci></apply><apply id="S2.Thmtheorem3.p1.4.4.m4.1.1.2.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2"><csymbol cd="ambiguous" id="S2.Thmtheorem3.p1.4.4.m4.1.1.2.1.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2">subscript</csymbol><ci id="S2.Thmtheorem3.p1.4.4.m4.1.1.2.2.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2.2">italic-ϕ</ci><ci id="S2.Thmtheorem3.p1.4.4.m4.1.1.2.3.cmml" xref="S2.Thmtheorem3.p1.4.4.m4.1.1.2.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem3.p1.4.4.m4.1c">\bigotimes_{i=1}^{z}\phi_{i}</annotation></semantics></math> can be computed in <math id="S2.Thmtheorem3.p1.5.5.m5.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.Thmtheorem3.p1.5.5.m5.1a"><mi id="S2.Thmtheorem3.p1.5.5.m5.1.1" xref="S2.Thmtheorem3.p1.5.5.m5.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem3.p1.5.5.m5.1b"><ci id="S2.Thmtheorem3.p1.5.5.m5.1.1.cmml" xref="S2.Thmtheorem3.p1.5.5.m5.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem3.p1.5.5.m5.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.Thmtheorem3.p1.6.6.m6.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.Thmtheorem3.p1.6.6.m6.1a"><msup id="S2.Thmtheorem3.p1.6.6.m6.1.1" xref="S2.Thmtheorem3.p1.6.6.m6.1.1.cmml"><mi id="S2.Thmtheorem3.p1.6.6.m6.1.1.2" xref="S2.Thmtheorem3.p1.6.6.m6.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.Thmtheorem3.p1.6.6.m6.1.1.3" xref="S2.Thmtheorem3.p1.6.6.m6.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem3.p1.6.6.m6.1b"><apply id="S2.Thmtheorem3.p1.6.6.m6.1.1.cmml" xref="S2.Thmtheorem3.p1.6.6.m6.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem3.p1.6.6.m6.1.1.1.cmml" xref="S2.Thmtheorem3.p1.6.6.m6.1.1">superscript</csymbol><ci id="S2.Thmtheorem3.p1.6.6.m6.1.1.2.cmml" xref="S2.Thmtheorem3.p1.6.6.m6.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.Thmtheorem3.p1.6.6.m6.1.1.3.cmml" xref="S2.Thmtheorem3.p1.6.6.m6.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem3.p1.6.6.m6.1c">\mathsf{TC}^{0}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="S2.Thmtheorem4" class="ltx_theorem ltx_theorem_lemma">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S2.Thmtheorem4.1.1.1" class="ltx_text ltx_font_bold">Lemma 2.4</span></span><span id="S2.Thmtheorem4.2.2" class="ltx_text ltx_font_bold"> </span>(Float matrix power)<span id="S2.Thmtheorem4.3.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S2.Thmtheorem4.p1" class="ltx_para">
<p id="S2.Thmtheorem4.p1.8" class="ltx_p"><span id="S2.Thmtheorem4.p1.8.8" class="ltx_text ltx_font_italic">Let <math id="S2.Thmtheorem4.p1.1.1.m1.2" class="ltx_Math" alttext="k,c\in\mathbb{Z}^{+}" display="inline"><semantics id="S2.Thmtheorem4.p1.1.1.m1.2a"><mrow id="S2.Thmtheorem4.p1.1.1.m1.2.3" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.cmml"><mrow id="S2.Thmtheorem4.p1.1.1.m1.2.3.2.2" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.2.1.cmml"><mi id="S2.Thmtheorem4.p1.1.1.m1.1.1" xref="S2.Thmtheorem4.p1.1.1.m1.1.1.cmml">k</mi><mo id="S2.Thmtheorem4.p1.1.1.m1.2.3.2.2.1" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.2.1.cmml">,</mo><mi id="S2.Thmtheorem4.p1.1.1.m1.2.2" xref="S2.Thmtheorem4.p1.1.1.m1.2.2.cmml">c</mi></mrow><mo id="S2.Thmtheorem4.p1.1.1.m1.2.3.1" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.1.cmml">∈</mo><msup id="S2.Thmtheorem4.p1.1.1.m1.2.3.3" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3.cmml"><mi id="S2.Thmtheorem4.p1.1.1.m1.2.3.3.2" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3.2.cmml">ℤ</mi><mo id="S2.Thmtheorem4.p1.1.1.m1.2.3.3.3" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3.3.cmml">+</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.1.1.m1.2b"><apply id="S2.Thmtheorem4.p1.1.1.m1.2.3.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3"><in id="S2.Thmtheorem4.p1.1.1.m1.2.3.1.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.1"></in><list id="S2.Thmtheorem4.p1.1.1.m1.2.3.2.1.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.2.2"><ci id="S2.Thmtheorem4.p1.1.1.m1.1.1.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.1.1">𝑘</ci><ci id="S2.Thmtheorem4.p1.1.1.m1.2.2.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.2">𝑐</ci></list><apply id="S2.Thmtheorem4.p1.1.1.m1.2.3.3.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3"><csymbol cd="ambiguous" id="S2.Thmtheorem4.p1.1.1.m1.2.3.3.1.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3">superscript</csymbol><ci id="S2.Thmtheorem4.p1.1.1.m1.2.3.3.2.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3.2">ℤ</ci><plus id="S2.Thmtheorem4.p1.1.1.m1.2.3.3.3.cmml" xref="S2.Thmtheorem4.p1.1.1.m1.2.3.3.3"></plus></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.1.1.m1.2c">k,c\in\mathbb{Z}^{+}</annotation></semantics></math> be fixed constants.
Let <math id="S2.Thmtheorem4.p1.2.2.m2.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S2.Thmtheorem4.p1.2.2.m2.1a"><mi id="S2.Thmtheorem4.p1.2.2.m2.1.1" xref="S2.Thmtheorem4.p1.2.2.m2.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.2.2.m2.1b"><ci id="S2.Thmtheorem4.p1.2.2.m2.1.1.cmml" xref="S2.Thmtheorem4.p1.2.2.m2.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.2.2.m2.1c">M</annotation></semantics></math> be a <math id="S2.Thmtheorem4.p1.3.3.m3.1" class="ltx_Math" alttext="k\times k" display="inline"><semantics id="S2.Thmtheorem4.p1.3.3.m3.1a"><mrow id="S2.Thmtheorem4.p1.3.3.m3.1.1" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.cmml"><mi id="S2.Thmtheorem4.p1.3.3.m3.1.1.2" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.2.cmml">k</mi><mo lspace="0.222em" rspace="0.222em" id="S2.Thmtheorem4.p1.3.3.m3.1.1.1" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.1.cmml">×</mo><mi id="S2.Thmtheorem4.p1.3.3.m3.1.1.3" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.3.cmml">k</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.3.3.m3.1b"><apply id="S2.Thmtheorem4.p1.3.3.m3.1.1.cmml" xref="S2.Thmtheorem4.p1.3.3.m3.1.1"><times id="S2.Thmtheorem4.p1.3.3.m3.1.1.1.cmml" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.1"></times><ci id="S2.Thmtheorem4.p1.3.3.m3.1.1.2.cmml" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.2">𝑘</ci><ci id="S2.Thmtheorem4.p1.3.3.m3.1.1.3.cmml" xref="S2.Thmtheorem4.p1.3.3.m3.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.3.3.m3.1c">k\times k</annotation></semantics></math> matrix over <math id="S2.Thmtheorem4.p1.4.4.m4.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="S2.Thmtheorem4.p1.4.4.m4.1a"><mrow id="S2.Thmtheorem4.p1.4.4.m4.1.1" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.cmml"><mi id="S2.Thmtheorem4.p1.4.4.m4.1.1.2" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="S2.Thmtheorem4.p1.4.4.m4.1.1.1" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.1.cmml">​</mo><mrow id="S2.Thmtheorem4.p1.4.4.m4.1.1.3" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3.cmml"><mi id="S2.Thmtheorem4.p1.4.4.m4.1.1.3.1" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="S2.Thmtheorem4.p1.4.4.m4.1.1.3a" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3.cmml">⁡</mo><mi id="S2.Thmtheorem4.p1.4.4.m4.1.1.3.2" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.4.4.m4.1b"><apply id="S2.Thmtheorem4.p1.4.4.m4.1.1.cmml" xref="S2.Thmtheorem4.p1.4.4.m4.1.1"><times id="S2.Thmtheorem4.p1.4.4.m4.1.1.1.cmml" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.1"></times><ci id="S2.Thmtheorem4.p1.4.4.m4.1.1.2.cmml" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.2">𝑐</ci><apply id="S2.Thmtheorem4.p1.4.4.m4.1.1.3.cmml" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3"><log id="S2.Thmtheorem4.p1.4.4.m4.1.1.3.1.cmml" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3.1"></log><ci id="S2.Thmtheorem4.p1.4.4.m4.1.1.3.2.cmml" xref="S2.Thmtheorem4.p1.4.4.m4.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.4.4.m4.1c">c\log n</annotation></semantics></math> precision floats. Let <math id="S2.Thmtheorem4.p1.5.5.m5.2" class="ltx_Math" alttext="z\leq n,z\in\mathbb{Z}^{+}" display="inline"><semantics id="S2.Thmtheorem4.p1.5.5.m5.2a"><mrow id="S2.Thmtheorem4.p1.5.5.m5.2.2.2" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.3.cmml"><mrow id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.cmml"><mi id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.2" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.2.cmml">z</mi><mo id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.1" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.1.cmml">≤</mo><mi id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.3" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.3.cmml">n</mi></mrow><mo id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.3" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.3a.cmml">,</mo><mrow id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.cmml"><mi id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.2" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.2.cmml">z</mi><mo id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.1" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.1.cmml">∈</mo><msup id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.cmml"><mi id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2.cmml">ℤ</mi><mo id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3.cmml">+</mo></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.5.5.m5.2b"><apply id="S2.Thmtheorem4.p1.5.5.m5.2.2.3.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2"><csymbol cd="ambiguous" id="S2.Thmtheorem4.p1.5.5.m5.2.2.3a.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.3">formulae-sequence</csymbol><apply id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1"><leq id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.1.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.1"></leq><ci id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.2.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.2">𝑧</ci><ci id="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.3.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.1.1.1.1.3">𝑛</ci></apply><apply id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2"><in id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.1.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.1"></in><ci id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.2.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.2">𝑧</ci><apply id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3"><csymbol cd="ambiguous" id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.1.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3">superscript</csymbol><ci id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2">ℤ</ci><plus id="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3.cmml" xref="S2.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3"></plus></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.5.5.m5.2c">z\leq n,z\in\mathbb{Z}^{+}</annotation></semantics></math>. Then float matrix power <math id="S2.Thmtheorem4.p1.6.6.m6.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="S2.Thmtheorem4.p1.6.6.m6.1a"><msup id="S2.Thmtheorem4.p1.6.6.m6.1.1" xref="S2.Thmtheorem4.p1.6.6.m6.1.1.cmml"><mi id="S2.Thmtheorem4.p1.6.6.m6.1.1.2" xref="S2.Thmtheorem4.p1.6.6.m6.1.1.2.cmml">M</mi><mi id="S2.Thmtheorem4.p1.6.6.m6.1.1.3" xref="S2.Thmtheorem4.p1.6.6.m6.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.6.6.m6.1b"><apply id="S2.Thmtheorem4.p1.6.6.m6.1.1.cmml" xref="S2.Thmtheorem4.p1.6.6.m6.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem4.p1.6.6.m6.1.1.1.cmml" xref="S2.Thmtheorem4.p1.6.6.m6.1.1">superscript</csymbol><ci id="S2.Thmtheorem4.p1.6.6.m6.1.1.2.cmml" xref="S2.Thmtheorem4.p1.6.6.m6.1.1.2">𝑀</ci><ci id="S2.Thmtheorem4.p1.6.6.m6.1.1.3.cmml" xref="S2.Thmtheorem4.p1.6.6.m6.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.6.6.m6.1c">M^{z}</annotation></semantics></math> can be computed in <math id="S2.Thmtheorem4.p1.7.7.m7.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.Thmtheorem4.p1.7.7.m7.1a"><mi id="S2.Thmtheorem4.p1.7.7.m7.1.1" xref="S2.Thmtheorem4.p1.7.7.m7.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.7.7.m7.1b"><ci id="S2.Thmtheorem4.p1.7.7.m7.1.1.cmml" xref="S2.Thmtheorem4.p1.7.7.m7.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.7.7.m7.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S2.Thmtheorem4.p1.8.8.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.Thmtheorem4.p1.8.8.m8.1a"><msup id="S2.Thmtheorem4.p1.8.8.m8.1.1" xref="S2.Thmtheorem4.p1.8.8.m8.1.1.cmml"><mi id="S2.Thmtheorem4.p1.8.8.m8.1.1.2" xref="S2.Thmtheorem4.p1.8.8.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.Thmtheorem4.p1.8.8.m8.1.1.3" xref="S2.Thmtheorem4.p1.8.8.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.Thmtheorem4.p1.8.8.m8.1b"><apply id="S2.Thmtheorem4.p1.8.8.m8.1.1.cmml" xref="S2.Thmtheorem4.p1.8.8.m8.1.1"><csymbol cd="ambiguous" id="S2.Thmtheorem4.p1.8.8.m8.1.1.1.cmml" xref="S2.Thmtheorem4.p1.8.8.m8.1.1">superscript</csymbol><ci id="S2.Thmtheorem4.p1.8.8.m8.1.1.2.cmml" xref="S2.Thmtheorem4.p1.8.8.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.Thmtheorem4.p1.8.8.m8.1.1.3.cmml" xref="S2.Thmtheorem4.p1.8.8.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Thmtheorem4.p1.8.8.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="S2.SS2.p7" class="ltx_para">
<p id="S2.SS2.p7.1" class="ltx_p">Combined with the result for iterated addition from <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>, this establishes that the three needed properties are met for log-precision floats.</p>
</div>
</section>
<section id="S2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.3 </span>Limits of Transformers via Circuit Complexity</h3>

<div id="S2.SS3.p1" class="ltx_para">
<p id="S2.SS3.p1.6" class="ltx_p">A line of recent work has used circuit complexity and logic formalisms to identify the expressiveness limitations of transformers on reasoning problems <cite class="ltx_cite ltx_citemacro_citep">(Angluin et al., <a href="#bib.bib1" title="" class="ltx_ref">2023</a>; Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>; Liu et al., <a href="#bib.bib21" title="" class="ltx_ref">2023</a>; Chiang et al., <a href="#bib.bib3" title="" class="ltx_ref">2023</a>; Merrill &amp; Sabharwal, <a href="#bib.bib26" title="" class="ltx_ref">2023b</a>; Hao et al., <a href="#bib.bib12" title="" class="ltx_ref">2022</a>)</cite>; see <cite class="ltx_cite ltx_citemacro_citep">Strobl et al., <a href="#bib.bib30" title="" class="ltx_ref">2023</a></cite> for a survey. In particular, <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite> showed that transformers can only solve problems in the complexity class <math id="S2.SS3.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS3.p1.1.m1.1a"><msup id="S2.SS3.p1.1.m1.1.1" xref="S2.SS3.p1.1.m1.1.1.cmml"><mi id="S2.SS3.p1.1.m1.1.1.2" xref="S2.SS3.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p1.1.m1.1.1.3" xref="S2.SS3.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.1.m1.1b"><apply id="S2.SS3.p1.1.m1.1.1.cmml" xref="S2.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p1.1.m1.1.1.1.cmml" xref="S2.SS3.p1.1.m1.1.1">superscript</csymbol><ci id="S2.SS3.p1.1.m1.1.1.2.cmml" xref="S2.SS3.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p1.1.m1.1.1.3.cmml" xref="S2.SS3.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>, which is defined as the set of problems that can be recognized by constant-depth, polynomial-size threshold circuit families. Such circuits, in addition to having standard AND, OR, and NOT gates (of arbitrary fan-in), can also use threshold gates that output <math id="S2.SS3.p1.2.m2.1" class="ltx_Math" alttext="1" display="inline"><semantics id="S2.SS3.p1.2.m2.1a"><mn id="S2.SS3.p1.2.m2.1.1" xref="S2.SS3.p1.2.m2.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.2.m2.1b"><cn type="integer" id="S2.SS3.p1.2.m2.1.1.cmml" xref="S2.SS3.p1.2.m2.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.2.m2.1c">1</annotation></semantics></math> iff at least <math id="S2.SS3.p1.3.m3.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S2.SS3.p1.3.m3.1a"><mi id="S2.SS3.p1.3.m3.1.1" xref="S2.SS3.p1.3.m3.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.3.m3.1b"><ci id="S2.SS3.p1.3.m3.1.1.cmml" xref="S2.SS3.p1.3.m3.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.3.m3.1c">k</annotation></semantics></math> of the inputs are <math id="S2.SS3.p1.4.m4.1" class="ltx_Math" alttext="1" display="inline"><semantics id="S2.SS3.p1.4.m4.1a"><mn id="S2.SS3.p1.4.m4.1.1" xref="S2.SS3.p1.4.m4.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.4.m4.1b"><cn type="integer" id="S2.SS3.p1.4.m4.1.1.cmml" xref="S2.SS3.p1.4.m4.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.4.m4.1c">1</annotation></semantics></math>, where <math id="S2.SS3.p1.5.m5.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S2.SS3.p1.5.m5.1a"><mi id="S2.SS3.p1.5.m5.1.1" xref="S2.SS3.p1.5.m5.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.5.m5.1b"><ci id="S2.SS3.p1.5.m5.1.1.cmml" xref="S2.SS3.p1.5.m5.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.5.m5.1c">k</annotation></semantics></math> is a parameter of the gate. Informally, <math id="S2.SS3.p1.6.m6.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS3.p1.6.m6.1a"><msup id="S2.SS3.p1.6.m6.1.1" xref="S2.SS3.p1.6.m6.1.1.cmml"><mi id="S2.SS3.p1.6.m6.1.1.2" xref="S2.SS3.p1.6.m6.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p1.6.m6.1.1.3" xref="S2.SS3.p1.6.m6.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.6.m6.1b"><apply id="S2.SS3.p1.6.m6.1.1.cmml" xref="S2.SS3.p1.6.m6.1.1"><csymbol cd="ambiguous" id="S2.SS3.p1.6.m6.1.1.1.cmml" xref="S2.SS3.p1.6.m6.1.1">superscript</csymbol><ci id="S2.SS3.p1.6.m6.1.1.2.cmml" xref="S2.SS3.p1.6.m6.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p1.6.m6.1.1.3.cmml" xref="S2.SS3.p1.6.m6.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.6.m6.1c">\mathsf{TC}^{0}</annotation></semantics></math> can be thought of as the class of problems that can be solved with extremely parallelized (constant-depth) computation.<span id="footnote3" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">3</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">3</sup><span class="ltx_tag ltx_tag_note">3</span>We use <math id="footnote3.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="footnote3.m1.1b"><msup id="footnote3.m1.1.1" xref="footnote3.m1.1.1.cmml"><mi id="footnote3.m1.1.1.2" xref="footnote3.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="footnote3.m1.1.1.3" xref="footnote3.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="footnote3.m1.1c"><apply id="footnote3.m1.1.1.cmml" xref="footnote3.m1.1.1"><csymbol cd="ambiguous" id="footnote3.m1.1.1.1.cmml" xref="footnote3.m1.1.1">superscript</csymbol><ci id="footnote3.m1.1.1.2.cmml" xref="footnote3.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="footnote3.m1.1.1.3.cmml" xref="footnote3.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m1.1d">\mathsf{TC}^{0}</annotation></semantics></math> to mean <math id="footnote3.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="footnote3.m2.1b"><mi id="footnote3.m2.1.1" xref="footnote3.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="footnote3.m2.1c"><ci id="footnote3.m2.1.1.cmml" xref="footnote3.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m2.1d">\mathsf{L}</annotation></semantics></math>-uniform <math id="footnote3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="footnote3.m3.1b"><msup id="footnote3.m3.1.1" xref="footnote3.m3.1.1.cmml"><mi id="footnote3.m3.1.1.2" xref="footnote3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="footnote3.m3.1.1.3" xref="footnote3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="footnote3.m3.1c"><apply id="footnote3.m3.1.1.cmml" xref="footnote3.m3.1.1"><csymbol cd="ambiguous" id="footnote3.m3.1.1.1.cmml" xref="footnote3.m3.1.1">superscript</csymbol><ci id="footnote3.m3.1.1.2.cmml" xref="footnote3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="footnote3.m3.1.1.3.cmml" xref="footnote3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m3.1d">\mathsf{TC}^{0}</annotation></semantics></math>, meaning the circuit family is constructible by a Turing machine that runs in space logarithmic in the size of the input <cite class="ltx_cite ltx_citemacro_citep">(cf. Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>; Strobl et al., <a href="#bib.bib30" title="" class="ltx_ref">2023</a>)</cite>.
We believe our results could be extended from <math id="footnote3.m4.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="footnote3.m4.1b"><mi id="footnote3.m4.1.1" xref="footnote3.m4.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="footnote3.m4.1c"><ci id="footnote3.m4.1.1.cmml" xref="footnote3.m4.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m4.1d">\mathsf{L}</annotation></semantics></math>-uniform <math id="footnote3.m5.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="footnote3.m5.1b"><msup id="footnote3.m5.1.1" xref="footnote3.m5.1.1.cmml"><mi id="footnote3.m5.1.1.2" xref="footnote3.m5.1.1.2.cmml">𝖳𝖢</mi><mn id="footnote3.m5.1.1.3" xref="footnote3.m5.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="footnote3.m5.1c"><apply id="footnote3.m5.1.1.cmml" xref="footnote3.m5.1.1"><csymbol cd="ambiguous" id="footnote3.m5.1.1.1.cmml" xref="footnote3.m5.1.1">superscript</csymbol><ci id="footnote3.m5.1.1.2.cmml" xref="footnote3.m5.1.1.2">𝖳𝖢</ci><cn type="integer" id="footnote3.m5.1.1.3.cmml" xref="footnote3.m5.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m5.1d">\mathsf{TC}^{0}</annotation></semantics></math> to <math id="footnote3.m6.1" class="ltx_Math" alttext="\mathsf{DLOGTIME}" display="inline"><semantics id="footnote3.m6.1b"><mi id="footnote3.m6.1.1" xref="footnote3.m6.1.1.cmml">𝖣𝖫𝖮𝖦𝖳𝖨𝖬𝖤</mi><annotation-xml encoding="MathML-Content" id="footnote3.m6.1c"><ci id="footnote3.m6.1.1.cmml" xref="footnote3.m6.1.1">𝖣𝖫𝖮𝖦𝖳𝖨𝖬𝖤</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m6.1d">\mathsf{DLOGTIME}</annotation></semantics></math>-uniform <math id="footnote3.m7.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="footnote3.m7.1b"><msup id="footnote3.m7.1.1" xref="footnote3.m7.1.1.cmml"><mi id="footnote3.m7.1.1.2" xref="footnote3.m7.1.1.2.cmml">𝖳𝖢</mi><mn id="footnote3.m7.1.1.3" xref="footnote3.m7.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="footnote3.m7.1c"><apply id="footnote3.m7.1.1.cmml" xref="footnote3.m7.1.1"><csymbol cd="ambiguous" id="footnote3.m7.1.1.1.cmml" xref="footnote3.m7.1.1">superscript</csymbol><ci id="footnote3.m7.1.1.2.cmml" xref="footnote3.m7.1.1.2">𝖳𝖢</ci><cn type="integer" id="footnote3.m7.1.1.3.cmml" xref="footnote3.m7.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m7.1d">\mathsf{TC}^{0}</annotation></semantics></math> using techniques similar to <cite class="ltx_cite ltx_citemacro_citet">Merrill &amp; Sabharwal (<a href="#bib.bib26" title="" class="ltx_ref">2023b</a>)</cite> for composing <math id="footnote3.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="footnote3.m8.1b"><msup id="footnote3.m8.1.1" xref="footnote3.m8.1.1.cmml"><mi id="footnote3.m8.1.1.2" xref="footnote3.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="footnote3.m8.1.1.3" xref="footnote3.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="footnote3.m8.1c"><apply id="footnote3.m8.1.1.cmml" xref="footnote3.m8.1.1"><csymbol cd="ambiguous" id="footnote3.m8.1.1.1.cmml" xref="footnote3.m8.1.1">superscript</csymbol><ci id="footnote3.m8.1.1.2.cmml" xref="footnote3.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="footnote3.m8.1.1.3.cmml" xref="footnote3.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m8.1d">\mathsf{TC}^{0}</annotation></semantics></math> circuits in a way that preserves <math id="footnote3.m9.1" class="ltx_Math" alttext="\mathsf{DLOGTIME}" display="inline"><semantics id="footnote3.m9.1b"><mi id="footnote3.m9.1.1" xref="footnote3.m9.1.1.cmml">𝖣𝖫𝖮𝖦𝖳𝖨𝖬𝖤</mi><annotation-xml encoding="MathML-Content" id="footnote3.m9.1c"><ci id="footnote3.m9.1.1.cmml" xref="footnote3.m9.1.1">𝖣𝖫𝖮𝖦𝖳𝖨𝖬𝖤</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote3.m9.1d">\mathsf{DLOGTIME}</annotation></semantics></math> uniformity.
</span></span></span></p>
</div>
<div id="S2.SS3.p2" class="ltx_para">
<p id="S2.SS3.p2.10" class="ltx_p">Problems outside <math id="S2.SS3.p2.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS3.p2.1.m1.1a"><msup id="S2.SS3.p2.1.m1.1.1" xref="S2.SS3.p2.1.m1.1.1.cmml"><mi id="S2.SS3.p2.1.m1.1.1.2" xref="S2.SS3.p2.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p2.1.m1.1.1.3" xref="S2.SS3.p2.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.1.m1.1b"><apply id="S2.SS3.p2.1.m1.1.1.cmml" xref="S2.SS3.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.1.m1.1.1.1.cmml" xref="S2.SS3.p2.1.m1.1.1">superscript</csymbol><ci id="S2.SS3.p2.1.m1.1.1.2.cmml" xref="S2.SS3.p2.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p2.1.m1.1.1.3.cmml" xref="S2.SS3.p2.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>, corresponding to problems that are inherently sequential and thus cannot be parallelized, cannot be solved by transformers. No problems in polynomial time are known unconditionally to be outside <math id="S2.SS3.p2.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS3.p2.2.m2.1a"><msup id="S2.SS3.p2.2.m2.1.1" xref="S2.SS3.p2.2.m2.1.1.cmml"><mi id="S2.SS3.p2.2.m2.1.1.2" xref="S2.SS3.p2.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p2.2.m2.1.1.3" xref="S2.SS3.p2.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.2.m2.1b"><apply id="S2.SS3.p2.2.m2.1.1.cmml" xref="S2.SS3.p2.2.m2.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.2.m2.1.1.1.cmml" xref="S2.SS3.p2.2.m2.1.1">superscript</csymbol><ci id="S2.SS3.p2.2.m2.1.1.2.cmml" xref="S2.SS3.p2.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p2.2.m2.1.1.3.cmml" xref="S2.SS3.p2.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math>, but unless the widely held conjecture that <math id="S2.SS3.p2.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}\neq\mathsf{NC}^{1}" display="inline"><semantics id="S2.SS3.p2.3.m3.1a"><mrow id="S2.SS3.p2.3.m3.1.1" xref="S2.SS3.p2.3.m3.1.1.cmml"><msup id="S2.SS3.p2.3.m3.1.1.2" xref="S2.SS3.p2.3.m3.1.1.2.cmml"><mi id="S2.SS3.p2.3.m3.1.1.2.2" xref="S2.SS3.p2.3.m3.1.1.2.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p2.3.m3.1.1.2.3" xref="S2.SS3.p2.3.m3.1.1.2.3.cmml">0</mn></msup><mo id="S2.SS3.p2.3.m3.1.1.1" xref="S2.SS3.p2.3.m3.1.1.1.cmml">≠</mo><msup id="S2.SS3.p2.3.m3.1.1.3" xref="S2.SS3.p2.3.m3.1.1.3.cmml"><mi id="S2.SS3.p2.3.m3.1.1.3.2" xref="S2.SS3.p2.3.m3.1.1.3.2.cmml">𝖭𝖢</mi><mn id="S2.SS3.p2.3.m3.1.1.3.3" xref="S2.SS3.p2.3.m3.1.1.3.3.cmml">1</mn></msup></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.3.m3.1b"><apply id="S2.SS3.p2.3.m3.1.1.cmml" xref="S2.SS3.p2.3.m3.1.1"><neq id="S2.SS3.p2.3.m3.1.1.1.cmml" xref="S2.SS3.p2.3.m3.1.1.1"></neq><apply id="S2.SS3.p2.3.m3.1.1.2.cmml" xref="S2.SS3.p2.3.m3.1.1.2"><csymbol cd="ambiguous" id="S2.SS3.p2.3.m3.1.1.2.1.cmml" xref="S2.SS3.p2.3.m3.1.1.2">superscript</csymbol><ci id="S2.SS3.p2.3.m3.1.1.2.2.cmml" xref="S2.SS3.p2.3.m3.1.1.2.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p2.3.m3.1.1.2.3.cmml" xref="S2.SS3.p2.3.m3.1.1.2.3">0</cn></apply><apply id="S2.SS3.p2.3.m3.1.1.3.cmml" xref="S2.SS3.p2.3.m3.1.1.3"><csymbol cd="ambiguous" id="S2.SS3.p2.3.m3.1.1.3.1.cmml" xref="S2.SS3.p2.3.m3.1.1.3">superscript</csymbol><ci id="S2.SS3.p2.3.m3.1.1.3.2.cmml" xref="S2.SS3.p2.3.m3.1.1.3.2">𝖭𝖢</ci><cn type="integer" id="S2.SS3.p2.3.m3.1.1.3.3.cmml" xref="S2.SS3.p2.3.m3.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.3.m3.1c">\mathsf{TC}^{0}\neq\mathsf{NC}^{1}</annotation></semantics></math> is false, many simple <math id="S2.SS3.p2.4.m4.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S2.SS3.p2.4.m4.1a"><msup id="S2.SS3.p2.4.m4.1.1" xref="S2.SS3.p2.4.m4.1.1.cmml"><mi id="S2.SS3.p2.4.m4.1.1.2" xref="S2.SS3.p2.4.m4.1.1.2.cmml">𝖭𝖢</mi><mn id="S2.SS3.p2.4.m4.1.1.3" xref="S2.SS3.p2.4.m4.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.4.m4.1b"><apply id="S2.SS3.p2.4.m4.1.1.cmml" xref="S2.SS3.p2.4.m4.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.4.m4.1.1.1.cmml" xref="S2.SS3.p2.4.m4.1.1">superscript</csymbol><ci id="S2.SS3.p2.4.m4.1.1.2.cmml" xref="S2.SS3.p2.4.m4.1.1.2">𝖭𝖢</ci><cn type="integer" id="S2.SS3.p2.4.m4.1.1.3.cmml" xref="S2.SS3.p2.4.m4.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.4.m4.1c">\mathsf{NC}^{1}</annotation></semantics></math>-hard problems are outside <math id="S2.SS3.p2.5.m5.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS3.p2.5.m5.1a"><msup id="S2.SS3.p2.5.m5.1.1" xref="S2.SS3.p2.5.m5.1.1.cmml"><mi id="S2.SS3.p2.5.m5.1.1.2" xref="S2.SS3.p2.5.m5.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p2.5.m5.1.1.3" xref="S2.SS3.p2.5.m5.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.5.m5.1b"><apply id="S2.SS3.p2.5.m5.1.1.cmml" xref="S2.SS3.p2.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.5.m5.1.1.1.cmml" xref="S2.SS3.p2.5.m5.1.1">superscript</csymbol><ci id="S2.SS3.p2.5.m5.1.1.2.cmml" xref="S2.SS3.p2.5.m5.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p2.5.m5.1.1.3.cmml" xref="S2.SS3.p2.5.m5.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.5.m5.1c">\mathsf{TC}^{0}</annotation></semantics></math>. In particular, this includes simulating finite automata (<math id="S2.SS3.p2.6.m6.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S2.SS3.p2.6.m6.1a"><msup id="S2.SS3.p2.6.m6.1.1" xref="S2.SS3.p2.6.m6.1.1.cmml"><mi id="S2.SS3.p2.6.m6.1.1.2" xref="S2.SS3.p2.6.m6.1.1.2.cmml">𝖭𝖢</mi><mn id="S2.SS3.p2.6.m6.1.1.3" xref="S2.SS3.p2.6.m6.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.6.m6.1b"><apply id="S2.SS3.p2.6.m6.1.1.cmml" xref="S2.SS3.p2.6.m6.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.6.m6.1.1.1.cmml" xref="S2.SS3.p2.6.m6.1.1">superscript</csymbol><ci id="S2.SS3.p2.6.m6.1.1.2.cmml" xref="S2.SS3.p2.6.m6.1.1.2">𝖭𝖢</ci><cn type="integer" id="S2.SS3.p2.6.m6.1.1.3.cmml" xref="S2.SS3.p2.6.m6.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.6.m6.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete), evaluating boolean formulas (<math id="S2.SS3.p2.7.m7.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S2.SS3.p2.7.m7.1a"><msup id="S2.SS3.p2.7.m7.1.1" xref="S2.SS3.p2.7.m7.1.1.cmml"><mi id="S2.SS3.p2.7.m7.1.1.2" xref="S2.SS3.p2.7.m7.1.1.2.cmml">𝖭𝖢</mi><mn id="S2.SS3.p2.7.m7.1.1.3" xref="S2.SS3.p2.7.m7.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.7.m7.1b"><apply id="S2.SS3.p2.7.m7.1.1.cmml" xref="S2.SS3.p2.7.m7.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.7.m7.1.1.1.cmml" xref="S2.SS3.p2.7.m7.1.1">superscript</csymbol><ci id="S2.SS3.p2.7.m7.1.1.2.cmml" xref="S2.SS3.p2.7.m7.1.1.2">𝖭𝖢</ci><cn type="integer" id="S2.SS3.p2.7.m7.1.1.3.cmml" xref="S2.SS3.p2.7.m7.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.7.m7.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete), determining graph connectivity (<math id="S2.SS3.p2.8.m8.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S2.SS3.p2.8.m8.1a"><mi id="S2.SS3.p2.8.m8.1.1" xref="S2.SS3.p2.8.m8.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.8.m8.1b"><ci id="S2.SS3.p2.8.m8.1.1.cmml" xref="S2.SS3.p2.8.m8.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.8.m8.1c">\mathsf{L}</annotation></semantics></math>-complete), and solving linear equations (<math id="S2.SS3.p2.9.m9.1" class="ltx_Math" alttext="\mathsf{P}" display="inline"><semantics id="S2.SS3.p2.9.m9.1a"><mi id="S2.SS3.p2.9.m9.1.1" xref="S2.SS3.p2.9.m9.1.1.cmml">𝖯</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.9.m9.1b"><ci id="S2.SS3.p2.9.m9.1.1.cmml" xref="S2.SS3.p2.9.m9.1.1">𝖯</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.9.m9.1c">\mathsf{P}</annotation></semantics></math>-complete). These problems have already been shown to be inexpressible by transformers <cite class="ltx_cite ltx_citemacro_citep">(Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>. By showing that SSMs can be simulated in <math id="S2.SS3.p2.10.m10.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S2.SS3.p2.10.m10.1a"><msup id="S2.SS3.p2.10.m10.1.1" xref="S2.SS3.p2.10.m10.1.1.cmml"><mi id="S2.SS3.p2.10.m10.1.1.2" xref="S2.SS3.p2.10.m10.1.1.2.cmml">𝖳𝖢</mi><mn id="S2.SS3.p2.10.m10.1.1.3" xref="S2.SS3.p2.10.m10.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.10.m10.1b"><apply id="S2.SS3.p2.10.m10.1.1.cmml" xref="S2.SS3.p2.10.m10.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.10.m10.1.1.1.cmml" xref="S2.SS3.p2.10.m10.1.1">superscript</csymbol><ci id="S2.SS3.p2.10.m10.1.1.2.cmml" xref="S2.SS3.p2.10.m10.1.1.2">𝖳𝖢</ci><cn type="integer" id="S2.SS3.p2.10.m10.1.1.3.cmml" xref="S2.SS3.p2.10.m10.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.10.m10.1c">\mathsf{TC}^{0}</annotation></semantics></math>, we will establish that they also cannot be solved by SSMs.</p>
</div>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>State Tracking</h2>

<div id="S3.p1" class="ltx_para">
<p id="S3.p1.1" class="ltx_p">Informally, a state-tracking problem is a problem where the text specifies some sequence of updates to the state of the world, and the goal of the problem is to say what the resulting world state is after the updates have been applied in sequence.
This circuit complexity view on the power of neural networks can be combined with other insights from algebraic formal language theory to analyze the kinds of state tracking that SSMs can express.
In particular, this theory comprehensively shows us which kinds of state-tracking problems are (likely) not in <math id="S3.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S3.p1.1.m1.1a"><msup id="S3.p1.1.m1.1.1" xref="S3.p1.1.m1.1.1.cmml"><mi id="S3.p1.1.m1.1.1.2" xref="S3.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S3.p1.1.m1.1.1.3" xref="S3.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.p1.1.m1.1b"><apply id="S3.p1.1.m1.1.1.cmml" xref="S3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S3.p1.1.m1.1.1.1.cmml" xref="S3.p1.1.m1.1.1">superscript</csymbol><ci id="S3.p1.1.m1.1.1.2.cmml" xref="S3.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S3.p1.1.m1.1.1.3.cmml" xref="S3.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>. This will, in turn, allow us to find examples of hard state tracking that models like SSMs will not be able to solve.</p>
</div>
<section id="S3.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.1 </span>State Tracking as a Monoid Word Problem</h3>

<div id="S3.SS1.p1" class="ltx_para">
<p id="S3.SS1.p1.1" class="ltx_p">From the perspective of algebraic formal language theory, state tracking over a finite world can be captured as a <em id="S3.SS1.p1.1.1" class="ltx_emph ltx_font_italic">word problem</em> on a <em id="S3.SS1.p1.1.2" class="ltx_emph ltx_font_italic">finite monoid</em> <cite class="ltx_cite ltx_citemacro_citep">(Liu et al., <a href="#bib.bib21" title="" class="ltx_ref">2023</a>)</cite>.<span id="footnote4" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">4</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">4</sup><span class="ltx_tag ltx_tag_note">4</span>We consider only finite monoids for simplicity, but, in principle, it would be possible to extend this approach to infinite (e.g., finitely generated) monoids as well.</span></span></span>
Different updates to the world become different elements in the monoid, and resolving the final world state after all the updates have been applied is equivalent to computing the product of a sequence of elements (also called a “word”).</p>
</div>
<div id="S3.Thmtheorem1" class="ltx_theorem ltx_theorem_definition">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S3.Thmtheorem1.1.1.1" class="ltx_text ltx_font_bold">Definition 3.1</span></span><span id="S3.Thmtheorem1.2.2" class="ltx_text ltx_font_bold"> </span>(Word Problem)<span id="S3.Thmtheorem1.3.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S3.Thmtheorem1.p1" class="ltx_para">
<p id="S3.Thmtheorem1.p1.7" class="ltx_p">Let <math id="S3.Thmtheorem1.p1.1.m1.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S3.Thmtheorem1.p1.1.m1.1a"><mi id="S3.Thmtheorem1.p1.1.m1.1.1" xref="S3.Thmtheorem1.p1.1.m1.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.1.m1.1b"><ci id="S3.Thmtheorem1.p1.1.m1.1.1.cmml" xref="S3.Thmtheorem1.p1.1.m1.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.1.m1.1c">M</annotation></semantics></math> be a finite set, and <math id="S3.Thmtheorem1.p1.2.m2.2" class="ltx_Math" alttext="(M,\cdot)" display="inline"><semantics id="S3.Thmtheorem1.p1.2.m2.2a"><mrow id="S3.Thmtheorem1.p1.2.m2.2.3.2" xref="S3.Thmtheorem1.p1.2.m2.2.3.1.cmml"><mo stretchy="false" id="S3.Thmtheorem1.p1.2.m2.2.3.2.1" xref="S3.Thmtheorem1.p1.2.m2.2.3.1.cmml">(</mo><mi id="S3.Thmtheorem1.p1.2.m2.1.1" xref="S3.Thmtheorem1.p1.2.m2.1.1.cmml">M</mi><mo rspace="0em" id="S3.Thmtheorem1.p1.2.m2.2.3.2.2" xref="S3.Thmtheorem1.p1.2.m2.2.3.1.cmml">,</mo><mo lspace="0em" rspace="0em" id="S3.Thmtheorem1.p1.2.m2.2.2" xref="S3.Thmtheorem1.p1.2.m2.2.2.cmml">⋅</mo><mo stretchy="false" id="S3.Thmtheorem1.p1.2.m2.2.3.2.3" xref="S3.Thmtheorem1.p1.2.m2.2.3.1.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.2.m2.2b"><interval closure="open" id="S3.Thmtheorem1.p1.2.m2.2.3.1.cmml" xref="S3.Thmtheorem1.p1.2.m2.2.3.2"><ci id="S3.Thmtheorem1.p1.2.m2.1.1.cmml" xref="S3.Thmtheorem1.p1.2.m2.1.1">𝑀</ci><ci id="S3.Thmtheorem1.p1.2.m2.2.2.cmml" xref="S3.Thmtheorem1.p1.2.m2.2.2">⋅</ci></interval></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.2.m2.2c">(M,\cdot)</annotation></semantics></math> a finite monoid (i.e., <math id="S3.Thmtheorem1.p1.3.m3.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S3.Thmtheorem1.p1.3.m3.1a"><mi id="S3.Thmtheorem1.p1.3.m3.1.1" xref="S3.Thmtheorem1.p1.3.m3.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.3.m3.1b"><ci id="S3.Thmtheorem1.p1.3.m3.1.1.cmml" xref="S3.Thmtheorem1.p1.3.m3.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.3.m3.1c">M</annotation></semantics></math> with identity and associative multiplication). The word problem for <math id="S3.Thmtheorem1.p1.4.m4.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S3.Thmtheorem1.p1.4.m4.1a"><mi id="S3.Thmtheorem1.p1.4.m4.1.1" xref="S3.Thmtheorem1.p1.4.m4.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.4.m4.1b"><ci id="S3.Thmtheorem1.p1.4.m4.1.1.cmml" xref="S3.Thmtheorem1.p1.4.m4.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.4.m4.1c">M</annotation></semantics></math> is
to reduce sequences in <math id="S3.Thmtheorem1.p1.5.m5.1" class="ltx_Math" alttext="M^{*}" display="inline"><semantics id="S3.Thmtheorem1.p1.5.m5.1a"><msup id="S3.Thmtheorem1.p1.5.m5.1.1" xref="S3.Thmtheorem1.p1.5.m5.1.1.cmml"><mi id="S3.Thmtheorem1.p1.5.m5.1.1.2" xref="S3.Thmtheorem1.p1.5.m5.1.1.2.cmml">M</mi><mo id="S3.Thmtheorem1.p1.5.m5.1.1.3" xref="S3.Thmtheorem1.p1.5.m5.1.1.3.cmml">∗</mo></msup><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.5.m5.1b"><apply id="S3.Thmtheorem1.p1.5.m5.1.1.cmml" xref="S3.Thmtheorem1.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.5.m5.1.1.1.cmml" xref="S3.Thmtheorem1.p1.5.m5.1.1">superscript</csymbol><ci id="S3.Thmtheorem1.p1.5.m5.1.1.2.cmml" xref="S3.Thmtheorem1.p1.5.m5.1.1.2">𝑀</ci><times id="S3.Thmtheorem1.p1.5.m5.1.1.3.cmml" xref="S3.Thmtheorem1.p1.5.m5.1.1.3"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.5.m5.1c">M^{*}</annotation></semantics></math> under multiplication; that is, send <math id="S3.Thmtheorem1.p1.6.m6.1" class="ltx_Math" alttext="m_{0}m_{1}\cdots m_{k}" display="inline"><semantics id="S3.Thmtheorem1.p1.6.m6.1a"><mrow id="S3.Thmtheorem1.p1.6.m6.1.1" xref="S3.Thmtheorem1.p1.6.m6.1.1.cmml"><msub id="S3.Thmtheorem1.p1.6.m6.1.1.2" xref="S3.Thmtheorem1.p1.6.m6.1.1.2.cmml"><mi id="S3.Thmtheorem1.p1.6.m6.1.1.2.2" xref="S3.Thmtheorem1.p1.6.m6.1.1.2.2.cmml">m</mi><mn id="S3.Thmtheorem1.p1.6.m6.1.1.2.3" xref="S3.Thmtheorem1.p1.6.m6.1.1.2.3.cmml">0</mn></msub><mo lspace="0em" rspace="0em" id="S3.Thmtheorem1.p1.6.m6.1.1.1" xref="S3.Thmtheorem1.p1.6.m6.1.1.1.cmml">​</mo><msub id="S3.Thmtheorem1.p1.6.m6.1.1.3" xref="S3.Thmtheorem1.p1.6.m6.1.1.3.cmml"><mi id="S3.Thmtheorem1.p1.6.m6.1.1.3.2" xref="S3.Thmtheorem1.p1.6.m6.1.1.3.2.cmml">m</mi><mn id="S3.Thmtheorem1.p1.6.m6.1.1.3.3" xref="S3.Thmtheorem1.p1.6.m6.1.1.3.3.cmml">1</mn></msub><mo lspace="0em" rspace="0em" id="S3.Thmtheorem1.p1.6.m6.1.1.1a" xref="S3.Thmtheorem1.p1.6.m6.1.1.1.cmml">​</mo><mi mathvariant="normal" id="S3.Thmtheorem1.p1.6.m6.1.1.4" xref="S3.Thmtheorem1.p1.6.m6.1.1.4.cmml">⋯</mi><mo lspace="0em" rspace="0em" id="S3.Thmtheorem1.p1.6.m6.1.1.1b" xref="S3.Thmtheorem1.p1.6.m6.1.1.1.cmml">​</mo><msub id="S3.Thmtheorem1.p1.6.m6.1.1.5" xref="S3.Thmtheorem1.p1.6.m6.1.1.5.cmml"><mi id="S3.Thmtheorem1.p1.6.m6.1.1.5.2" xref="S3.Thmtheorem1.p1.6.m6.1.1.5.2.cmml">m</mi><mi id="S3.Thmtheorem1.p1.6.m6.1.1.5.3" xref="S3.Thmtheorem1.p1.6.m6.1.1.5.3.cmml">k</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.6.m6.1b"><apply id="S3.Thmtheorem1.p1.6.m6.1.1.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1"><times id="S3.Thmtheorem1.p1.6.m6.1.1.1.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.1"></times><apply id="S3.Thmtheorem1.p1.6.m6.1.1.2.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.2"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.6.m6.1.1.2.1.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.2">subscript</csymbol><ci id="S3.Thmtheorem1.p1.6.m6.1.1.2.2.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.2.2">𝑚</ci><cn type="integer" id="S3.Thmtheorem1.p1.6.m6.1.1.2.3.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.2.3">0</cn></apply><apply id="S3.Thmtheorem1.p1.6.m6.1.1.3.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.3"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.6.m6.1.1.3.1.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.3">subscript</csymbol><ci id="S3.Thmtheorem1.p1.6.m6.1.1.3.2.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.3.2">𝑚</ci><cn type="integer" id="S3.Thmtheorem1.p1.6.m6.1.1.3.3.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.3.3">1</cn></apply><ci id="S3.Thmtheorem1.p1.6.m6.1.1.4.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.4">⋯</ci><apply id="S3.Thmtheorem1.p1.6.m6.1.1.5.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.5"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.6.m6.1.1.5.1.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.5">subscript</csymbol><ci id="S3.Thmtheorem1.p1.6.m6.1.1.5.2.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.5.2">𝑚</ci><ci id="S3.Thmtheorem1.p1.6.m6.1.1.5.3.cmml" xref="S3.Thmtheorem1.p1.6.m6.1.1.5.3">𝑘</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.6.m6.1c">m_{0}m_{1}\cdots m_{k}</annotation></semantics></math> to <math id="S3.Thmtheorem1.p1.7.m7.1" class="ltx_Math" alttext="m_{0}\cdot m_{1}\cdot\ldots\cdot m_{k}\in M" display="inline"><semantics id="S3.Thmtheorem1.p1.7.m7.1a"><mrow id="S3.Thmtheorem1.p1.7.m7.1.1" xref="S3.Thmtheorem1.p1.7.m7.1.1.cmml"><mrow id="S3.Thmtheorem1.p1.7.m7.1.1.2" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.cmml"><msub id="S3.Thmtheorem1.p1.7.m7.1.1.2.2" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2.cmml"><mi id="S3.Thmtheorem1.p1.7.m7.1.1.2.2.2" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2.2.cmml">m</mi><mn id="S3.Thmtheorem1.p1.7.m7.1.1.2.2.3" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2.3.cmml">0</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S3.Thmtheorem1.p1.7.m7.1.1.2.1" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.1.cmml">⋅</mo><msub id="S3.Thmtheorem1.p1.7.m7.1.1.2.3" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3.cmml"><mi id="S3.Thmtheorem1.p1.7.m7.1.1.2.3.2" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3.2.cmml">m</mi><mn id="S3.Thmtheorem1.p1.7.m7.1.1.2.3.3" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3.3.cmml">1</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S3.Thmtheorem1.p1.7.m7.1.1.2.1a" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.1.cmml">⋅</mo><mi mathvariant="normal" id="S3.Thmtheorem1.p1.7.m7.1.1.2.4" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.4.cmml">…</mi><mo lspace="0.222em" rspace="0.222em" id="S3.Thmtheorem1.p1.7.m7.1.1.2.1b" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.1.cmml">⋅</mo><msub id="S3.Thmtheorem1.p1.7.m7.1.1.2.5" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5.cmml"><mi id="S3.Thmtheorem1.p1.7.m7.1.1.2.5.2" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5.2.cmml">m</mi><mi id="S3.Thmtheorem1.p1.7.m7.1.1.2.5.3" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5.3.cmml">k</mi></msub></mrow><mo id="S3.Thmtheorem1.p1.7.m7.1.1.1" xref="S3.Thmtheorem1.p1.7.m7.1.1.1.cmml">∈</mo><mi id="S3.Thmtheorem1.p1.7.m7.1.1.3" xref="S3.Thmtheorem1.p1.7.m7.1.1.3.cmml">M</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem1.p1.7.m7.1b"><apply id="S3.Thmtheorem1.p1.7.m7.1.1.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1"><in id="S3.Thmtheorem1.p1.7.m7.1.1.1.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.1"></in><apply id="S3.Thmtheorem1.p1.7.m7.1.1.2.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2"><ci id="S3.Thmtheorem1.p1.7.m7.1.1.2.1.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.1">⋅</ci><apply id="S3.Thmtheorem1.p1.7.m7.1.1.2.2.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.7.m7.1.1.2.2.1.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2">subscript</csymbol><ci id="S3.Thmtheorem1.p1.7.m7.1.1.2.2.2.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2.2">𝑚</ci><cn type="integer" id="S3.Thmtheorem1.p1.7.m7.1.1.2.2.3.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.2.3">0</cn></apply><apply id="S3.Thmtheorem1.p1.7.m7.1.1.2.3.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.7.m7.1.1.2.3.1.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3">subscript</csymbol><ci id="S3.Thmtheorem1.p1.7.m7.1.1.2.3.2.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3.2">𝑚</ci><cn type="integer" id="S3.Thmtheorem1.p1.7.m7.1.1.2.3.3.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.3.3">1</cn></apply><ci id="S3.Thmtheorem1.p1.7.m7.1.1.2.4.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.4">…</ci><apply id="S3.Thmtheorem1.p1.7.m7.1.1.2.5.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5"><csymbol cd="ambiguous" id="S3.Thmtheorem1.p1.7.m7.1.1.2.5.1.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5">subscript</csymbol><ci id="S3.Thmtheorem1.p1.7.m7.1.1.2.5.2.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5.2">𝑚</ci><ci id="S3.Thmtheorem1.p1.7.m7.1.1.2.5.3.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.2.5.3">𝑘</ci></apply></apply><ci id="S3.Thmtheorem1.p1.7.m7.1.1.3.cmml" xref="S3.Thmtheorem1.p1.7.m7.1.1.3">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem1.p1.7.m7.1c">m_{0}\cdot m_{1}\cdot\ldots\cdot m_{k}\in M</annotation></semantics></math>. Solving the word problem requires reducing sequences of arbitrary length.</p>
</div>
</div>
<div id="S3.Thmtheorem2" class="ltx_theorem ltx_theorem_example">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S3.Thmtheorem2.1.1.1" class="ltx_text ltx_font_italic">Example 3.2</span></span><span id="S3.Thmtheorem2.2.2" class="ltx_text ltx_font_italic">.</span>
</h6>
<div id="S3.Thmtheorem2.p1" class="ltx_para">
<p id="S3.Thmtheorem2.p1.5" class="ltx_p">Consider the monoid <math id="S3.Thmtheorem2.p1.1.m1.2" class="ltx_Math" alttext="\{0,1\}" display="inline"><semantics id="S3.Thmtheorem2.p1.1.m1.2a"><mrow id="S3.Thmtheorem2.p1.1.m1.2.3.2" xref="S3.Thmtheorem2.p1.1.m1.2.3.1.cmml"><mo stretchy="false" id="S3.Thmtheorem2.p1.1.m1.2.3.2.1" xref="S3.Thmtheorem2.p1.1.m1.2.3.1.cmml">{</mo><mn id="S3.Thmtheorem2.p1.1.m1.1.1" xref="S3.Thmtheorem2.p1.1.m1.1.1.cmml">0</mn><mo id="S3.Thmtheorem2.p1.1.m1.2.3.2.2" xref="S3.Thmtheorem2.p1.1.m1.2.3.1.cmml">,</mo><mn id="S3.Thmtheorem2.p1.1.m1.2.2" xref="S3.Thmtheorem2.p1.1.m1.2.2.cmml">1</mn><mo stretchy="false" id="S3.Thmtheorem2.p1.1.m1.2.3.2.3" xref="S3.Thmtheorem2.p1.1.m1.2.3.1.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem2.p1.1.m1.2b"><set id="S3.Thmtheorem2.p1.1.m1.2.3.1.cmml" xref="S3.Thmtheorem2.p1.1.m1.2.3.2"><cn type="integer" id="S3.Thmtheorem2.p1.1.m1.1.1.cmml" xref="S3.Thmtheorem2.p1.1.m1.1.1">0</cn><cn type="integer" id="S3.Thmtheorem2.p1.1.m1.2.2.cmml" xref="S3.Thmtheorem2.p1.1.m1.2.2">1</cn></set></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem2.p1.1.m1.2c">\{0,1\}</annotation></semantics></math> where <math id="S3.Thmtheorem2.p1.2.m2.1" class="ltx_Math" alttext="\cdot" display="inline"><semantics id="S3.Thmtheorem2.p1.2.m2.1a"><mo id="S3.Thmtheorem2.p1.2.m2.1.1" xref="S3.Thmtheorem2.p1.2.m2.1.1.cmml">⋅</mo><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem2.p1.2.m2.1b"><ci id="S3.Thmtheorem2.p1.2.m2.1.1.cmml" xref="S3.Thmtheorem2.p1.2.m2.1.1">⋅</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem2.p1.2.m2.1c">\cdot</annotation></semantics></math> is addition modulo 2. The word problem involves computing the parity of a string, e.g., <math id="S3.Thmtheorem2.p1.3.m3.1" class="ltx_Math" alttext="0011\mapsto 0" display="inline"><semantics id="S3.Thmtheorem2.p1.3.m3.1a"><mrow id="S3.Thmtheorem2.p1.3.m3.1.1" xref="S3.Thmtheorem2.p1.3.m3.1.1.cmml"><mn id="S3.Thmtheorem2.p1.3.m3.1.1.2" xref="S3.Thmtheorem2.p1.3.m3.1.1.2.cmml">0011</mn><mo stretchy="false" id="S3.Thmtheorem2.p1.3.m3.1.1.1" xref="S3.Thmtheorem2.p1.3.m3.1.1.1.cmml">↦</mo><mn id="S3.Thmtheorem2.p1.3.m3.1.1.3" xref="S3.Thmtheorem2.p1.3.m3.1.1.3.cmml">0</mn></mrow><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem2.p1.3.m3.1b"><apply id="S3.Thmtheorem2.p1.3.m3.1.1.cmml" xref="S3.Thmtheorem2.p1.3.m3.1.1"><csymbol cd="latexml" id="S3.Thmtheorem2.p1.3.m3.1.1.1.cmml" xref="S3.Thmtheorem2.p1.3.m3.1.1.1">maps-to</csymbol><cn type="integer" id="S3.Thmtheorem2.p1.3.m3.1.1.2.cmml" xref="S3.Thmtheorem2.p1.3.m3.1.1.2">0011</cn><cn type="integer" id="S3.Thmtheorem2.p1.3.m3.1.1.3.cmml" xref="S3.Thmtheorem2.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem2.p1.3.m3.1c">0011\mapsto 0</annotation></semantics></math>. From a state-tracking perspective, this monoid captures a world with a single light switch. The identity <math id="S3.Thmtheorem2.p1.4.m4.1" class="ltx_Math" alttext="0" display="inline"><semantics id="S3.Thmtheorem2.p1.4.m4.1a"><mn id="S3.Thmtheorem2.p1.4.m4.1.1" xref="S3.Thmtheorem2.p1.4.m4.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem2.p1.4.m4.1b"><cn type="integer" id="S3.Thmtheorem2.p1.4.m4.1.1.cmml" xref="S3.Thmtheorem2.p1.4.m4.1.1">0</cn></annotation-xml></semantics></math> corresponds to no action whereas <math id="S3.Thmtheorem2.p1.5.m5.1" class="ltx_Math" alttext="1" display="inline"><semantics id="S3.Thmtheorem2.p1.5.m5.1a"><mn id="S3.Thmtheorem2.p1.5.m5.1.1" xref="S3.Thmtheorem2.p1.5.m5.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem2.p1.5.m5.1b"><cn type="integer" id="S3.Thmtheorem2.p1.5.m5.1.1.cmml" xref="S3.Thmtheorem2.p1.5.m5.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem2.p1.5.m5.1c">1</annotation></semantics></math> is an update that flips the switch.</p>
</div>
</div>
<div id="S3.SS1.p2" class="ltx_para">
<p id="S3.SS1.p2.4" class="ltx_p">Modeling state tracking with word problems lets us draw connections between circuit complexity and abstract algebra to understand which word problems are “hard” to solve. <cite class="ltx_cite ltx_citemacro_citet">Krohn &amp; Rhodes (<a href="#bib.bib20" title="" class="ltx_ref">1965</a>)</cite> established that not all word problems are created equal: some, like <a href="#S3.Thmtheorem2" title="Example 3.2. ‣ 3.1 State Tracking as a Monoid Word Problem ‣ 3 State Tracking ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Example</span> <span class="ltx_text ltx_ref_tag">3.2</span></a>, are in <math id="S3.SS1.p2.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S3.SS1.p2.1.m1.1a"><msup id="S3.SS1.p2.1.m1.1.1" xref="S3.SS1.p2.1.m1.1.1.cmml"><mi id="S3.SS1.p2.1.m1.1.1.2" xref="S3.SS1.p2.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S3.SS1.p2.1.m1.1.1.3" xref="S3.SS1.p2.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.1.m1.1b"><apply id="S3.SS1.p2.1.m1.1.1.cmml" xref="S3.SS1.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.1.m1.1.1.1.cmml" xref="S3.SS1.p2.1.m1.1.1">superscript</csymbol><ci id="S3.SS1.p2.1.m1.1.1.2.cmml" xref="S3.SS1.p2.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S3.SS1.p2.1.m1.1.1.3.cmml" xref="S3.SS1.p2.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>, while others are <math id="S3.SS1.p2.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS1.p2.2.m2.1a"><msup id="S3.SS1.p2.2.m2.1.1" xref="S3.SS1.p2.2.m2.1.1.cmml"><mi id="S3.SS1.p2.2.m2.1.1.2" xref="S3.SS1.p2.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS1.p2.2.m2.1.1.3" xref="S3.SS1.p2.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.2.m2.1b"><apply id="S3.SS1.p2.2.m2.1.1.cmml" xref="S3.SS1.p2.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.2.m2.1.1.1.cmml" xref="S3.SS1.p2.2.m2.1.1">superscript</csymbol><ci id="S3.SS1.p2.2.m2.1.1.2.cmml" xref="S3.SS1.p2.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS1.p2.2.m2.1.1.3.cmml" xref="S3.SS1.p2.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete, requiring recurrent processing to solve <cite class="ltx_cite ltx_citemacro_citep">(Immerman &amp; Landau, <a href="#bib.bib18" title="" class="ltx_ref">1989</a>; Barrington, <a href="#bib.bib2" title="" class="ltx_ref">1989</a>)</cite>. Because we will show SSMs can be simulated in <math id="S3.SS1.p2.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S3.SS1.p2.3.m3.1a"><msup id="S3.SS1.p2.3.m3.1.1" xref="S3.SS1.p2.3.m3.1.1.cmml"><mi id="S3.SS1.p2.3.m3.1.1.2" xref="S3.SS1.p2.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S3.SS1.p2.3.m3.1.1.3" xref="S3.SS1.p2.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.3.m3.1b"><apply id="S3.SS1.p2.3.m3.1.1.cmml" xref="S3.SS1.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.3.m3.1.1.1.cmml" xref="S3.SS1.p2.3.m3.1.1">superscript</csymbol><ci id="S3.SS1.p2.3.m3.1.1.2.cmml" xref="S3.SS1.p2.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S3.SS1.p2.3.m3.1.1.3.cmml" xref="S3.SS1.p2.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math>, it follows that <math id="S3.SS1.p2.4.m4.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS1.p2.4.m4.1a"><msup id="S3.SS1.p2.4.m4.1.1" xref="S3.SS1.p2.4.m4.1.1.cmml"><mi id="S3.SS1.p2.4.m4.1.1.2" xref="S3.SS1.p2.4.m4.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS1.p2.4.m4.1.1.3" xref="S3.SS1.p2.4.m4.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.4.m4.1b"><apply id="S3.SS1.p2.4.m4.1.1.cmml" xref="S3.SS1.p2.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS1.p2.4.m4.1.1.1.cmml" xref="S3.SS1.p2.4.m4.1.1">superscript</csymbol><ci id="S3.SS1.p2.4.m4.1.1.2.cmml" xref="S3.SS1.p2.4.m4.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS1.p2.4.m4.1.1.3.cmml" xref="S3.SS1.p2.4.m4.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.4.m4.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete state-tracking problems cannot be solved by SSMs (cf. <a href="#S1.F2" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">2</span></a>).</p>
</div>
<div id="S3.SS1.p3" class="ltx_para">
<p id="S3.SS1.p3.8" class="ltx_p">Whether or not a word problem is <math id="S3.SS1.p3.1.m1.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS1.p3.1.m1.1a"><msup id="S3.SS1.p3.1.m1.1.1" xref="S3.SS1.p3.1.m1.1.1.cmml"><mi id="S3.SS1.p3.1.m1.1.1.2" xref="S3.SS1.p3.1.m1.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS1.p3.1.m1.1.1.3" xref="S3.SS1.p3.1.m1.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.1.m1.1b"><apply id="S3.SS1.p3.1.m1.1.1.cmml" xref="S3.SS1.p3.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.1.m1.1.1.1.cmml" xref="S3.SS1.p3.1.m1.1.1">superscript</csymbol><ci id="S3.SS1.p3.1.m1.1.1.2.cmml" xref="S3.SS1.p3.1.m1.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS1.p3.1.m1.1.1.3.cmml" xref="S3.SS1.p3.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.1.m1.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete depends on the algebraic structure of the underlying monoid.<span id="footnote5" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">5</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">5</sup><span class="ltx_tag ltx_tag_note">5</span>We focus on word problems on <em id="footnote5.1" class="ltx_emph ltx_font_italic">groups</em>, which are monoids with inverses.</span></span></span> <cite class="ltx_cite ltx_citemacro_citet">Barrington (<a href="#bib.bib2" title="" class="ltx_ref">1989</a>)</cite> showed that the word problem of every finite non-solvable<span id="footnote6" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">6</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">6</sup><span class="ltx_tag ltx_tag_note">6</span>Formally, a group <math id="footnote6.m1.1" class="ltx_Math" alttext="G" display="inline"><semantics id="footnote6.m1.1b"><mi id="footnote6.m1.1.1" xref="footnote6.m1.1.1.cmml">G</mi><annotation-xml encoding="MathML-Content" id="footnote6.m1.1c"><ci id="footnote6.m1.1.1.cmml" xref="footnote6.m1.1.1">𝐺</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote6.m1.1d">G</annotation></semantics></math> is solvable exactly when there is a series of subgroups <math id="footnote6.m2.1" class="ltx_Math" alttext="1=G_{0}&lt;G_{1}&lt;\cdots&lt;G_{k}=G" display="inline"><semantics id="footnote6.m2.1b"><mrow id="footnote6.m2.1.1" xref="footnote6.m2.1.1.cmml"><mn id="footnote6.m2.1.1.2" xref="footnote6.m2.1.1.2.cmml">1</mn><mo id="footnote6.m2.1.1.3" xref="footnote6.m2.1.1.3.cmml">=</mo><msub id="footnote6.m2.1.1.4" xref="footnote6.m2.1.1.4.cmml"><mi id="footnote6.m2.1.1.4.2" xref="footnote6.m2.1.1.4.2.cmml">G</mi><mn id="footnote6.m2.1.1.4.3" xref="footnote6.m2.1.1.4.3.cmml">0</mn></msub><mo id="footnote6.m2.1.1.5" xref="footnote6.m2.1.1.5.cmml">&lt;</mo><msub id="footnote6.m2.1.1.6" xref="footnote6.m2.1.1.6.cmml"><mi id="footnote6.m2.1.1.6.2" xref="footnote6.m2.1.1.6.2.cmml">G</mi><mn id="footnote6.m2.1.1.6.3" xref="footnote6.m2.1.1.6.3.cmml">1</mn></msub><mo id="footnote6.m2.1.1.7" xref="footnote6.m2.1.1.7.cmml">&lt;</mo><mi mathvariant="normal" id="footnote6.m2.1.1.8" xref="footnote6.m2.1.1.8.cmml">⋯</mi><mo id="footnote6.m2.1.1.9" xref="footnote6.m2.1.1.9.cmml">&lt;</mo><msub id="footnote6.m2.1.1.10" xref="footnote6.m2.1.1.10.cmml"><mi id="footnote6.m2.1.1.10.2" xref="footnote6.m2.1.1.10.2.cmml">G</mi><mi id="footnote6.m2.1.1.10.3" xref="footnote6.m2.1.1.10.3.cmml">k</mi></msub><mo id="footnote6.m2.1.1.11" xref="footnote6.m2.1.1.11.cmml">=</mo><mi id="footnote6.m2.1.1.12" xref="footnote6.m2.1.1.12.cmml">G</mi></mrow><annotation-xml encoding="MathML-Content" id="footnote6.m2.1c"><apply id="footnote6.m2.1.1.cmml" xref="footnote6.m2.1.1"><and id="footnote6.m2.1.1a.cmml" xref="footnote6.m2.1.1"></and><apply id="footnote6.m2.1.1b.cmml" xref="footnote6.m2.1.1"><eq id="footnote6.m2.1.1.3.cmml" xref="footnote6.m2.1.1.3"></eq><cn type="integer" id="footnote6.m2.1.1.2.cmml" xref="footnote6.m2.1.1.2">1</cn><apply id="footnote6.m2.1.1.4.cmml" xref="footnote6.m2.1.1.4"><csymbol cd="ambiguous" id="footnote6.m2.1.1.4.1.cmml" xref="footnote6.m2.1.1.4">subscript</csymbol><ci id="footnote6.m2.1.1.4.2.cmml" xref="footnote6.m2.1.1.4.2">𝐺</ci><cn type="integer" id="footnote6.m2.1.1.4.3.cmml" xref="footnote6.m2.1.1.4.3">0</cn></apply></apply><apply id="footnote6.m2.1.1c.cmml" xref="footnote6.m2.1.1"><lt id="footnote6.m2.1.1.5.cmml" xref="footnote6.m2.1.1.5"></lt><share href="#footnote6.m2.1.1.4.cmml" id="footnote6.m2.1.1d.cmml" xref="footnote6.m2.1.1"></share><apply id="footnote6.m2.1.1.6.cmml" xref="footnote6.m2.1.1.6"><csymbol cd="ambiguous" id="footnote6.m2.1.1.6.1.cmml" xref="footnote6.m2.1.1.6">subscript</csymbol><ci id="footnote6.m2.1.1.6.2.cmml" xref="footnote6.m2.1.1.6.2">𝐺</ci><cn type="integer" id="footnote6.m2.1.1.6.3.cmml" xref="footnote6.m2.1.1.6.3">1</cn></apply></apply><apply id="footnote6.m2.1.1e.cmml" xref="footnote6.m2.1.1"><lt id="footnote6.m2.1.1.7.cmml" xref="footnote6.m2.1.1.7"></lt><share href="#footnote6.m2.1.1.6.cmml" id="footnote6.m2.1.1f.cmml" xref="footnote6.m2.1.1"></share><ci id="footnote6.m2.1.1.8.cmml" xref="footnote6.m2.1.1.8">⋯</ci></apply><apply id="footnote6.m2.1.1g.cmml" xref="footnote6.m2.1.1"><lt id="footnote6.m2.1.1.9.cmml" xref="footnote6.m2.1.1.9"></lt><share href="#footnote6.m2.1.1.8.cmml" id="footnote6.m2.1.1h.cmml" xref="footnote6.m2.1.1"></share><apply id="footnote6.m2.1.1.10.cmml" xref="footnote6.m2.1.1.10"><csymbol cd="ambiguous" id="footnote6.m2.1.1.10.1.cmml" xref="footnote6.m2.1.1.10">subscript</csymbol><ci id="footnote6.m2.1.1.10.2.cmml" xref="footnote6.m2.1.1.10.2">𝐺</ci><ci id="footnote6.m2.1.1.10.3.cmml" xref="footnote6.m2.1.1.10.3">𝑘</ci></apply></apply><apply id="footnote6.m2.1.1i.cmml" xref="footnote6.m2.1.1"><eq id="footnote6.m2.1.1.11.cmml" xref="footnote6.m2.1.1.11"></eq><share href="#footnote6.m2.1.1.10.cmml" id="footnote6.m2.1.1j.cmml" xref="footnote6.m2.1.1"></share><ci id="footnote6.m2.1.1.12.cmml" xref="footnote6.m2.1.1.12">𝐺</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote6.m2.1d">1=G_{0}&lt;G_{1}&lt;\cdots&lt;G_{k}=G</annotation></semantics></math> such that <math id="footnote6.m3.1" class="ltx_Math" alttext="G_{i-1}" display="inline"><semantics id="footnote6.m3.1b"><msub id="footnote6.m3.1.1" xref="footnote6.m3.1.1.cmml"><mi id="footnote6.m3.1.1.2" xref="footnote6.m3.1.1.2.cmml">G</mi><mrow id="footnote6.m3.1.1.3" xref="footnote6.m3.1.1.3.cmml"><mi id="footnote6.m3.1.1.3.2" xref="footnote6.m3.1.1.3.2.cmml">i</mi><mo id="footnote6.m3.1.1.3.1" xref="footnote6.m3.1.1.3.1.cmml">−</mo><mn id="footnote6.m3.1.1.3.3" xref="footnote6.m3.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="footnote6.m3.1c"><apply id="footnote6.m3.1.1.cmml" xref="footnote6.m3.1.1"><csymbol cd="ambiguous" id="footnote6.m3.1.1.1.cmml" xref="footnote6.m3.1.1">subscript</csymbol><ci id="footnote6.m3.1.1.2.cmml" xref="footnote6.m3.1.1.2">𝐺</ci><apply id="footnote6.m3.1.1.3.cmml" xref="footnote6.m3.1.1.3"><minus id="footnote6.m3.1.1.3.1.cmml" xref="footnote6.m3.1.1.3.1"></minus><ci id="footnote6.m3.1.1.3.2.cmml" xref="footnote6.m3.1.1.3.2">𝑖</ci><cn type="integer" id="footnote6.m3.1.1.3.3.cmml" xref="footnote6.m3.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote6.m3.1d">G_{i-1}</annotation></semantics></math> is normal in <math id="footnote6.m4.1" class="ltx_Math" alttext="G_{i}" display="inline"><semantics id="footnote6.m4.1b"><msub id="footnote6.m4.1.1" xref="footnote6.m4.1.1.cmml"><mi id="footnote6.m4.1.1.2" xref="footnote6.m4.1.1.2.cmml">G</mi><mi id="footnote6.m4.1.1.3" xref="footnote6.m4.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="footnote6.m4.1c"><apply id="footnote6.m4.1.1.cmml" xref="footnote6.m4.1.1"><csymbol cd="ambiguous" id="footnote6.m4.1.1.1.cmml" xref="footnote6.m4.1.1">subscript</csymbol><ci id="footnote6.m4.1.1.2.cmml" xref="footnote6.m4.1.1.2">𝐺</ci><ci id="footnote6.m4.1.1.3.cmml" xref="footnote6.m4.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote6.m4.1d">G_{i}</annotation></semantics></math> and <math id="footnote6.m5.1" class="ltx_Math" alttext="G_{i}/G_{i-1}" display="inline"><semantics id="footnote6.m5.1b"><mrow id="footnote6.m5.1.1" xref="footnote6.m5.1.1.cmml"><msub id="footnote6.m5.1.1.2" xref="footnote6.m5.1.1.2.cmml"><mi id="footnote6.m5.1.1.2.2" xref="footnote6.m5.1.1.2.2.cmml">G</mi><mi id="footnote6.m5.1.1.2.3" xref="footnote6.m5.1.1.2.3.cmml">i</mi></msub><mo id="footnote6.m5.1.1.1" xref="footnote6.m5.1.1.1.cmml">/</mo><msub id="footnote6.m5.1.1.3" xref="footnote6.m5.1.1.3.cmml"><mi id="footnote6.m5.1.1.3.2" xref="footnote6.m5.1.1.3.2.cmml">G</mi><mrow id="footnote6.m5.1.1.3.3" xref="footnote6.m5.1.1.3.3.cmml"><mi id="footnote6.m5.1.1.3.3.2" xref="footnote6.m5.1.1.3.3.2.cmml">i</mi><mo id="footnote6.m5.1.1.3.3.1" xref="footnote6.m5.1.1.3.3.1.cmml">−</mo><mn id="footnote6.m5.1.1.3.3.3" xref="footnote6.m5.1.1.3.3.3.cmml">1</mn></mrow></msub></mrow><annotation-xml encoding="MathML-Content" id="footnote6.m5.1c"><apply id="footnote6.m5.1.1.cmml" xref="footnote6.m5.1.1"><divide id="footnote6.m5.1.1.1.cmml" xref="footnote6.m5.1.1.1"></divide><apply id="footnote6.m5.1.1.2.cmml" xref="footnote6.m5.1.1.2"><csymbol cd="ambiguous" id="footnote6.m5.1.1.2.1.cmml" xref="footnote6.m5.1.1.2">subscript</csymbol><ci id="footnote6.m5.1.1.2.2.cmml" xref="footnote6.m5.1.1.2.2">𝐺</ci><ci id="footnote6.m5.1.1.2.3.cmml" xref="footnote6.m5.1.1.2.3">𝑖</ci></apply><apply id="footnote6.m5.1.1.3.cmml" xref="footnote6.m5.1.1.3"><csymbol cd="ambiguous" id="footnote6.m5.1.1.3.1.cmml" xref="footnote6.m5.1.1.3">subscript</csymbol><ci id="footnote6.m5.1.1.3.2.cmml" xref="footnote6.m5.1.1.3.2">𝐺</ci><apply id="footnote6.m5.1.1.3.3.cmml" xref="footnote6.m5.1.1.3.3"><minus id="footnote6.m5.1.1.3.3.1.cmml" xref="footnote6.m5.1.1.3.3.1"></minus><ci id="footnote6.m5.1.1.3.3.2.cmml" xref="footnote6.m5.1.1.3.3.2">𝑖</ci><cn type="integer" id="footnote6.m5.1.1.3.3.3.cmml" xref="footnote6.m5.1.1.3.3.3">1</cn></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote6.m5.1d">G_{i}/G_{i-1}</annotation></semantics></math> is abelian.</span></span></span> group is <math id="S3.SS1.p3.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS1.p3.2.m2.1a"><msup id="S3.SS1.p3.2.m2.1.1" xref="S3.SS1.p3.2.m2.1.1.cmml"><mi id="S3.SS1.p3.2.m2.1.1.2" xref="S3.SS1.p3.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS1.p3.2.m2.1.1.3" xref="S3.SS1.p3.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.2.m2.1b"><apply id="S3.SS1.p3.2.m2.1.1.cmml" xref="S3.SS1.p3.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.2.m2.1.1.1.cmml" xref="S3.SS1.p3.2.m2.1.1">superscript</csymbol><ci id="S3.SS1.p3.2.m2.1.1.2.cmml" xref="S3.SS1.p3.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS1.p3.2.m2.1.1.3.cmml" xref="S3.SS1.p3.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete. That non-solvable groups have <math id="S3.SS1.p3.3.m3.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS1.p3.3.m3.1a"><msup id="S3.SS1.p3.3.m3.1.1" xref="S3.SS1.p3.3.m3.1.1.cmml"><mi id="S3.SS1.p3.3.m3.1.1.2" xref="S3.SS1.p3.3.m3.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS1.p3.3.m3.1.1.3" xref="S3.SS1.p3.3.m3.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.3.m3.1b"><apply id="S3.SS1.p3.3.m3.1.1.cmml" xref="S3.SS1.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.1.1.1.cmml" xref="S3.SS1.p3.3.m3.1.1">superscript</csymbol><ci id="S3.SS1.p3.3.m3.1.1.2.cmml" xref="S3.SS1.p3.3.m3.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS1.p3.3.m3.1.1.3.cmml" xref="S3.SS1.p3.3.m3.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.3.m3.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete word problems is notable because of the ubiquity with which non-solvable groups show up in tasks involving state tracking. The canonical example of an <math id="S3.SS1.p3.4.m4.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS1.p3.4.m4.1a"><msup id="S3.SS1.p3.4.m4.1.1" xref="S3.SS1.p3.4.m4.1.1.cmml"><mi id="S3.SS1.p3.4.m4.1.1.2" xref="S3.SS1.p3.4.m4.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS1.p3.4.m4.1.1.3" xref="S3.SS1.p3.4.m4.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.4.m4.1b"><apply id="S3.SS1.p3.4.m4.1.1.cmml" xref="S3.SS1.p3.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.4.m4.1.1.1.cmml" xref="S3.SS1.p3.4.m4.1.1">superscript</csymbol><ci id="S3.SS1.p3.4.m4.1.1.2.cmml" xref="S3.SS1.p3.4.m4.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS1.p3.4.m4.1.1.3.cmml" xref="S3.SS1.p3.4.m4.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.4.m4.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete word problem is that of <math id="S3.SS1.p3.5.m5.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS1.p3.5.m5.1a"><msub id="S3.SS1.p3.5.m5.1.1" xref="S3.SS1.p3.5.m5.1.1.cmml"><mi id="S3.SS1.p3.5.m5.1.1.2" xref="S3.SS1.p3.5.m5.1.1.2.cmml">S</mi><mn id="S3.SS1.p3.5.m5.1.1.3" xref="S3.SS1.p3.5.m5.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.5.m5.1b"><apply id="S3.SS1.p3.5.m5.1.1.cmml" xref="S3.SS1.p3.5.m5.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.5.m5.1.1.1.cmml" xref="S3.SS1.p3.5.m5.1.1">subscript</csymbol><ci id="S3.SS1.p3.5.m5.1.1.2.cmml" xref="S3.SS1.p3.5.m5.1.1.2">𝑆</ci><cn type="integer" id="S3.SS1.p3.5.m5.1.1.3.cmml" xref="S3.SS1.p3.5.m5.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.5.m5.1c">S_{5}</annotation></semantics></math>, the symmetric group on five elements that encodes the permutations over five objects. As an immediate instantiation of this, consider a document describing arbitrarily many sequences of transpositions: <em id="S3.SS1.p3.8.1" class="ltx_emph ltx_font_italic">“swap ball 1 and 3, swap ball 3 and 5, swap ball 4 and 2, …”</em><span id="footnote7" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">7</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">7</sup><span class="ltx_tag ltx_tag_note">7</span>Without less of generality, any permutation can be factored into a sequence of transpositions, or swaps. This means the transpositions over five elements are a generator for <math id="footnote7.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="footnote7.m1.1b"><msub id="footnote7.m1.1.1" xref="footnote7.m1.1.1.cmml"><mi id="footnote7.m1.1.1.2" xref="footnote7.m1.1.1.2.cmml">S</mi><mn id="footnote7.m1.1.1.3" xref="footnote7.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="footnote7.m1.1c"><apply id="footnote7.m1.1.1.cmml" xref="footnote7.m1.1.1"><csymbol cd="ambiguous" id="footnote7.m1.1.1.1.cmml" xref="footnote7.m1.1.1">subscript</csymbol><ci id="footnote7.m1.1.1.2.cmml" xref="footnote7.m1.1.1.2">𝑆</ci><cn type="integer" id="footnote7.m1.1.1.3.cmml" xref="footnote7.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote7.m1.1d">S_{5}</annotation></semantics></math>.</span></span></span>. Being able to answer the question <em id="S3.SS1.p3.8.2" class="ltx_emph ltx_font_italic">“where does ball 5 end up?”</em> requires solving the <math id="S3.SS1.p3.6.m6.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS1.p3.6.m6.1a"><msub id="S3.SS1.p3.6.m6.1.1" xref="S3.SS1.p3.6.m6.1.1.cmml"><mi id="S3.SS1.p3.6.m6.1.1.2" xref="S3.SS1.p3.6.m6.1.1.2.cmml">S</mi><mn id="S3.SS1.p3.6.m6.1.1.3" xref="S3.SS1.p3.6.m6.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.6.m6.1b"><apply id="S3.SS1.p3.6.m6.1.1.cmml" xref="S3.SS1.p3.6.m6.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.6.m6.1.1.1.cmml" xref="S3.SS1.p3.6.m6.1.1">subscript</csymbol><ci id="S3.SS1.p3.6.m6.1.1.2.cmml" xref="S3.SS1.p3.6.m6.1.1.2">𝑆</ci><cn type="integer" id="S3.SS1.p3.6.m6.1.1.3.cmml" xref="S3.SS1.p3.6.m6.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.6.m6.1c">S_{5}</annotation></semantics></math> word problem. Beyond permutations, <a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a> shows how many natural state-tracking problems like tracking chess moves, evaluating code, or tracking entities also encode the structure of <math id="S3.SS1.p3.7.m7.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS1.p3.7.m7.1a"><msub id="S3.SS1.p3.7.m7.1.1" xref="S3.SS1.p3.7.m7.1.1.cmml"><mi id="S3.SS1.p3.7.m7.1.1.2" xref="S3.SS1.p3.7.m7.1.1.2.cmml">S</mi><mn id="S3.SS1.p3.7.m7.1.1.3" xref="S3.SS1.p3.7.m7.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.7.m7.1b"><apply id="S3.SS1.p3.7.m7.1.1.cmml" xref="S3.SS1.p3.7.m7.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.7.m7.1.1.1.cmml" xref="S3.SS1.p3.7.m7.1.1">subscript</csymbol><ci id="S3.SS1.p3.7.m7.1.1.2.cmml" xref="S3.SS1.p3.7.m7.1.1.2">𝑆</ci><cn type="integer" id="S3.SS1.p3.7.m7.1.1.3.cmml" xref="S3.SS1.p3.7.m7.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.7.m7.1c">S_{5}</annotation></semantics></math>, meaning these state-tracking problems also cannot be expressed by a model in <math id="S3.SS1.p3.8.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S3.SS1.p3.8.m8.1a"><msup id="S3.SS1.p3.8.m8.1.1" xref="S3.SS1.p3.8.m8.1.1.cmml"><mi id="S3.SS1.p3.8.m8.1.1.2" xref="S3.SS1.p3.8.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="S3.SS1.p3.8.m8.1.1.3" xref="S3.SS1.p3.8.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.8.m8.1b"><apply id="S3.SS1.p3.8.m8.1.1.cmml" xref="S3.SS1.p3.8.m8.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.8.m8.1.1.1.cmml" xref="S3.SS1.p3.8.m8.1.1">superscript</csymbol><ci id="S3.SS1.p3.8.m8.1.1.2.cmml" xref="S3.SS1.p3.8.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="S3.SS1.p3.8.m8.1.1.3.cmml" xref="S3.SS1.p3.8.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.8.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math>. Rather, in order to solve these problems, the depth of the model would have to be expanded to accommodate longer inputs.</p>
</div>
<div id="S3.SS1.p4" class="ltx_para">
<p id="S3.SS1.p4.9" class="ltx_p">Although the <math id="S3.SS1.p4.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS1.p4.1.m1.1a"><msub id="S3.SS1.p4.1.m1.1.1" xref="S3.SS1.p4.1.m1.1.1.cmml"><mi id="S3.SS1.p4.1.m1.1.1.2" xref="S3.SS1.p4.1.m1.1.1.2.cmml">S</mi><mn id="S3.SS1.p4.1.m1.1.1.3" xref="S3.SS1.p4.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.1.m1.1b"><apply id="S3.SS1.p4.1.m1.1.1.cmml" xref="S3.SS1.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.1.m1.1.1.1.cmml" xref="S3.SS1.p4.1.m1.1.1">subscript</csymbol><ci id="S3.SS1.p4.1.m1.1.1.2.cmml" xref="S3.SS1.p4.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.SS1.p4.1.m1.1.1.3.cmml" xref="S3.SS1.p4.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.1.m1.1c">S_{5}</annotation></semantics></math> word problem is canonical, in this paper we will consider the word problem on a closely related group <math id="S3.SS1.p4.2.m2.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S3.SS1.p4.2.m2.1a"><msub id="S3.SS1.p4.2.m2.1.1" xref="S3.SS1.p4.2.m2.1.1.cmml"><mi id="S3.SS1.p4.2.m2.1.1.2" xref="S3.SS1.p4.2.m2.1.1.2.cmml">A</mi><mn id="S3.SS1.p4.2.m2.1.1.3" xref="S3.SS1.p4.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.2.m2.1b"><apply id="S3.SS1.p4.2.m2.1.1.cmml" xref="S3.SS1.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.2.m2.1.1.1.cmml" xref="S3.SS1.p4.2.m2.1.1">subscript</csymbol><ci id="S3.SS1.p4.2.m2.1.1.2.cmml" xref="S3.SS1.p4.2.m2.1.1.2">𝐴</ci><cn type="integer" id="S3.SS1.p4.2.m2.1.1.3.cmml" xref="S3.SS1.p4.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.2.m2.1c">A_{5}</annotation></semantics></math>: the <em id="S3.SS1.p4.9.1" class="ltx_emph ltx_font_italic">alternating</em> group on five elements. We do this for simplicity: <math id="S3.SS1.p4.3.m3.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S3.SS1.p4.3.m3.1a"><msub id="S3.SS1.p4.3.m3.1.1" xref="S3.SS1.p4.3.m3.1.1.cmml"><mi id="S3.SS1.p4.3.m3.1.1.2" xref="S3.SS1.p4.3.m3.1.1.2.cmml">A</mi><mn id="S3.SS1.p4.3.m3.1.1.3" xref="S3.SS1.p4.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.3.m3.1b"><apply id="S3.SS1.p4.3.m3.1.1.cmml" xref="S3.SS1.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.3.m3.1.1.1.cmml" xref="S3.SS1.p4.3.m3.1.1">subscript</csymbol><ci id="S3.SS1.p4.3.m3.1.1.2.cmml" xref="S3.SS1.p4.3.m3.1.1.2">𝐴</ci><cn type="integer" id="S3.SS1.p4.3.m3.1.1.3.cmml" xref="S3.SS1.p4.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.3.m3.1c">A_{5}</annotation></semantics></math> is a subgroup of <math id="S3.SS1.p4.4.m4.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS1.p4.4.m4.1a"><msub id="S3.SS1.p4.4.m4.1.1" xref="S3.SS1.p4.4.m4.1.1.cmml"><mi id="S3.SS1.p4.4.m4.1.1.2" xref="S3.SS1.p4.4.m4.1.1.2.cmml">S</mi><mn id="S3.SS1.p4.4.m4.1.1.3" xref="S3.SS1.p4.4.m4.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.4.m4.1b"><apply id="S3.SS1.p4.4.m4.1.1.cmml" xref="S3.SS1.p4.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.4.m4.1.1.1.cmml" xref="S3.SS1.p4.4.m4.1.1">subscript</csymbol><ci id="S3.SS1.p4.4.m4.1.1.2.cmml" xref="S3.SS1.p4.4.m4.1.1.2">𝑆</ci><cn type="integer" id="S3.SS1.p4.4.m4.1.1.3.cmml" xref="S3.SS1.p4.4.m4.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.4.m4.1c">S_{5}</annotation></semantics></math> containing only even permutations, and is the smallest non-solvable subgroup. We will compare the word problem on <math id="S3.SS1.p4.5.m5.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S3.SS1.p4.5.m5.1a"><msub id="S3.SS1.p4.5.m5.1.1" xref="S3.SS1.p4.5.m5.1.1.cmml"><mi id="S3.SS1.p4.5.m5.1.1.2" xref="S3.SS1.p4.5.m5.1.1.2.cmml">A</mi><mn id="S3.SS1.p4.5.m5.1.1.3" xref="S3.SS1.p4.5.m5.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.5.m5.1b"><apply id="S3.SS1.p4.5.m5.1.1.cmml" xref="S3.SS1.p4.5.m5.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.5.m5.1.1.1.cmml" xref="S3.SS1.p4.5.m5.1.1">subscript</csymbol><ci id="S3.SS1.p4.5.m5.1.1.2.cmml" xref="S3.SS1.p4.5.m5.1.1.2">𝐴</ci><cn type="integer" id="S3.SS1.p4.5.m5.1.1.3.cmml" xref="S3.SS1.p4.5.m5.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.5.m5.1c">A_{5}</annotation></semantics></math> to two other baseline groups: <math id="S3.SS1.p4.6.m6.1" class="ltx_Math" alttext="A_{4}\times\mathbb{Z}_{5}" display="inline"><semantics id="S3.SS1.p4.6.m6.1a"><mrow id="S3.SS1.p4.6.m6.1.1" xref="S3.SS1.p4.6.m6.1.1.cmml"><msub id="S3.SS1.p4.6.m6.1.1.2" xref="S3.SS1.p4.6.m6.1.1.2.cmml"><mi id="S3.SS1.p4.6.m6.1.1.2.2" xref="S3.SS1.p4.6.m6.1.1.2.2.cmml">A</mi><mn id="S3.SS1.p4.6.m6.1.1.2.3" xref="S3.SS1.p4.6.m6.1.1.2.3.cmml">4</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S3.SS1.p4.6.m6.1.1.1" xref="S3.SS1.p4.6.m6.1.1.1.cmml">×</mo><msub id="S3.SS1.p4.6.m6.1.1.3" xref="S3.SS1.p4.6.m6.1.1.3.cmml"><mi id="S3.SS1.p4.6.m6.1.1.3.2" xref="S3.SS1.p4.6.m6.1.1.3.2.cmml">ℤ</mi><mn id="S3.SS1.p4.6.m6.1.1.3.3" xref="S3.SS1.p4.6.m6.1.1.3.3.cmml">5</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.6.m6.1b"><apply id="S3.SS1.p4.6.m6.1.1.cmml" xref="S3.SS1.p4.6.m6.1.1"><times id="S3.SS1.p4.6.m6.1.1.1.cmml" xref="S3.SS1.p4.6.m6.1.1.1"></times><apply id="S3.SS1.p4.6.m6.1.1.2.cmml" xref="S3.SS1.p4.6.m6.1.1.2"><csymbol cd="ambiguous" id="S3.SS1.p4.6.m6.1.1.2.1.cmml" xref="S3.SS1.p4.6.m6.1.1.2">subscript</csymbol><ci id="S3.SS1.p4.6.m6.1.1.2.2.cmml" xref="S3.SS1.p4.6.m6.1.1.2.2">𝐴</ci><cn type="integer" id="S3.SS1.p4.6.m6.1.1.2.3.cmml" xref="S3.SS1.p4.6.m6.1.1.2.3">4</cn></apply><apply id="S3.SS1.p4.6.m6.1.1.3.cmml" xref="S3.SS1.p4.6.m6.1.1.3"><csymbol cd="ambiguous" id="S3.SS1.p4.6.m6.1.1.3.1.cmml" xref="S3.SS1.p4.6.m6.1.1.3">subscript</csymbol><ci id="S3.SS1.p4.6.m6.1.1.3.2.cmml" xref="S3.SS1.p4.6.m6.1.1.3.2">ℤ</ci><cn type="integer" id="S3.SS1.p4.6.m6.1.1.3.3.cmml" xref="S3.SS1.p4.6.m6.1.1.3.3">5</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.6.m6.1c">A_{4}\times\mathbb{Z}_{5}</annotation></semantics></math>, a non-abelian but solvable group; and <math id="S3.SS1.p4.7.m7.1" class="ltx_Math" alttext="\mathbb{Z}_{60}" display="inline"><semantics id="S3.SS1.p4.7.m7.1a"><msub id="S3.SS1.p4.7.m7.1.1" xref="S3.SS1.p4.7.m7.1.1.cmml"><mi id="S3.SS1.p4.7.m7.1.1.2" xref="S3.SS1.p4.7.m7.1.1.2.cmml">ℤ</mi><mn id="S3.SS1.p4.7.m7.1.1.3" xref="S3.SS1.p4.7.m7.1.1.3.cmml">60</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.7.m7.1b"><apply id="S3.SS1.p4.7.m7.1.1.cmml" xref="S3.SS1.p4.7.m7.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.7.m7.1.1.1.cmml" xref="S3.SS1.p4.7.m7.1.1">subscript</csymbol><ci id="S3.SS1.p4.7.m7.1.1.2.cmml" xref="S3.SS1.p4.7.m7.1.1.2">ℤ</ci><cn type="integer" id="S3.SS1.p4.7.m7.1.1.3.cmml" xref="S3.SS1.p4.7.m7.1.1.3">60</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.7.m7.1c">\mathbb{Z}_{60}</annotation></semantics></math>, an abelian group encoding mod-<math id="S3.SS1.p4.8.m8.1" class="ltx_Math" alttext="60" display="inline"><semantics id="S3.SS1.p4.8.m8.1a"><mn id="S3.SS1.p4.8.m8.1.1" xref="S3.SS1.p4.8.m8.1.1.cmml">60</mn><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.8.m8.1b"><cn type="integer" id="S3.SS1.p4.8.m8.1.1.cmml" xref="S3.SS1.p4.8.m8.1.1">60</cn></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.8.m8.1c">60</annotation></semantics></math> addition. We choose these groups as points of comparison because they all have <math id="S3.SS1.p4.9.m9.1" class="ltx_Math" alttext="60" display="inline"><semantics id="S3.SS1.p4.9.m9.1a"><mn id="S3.SS1.p4.9.m9.1.1" xref="S3.SS1.p4.9.m9.1.1.cmml">60</mn><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.9.m9.1b"><cn type="integer" id="S3.SS1.p4.9.m9.1.1.cmml" xref="S3.SS1.p4.9.m9.1.1">60</cn></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.9.m9.1c">60</annotation></semantics></math> distinct elements, meaning that the difficulty in learning their word problems will come only from the complexity of learning the group multiplication operation.</p>
</div>
</section>
<section id="S3.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.2 </span>Encoding <math id="S3.SS2.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.1.m1.1b"><msub id="S3.SS2.1.m1.1.1" xref="S3.SS2.1.m1.1.1.cmml"><mi id="S3.SS2.1.m1.1.1.2" xref="S3.SS2.1.m1.1.1.2.cmml">S</mi><mn id="S3.SS2.1.m1.1.1.3" xref="S3.SS2.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.1.m1.1c"><apply id="S3.SS2.1.m1.1.1.cmml" xref="S3.SS2.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.1.m1.1.1.1.cmml" xref="S3.SS2.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.1.m1.1.1.2.cmml" xref="S3.SS2.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.1.m1.1.1.3.cmml" xref="S3.SS2.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.1.m1.1d">S_{5}</annotation></semantics></math> in Chess State Tracking</h3>

<div id="S3.SS2.p1" class="ltx_para">
<p id="S3.SS2.p1.4" class="ltx_p"><a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a> already gives some intuition into how state-tracking problems encode <math id="S3.SS2.p1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p1.1.m1.1a"><msub id="S3.SS2.p1.1.m1.1.1" xref="S3.SS2.p1.1.m1.1.1.cmml"><mi id="S3.SS2.p1.1.m1.1.1.2" xref="S3.SS2.p1.1.m1.1.1.2.cmml">S</mi><mn id="S3.SS2.p1.1.m1.1.1.3" xref="S3.SS2.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.1.m1.1b"><apply id="S3.SS2.p1.1.m1.1.1.cmml" xref="S3.SS2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.1.m1.1.1.1.cmml" xref="S3.SS2.p1.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p1.1.m1.1.1.2.cmml" xref="S3.SS2.p1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p1.1.m1.1.1.3.cmml" xref="S3.SS2.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.1.m1.1c">S_{5}</annotation></semantics></math>. Out of these examples, the most intricated case is chess. We now give a proper reduction from <math id="S3.SS2.p1.2.m2.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p1.2.m2.1a"><msub id="S3.SS2.p1.2.m2.1.1" xref="S3.SS2.p1.2.m2.1.1.cmml"><mi id="S3.SS2.p1.2.m2.1.1.2" xref="S3.SS2.p1.2.m2.1.1.2.cmml">S</mi><mn id="S3.SS2.p1.2.m2.1.1.3" xref="S3.SS2.p1.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.2.m2.1b"><apply id="S3.SS2.p1.2.m2.1.1.cmml" xref="S3.SS2.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.2.m2.1.1.1.cmml" xref="S3.SS2.p1.2.m2.1.1">subscript</csymbol><ci id="S3.SS2.p1.2.m2.1.1.2.cmml" xref="S3.SS2.p1.2.m2.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p1.2.m2.1.1.3.cmml" xref="S3.SS2.p1.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.2.m2.1c">S_{5}</annotation></semantics></math> to tracking chess moves, showing formally that not just <math id="S3.SS2.p1.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p1.3.m3.1a"><msub id="S3.SS2.p1.3.m3.1.1" xref="S3.SS2.p1.3.m3.1.1.cmml"><mi id="S3.SS2.p1.3.m3.1.1.2" xref="S3.SS2.p1.3.m3.1.1.2.cmml">S</mi><mn id="S3.SS2.p1.3.m3.1.1.3" xref="S3.SS2.p1.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.3.m3.1b"><apply id="S3.SS2.p1.3.m3.1.1.cmml" xref="S3.SS2.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.3.m3.1.1.1.cmml" xref="S3.SS2.p1.3.m3.1.1">subscript</csymbol><ci id="S3.SS2.p1.3.m3.1.1.2.cmml" xref="S3.SS2.p1.3.m3.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p1.3.m3.1.1.3.cmml" xref="S3.SS2.p1.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.3.m3.1c">S_{5}</annotation></semantics></math>, but chess state tracking as well, is <math id="S3.SS2.p1.4.m4.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS2.p1.4.m4.1a"><msup id="S3.SS2.p1.4.m4.1.1" xref="S3.SS2.p1.4.m4.1.1.cmml"><mi id="S3.SS2.p1.4.m4.1.1.2" xref="S3.SS2.p1.4.m4.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS2.p1.4.m4.1.1.3" xref="S3.SS2.p1.4.m4.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS2.p1.4.m4.1b"><apply id="S3.SS2.p1.4.m4.1.1.cmml" xref="S3.SS2.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS2.p1.4.m4.1.1.1.cmml" xref="S3.SS2.p1.4.m4.1.1">superscript</csymbol><ci id="S3.SS2.p1.4.m4.1.1.2.cmml" xref="S3.SS2.p1.4.m4.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS2.p1.4.m4.1.1.3.cmml" xref="S3.SS2.p1.4.m4.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p1.4.m4.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete.</p>
</div>
<div id="S3.SS2.p2" class="ltx_para">
<p id="S3.SS2.p2.1" class="ltx_p">We define the chess state-tracking problem as follows:</p>
<ul id="S3.I1" class="ltx_itemize">
<li id="S3.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i1.p1" class="ltx_para">
<p id="S3.I1.i1.p1.1" class="ltx_p"><span id="S3.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">Input:</span> The <span id="S3.I1.i1.p1.1.2" class="ltx_text ltx_font_bold">state of a chessboard</span> as well as a <span id="S3.I1.i1.p1.1.3" class="ltx_text ltx_font_bold">sequence of chess moves</span>, where each move is specified as a tuple (source square, target square). Note that this differs from the standard notation that represents a move as a piece along with target square and potential disambiguating information.</p>
</div>
</li>
<li id="S3.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i2.p1" class="ltx_para">
<p id="S3.I1.i2.p1.1" class="ltx_p"><span id="S3.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Output:</span> The resulting board state after starting in the initial board state and applying the sequence of moves one after another, ignoring draws. If any move is illegal given its intermediate board state, we enter a special null board state.</p>
</div>
</li>
</ul>
</div>
<div id="S3.SS2.p3" class="ltx_para">
<p id="S3.SS2.p3.4" class="ltx_p">We show that <math id="S3.SS2.p3.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p3.1.m1.1a"><msub id="S3.SS2.p3.1.m1.1.1" xref="S3.SS2.p3.1.m1.1.1.cmml"><mi id="S3.SS2.p3.1.m1.1.1.2" xref="S3.SS2.p3.1.m1.1.1.2.cmml">S</mi><mn id="S3.SS2.p3.1.m1.1.1.3" xref="S3.SS2.p3.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.1.m1.1b"><apply id="S3.SS2.p3.1.m1.1.1.cmml" xref="S3.SS2.p3.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.1.m1.1.1.1.cmml" xref="S3.SS2.p3.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p3.1.m1.1.1.2.cmml" xref="S3.SS2.p3.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p3.1.m1.1.1.3.cmml" xref="S3.SS2.p3.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.1.m1.1c">S_{5}</annotation></semantics></math> can be reduced to chess state tracking, establishing the <math id="S3.SS2.p3.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS2.p3.2.m2.1a"><msup id="S3.SS2.p3.2.m2.1.1" xref="S3.SS2.p3.2.m2.1.1.cmml"><mi id="S3.SS2.p3.2.m2.1.1.2" xref="S3.SS2.p3.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS2.p3.2.m2.1.1.3" xref="S3.SS2.p3.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.2.m2.1b"><apply id="S3.SS2.p3.2.m2.1.1.cmml" xref="S3.SS2.p3.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.2.m2.1.1.1.cmml" xref="S3.SS2.p3.2.m2.1.1">superscript</csymbol><ci id="S3.SS2.p3.2.m2.1.1.2.cmml" xref="S3.SS2.p3.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS2.p3.2.m2.1.1.3.cmml" xref="S3.SS2.p3.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math>-completeness of chess state tracking. In other words, we can map any <math id="S3.SS2.p3.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p3.3.m3.1a"><msub id="S3.SS2.p3.3.m3.1.1" xref="S3.SS2.p3.3.m3.1.1.cmml"><mi id="S3.SS2.p3.3.m3.1.1.2" xref="S3.SS2.p3.3.m3.1.1.2.cmml">S</mi><mn id="S3.SS2.p3.3.m3.1.1.3" xref="S3.SS2.p3.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.3.m3.1b"><apply id="S3.SS2.p3.3.m3.1.1.cmml" xref="S3.SS2.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.3.m3.1.1.1.cmml" xref="S3.SS2.p3.3.m3.1.1">subscript</csymbol><ci id="S3.SS2.p3.3.m3.1.1.2.cmml" xref="S3.SS2.p3.3.m3.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p3.3.m3.1.1.3.cmml" xref="S3.SS2.p3.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.3.m3.1c">S_{5}</annotation></semantics></math> sequence to a sequence of chess moves and read off the answer to the <math id="S3.SS2.p3.4.m4.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p3.4.m4.1a"><msub id="S3.SS2.p3.4.m4.1.1" xref="S3.SS2.p3.4.m4.1.1.cmml"><mi id="S3.SS2.p3.4.m4.1.1.2" xref="S3.SS2.p3.4.m4.1.1.2.cmml">S</mi><mn id="S3.SS2.p3.4.m4.1.1.3" xref="S3.SS2.p3.4.m4.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p3.4.m4.1b"><apply id="S3.SS2.p3.4.m4.1.1.cmml" xref="S3.SS2.p3.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS2.p3.4.m4.1.1.1.cmml" xref="S3.SS2.p3.4.m4.1.1">subscript</csymbol><ci id="S3.SS2.p3.4.m4.1.1.2.cmml" xref="S3.SS2.p3.4.m4.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p3.4.m4.1.1.3.cmml" xref="S3.SS2.p3.4.m4.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p3.4.m4.1c">S_{5}</annotation></semantics></math> instance from the final chessboard state.</p>
</div>
<div id="S3.Thmtheorem3" class="ltx_theorem ltx_theorem_proposition">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S3.Thmtheorem3.1.1.1" class="ltx_text ltx_font_bold">Proposition 3.3</span></span><span id="S3.Thmtheorem3.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S3.Thmtheorem3.p1" class="ltx_para">
<p id="S3.Thmtheorem3.p1.2" class="ltx_p"><math id="S3.Thmtheorem3.p1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.Thmtheorem3.p1.1.m1.1a"><msub id="S3.Thmtheorem3.p1.1.m1.1.1" xref="S3.Thmtheorem3.p1.1.m1.1.1.cmml"><mi id="S3.Thmtheorem3.p1.1.m1.1.1.2" xref="S3.Thmtheorem3.p1.1.m1.1.1.2.cmml">S</mi><mn id="S3.Thmtheorem3.p1.1.m1.1.1.3" xref="S3.Thmtheorem3.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem3.p1.1.m1.1b"><apply id="S3.Thmtheorem3.p1.1.m1.1.1.cmml" xref="S3.Thmtheorem3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S3.Thmtheorem3.p1.1.m1.1.1.1.cmml" xref="S3.Thmtheorem3.p1.1.m1.1.1">subscript</csymbol><ci id="S3.Thmtheorem3.p1.1.m1.1.1.2.cmml" xref="S3.Thmtheorem3.p1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.Thmtheorem3.p1.1.m1.1.1.3.cmml" xref="S3.Thmtheorem3.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem3.p1.1.m1.1c">S_{5}</annotation></semantics></math><span id="S3.Thmtheorem3.p1.2.1" class="ltx_text ltx_font_italic"> can be reduced to chess state tracking via <math id="S3.Thmtheorem3.p1.2.1.m1.1" class="ltx_Math" alttext="\mathsf{NC}^{0}" display="inline"><semantics id="S3.Thmtheorem3.p1.2.1.m1.1a"><msup id="S3.Thmtheorem3.p1.2.1.m1.1.1" xref="S3.Thmtheorem3.p1.2.1.m1.1.1.cmml"><mi id="S3.Thmtheorem3.p1.2.1.m1.1.1.2" xref="S3.Thmtheorem3.p1.2.1.m1.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.Thmtheorem3.p1.2.1.m1.1.1.3" xref="S3.Thmtheorem3.p1.2.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem3.p1.2.1.m1.1b"><apply id="S3.Thmtheorem3.p1.2.1.m1.1.1.cmml" xref="S3.Thmtheorem3.p1.2.1.m1.1.1"><csymbol cd="ambiguous" id="S3.Thmtheorem3.p1.2.1.m1.1.1.1.cmml" xref="S3.Thmtheorem3.p1.2.1.m1.1.1">superscript</csymbol><ci id="S3.Thmtheorem3.p1.2.1.m1.1.1.2.cmml" xref="S3.Thmtheorem3.p1.2.1.m1.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.Thmtheorem3.p1.2.1.m1.1.1.3.cmml" xref="S3.Thmtheorem3.p1.2.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem3.p1.2.1.m1.1c">\mathsf{NC}^{0}</annotation></semantics></math> reductions.</span></p>
</div>
</div>
<div id="S3.SS2.2" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S3.SS2.2.p1" class="ltx_para">
<p id="S3.SS2.2.p1.1" class="ltx_p">The idea, as illustrated in <a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a>, is to map each <math id="S3.SS2.2.p1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.2.p1.1.m1.1a"><msub id="S3.SS2.2.p1.1.m1.1.1" xref="S3.SS2.2.p1.1.m1.1.1.cmml"><mi id="S3.SS2.2.p1.1.m1.1.1.2" xref="S3.SS2.2.p1.1.m1.1.1.2.cmml">S</mi><mn id="S3.SS2.2.p1.1.m1.1.1.3" xref="S3.SS2.2.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.2.p1.1.m1.1b"><apply id="S3.SS2.2.p1.1.m1.1.1.cmml" xref="S3.SS2.2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.2.p1.1.m1.1.1.1.cmml" xref="S3.SS2.2.p1.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.2.p1.1.m1.1.1.2.cmml" xref="S3.SS2.2.p1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.2.p1.1.m1.1.1.3.cmml" xref="S3.SS2.2.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.2.p1.1.m1.1c">S_{5}</annotation></semantics></math> element to a sequence of chess moves that permutes five pieces on the chessboard.
Then, the final chessboard state will allow us to determine the composition of the permutation sequence.
We defer a detailed proof to <a href="#A2" title="Appendix B Chess Reduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">B</span></a>.
∎</p>
</div>
</div>
<div id="S3.SS2.p4" class="ltx_para">
<p id="S3.SS2.p4.3" class="ltx_p">Since <math id="S3.SS2.p4.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S3.SS2.p4.1.m1.1a"><msub id="S3.SS2.p4.1.m1.1.1" xref="S3.SS2.p4.1.m1.1.1.cmml"><mi id="S3.SS2.p4.1.m1.1.1.2" xref="S3.SS2.p4.1.m1.1.1.2.cmml">S</mi><mn id="S3.SS2.p4.1.m1.1.1.3" xref="S3.SS2.p4.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.1.m1.1b"><apply id="S3.SS2.p4.1.m1.1.1.cmml" xref="S3.SS2.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.1.m1.1.1.1.cmml" xref="S3.SS2.p4.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p4.1.m1.1.1.2.cmml" xref="S3.SS2.p4.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S3.SS2.p4.1.m1.1.1.3.cmml" xref="S3.SS2.p4.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.1.m1.1c">S_{5}</annotation></semantics></math> is <math id="S3.SS2.p4.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS2.p4.2.m2.1a"><msup id="S3.SS2.p4.2.m2.1.1" xref="S3.SS2.p4.2.m2.1.1.cmml"><mi id="S3.SS2.p4.2.m2.1.1.2" xref="S3.SS2.p4.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS2.p4.2.m2.1.1.3" xref="S3.SS2.p4.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.2.m2.1b"><apply id="S3.SS2.p4.2.m2.1.1.cmml" xref="S3.SS2.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.2.m2.1.1.1.cmml" xref="S3.SS2.p4.2.m2.1.1">superscript</csymbol><ci id="S3.SS2.p4.2.m2.1.1.2.cmml" xref="S3.SS2.p4.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS2.p4.2.m2.1.1.3.cmml" xref="S3.SS2.p4.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete under <math id="S3.SS2.p4.3.m3.1" class="ltx_Math" alttext="\mathsf{AC}^{0}" display="inline"><semantics id="S3.SS2.p4.3.m3.1a"><msup id="S3.SS2.p4.3.m3.1.1" xref="S3.SS2.p4.3.m3.1.1.cmml"><mi id="S3.SS2.p4.3.m3.1.1.2" xref="S3.SS2.p4.3.m3.1.1.2.cmml">𝖠𝖢</mi><mn id="S3.SS2.p4.3.m3.1.1.3" xref="S3.SS2.p4.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.3.m3.1b"><apply id="S3.SS2.p4.3.m3.1.1.cmml" xref="S3.SS2.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.3.m3.1.1.1.cmml" xref="S3.SS2.p4.3.m3.1.1">superscript</csymbol><ci id="S3.SS2.p4.3.m3.1.1.2.cmml" xref="S3.SS2.p4.3.m3.1.1.2">𝖠𝖢</ci><cn type="integer" id="S3.SS2.p4.3.m3.1.1.3.cmml" xref="S3.SS2.p4.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.3.m3.1c">\mathsf{AC}^{0}</annotation></semantics></math> reductions:</p>
</div>
<div id="S3.Thmtheorem4" class="ltx_theorem ltx_theorem_corollary">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S3.Thmtheorem4.1.1.1" class="ltx_text ltx_font_bold">Corollary 3.4</span></span><span id="S3.Thmtheorem4.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S3.Thmtheorem4.p1" class="ltx_para">
<p id="S3.Thmtheorem4.p1.2" class="ltx_p"><span id="S3.Thmtheorem4.p1.2.2" class="ltx_text ltx_font_italic">The chess state-tracking problem is <math id="S3.Thmtheorem4.p1.1.1.m1.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.Thmtheorem4.p1.1.1.m1.1a"><msup id="S3.Thmtheorem4.p1.1.1.m1.1.1" xref="S3.Thmtheorem4.p1.1.1.m1.1.1.cmml"><mi id="S3.Thmtheorem4.p1.1.1.m1.1.1.2" xref="S3.Thmtheorem4.p1.1.1.m1.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.Thmtheorem4.p1.1.1.m1.1.1.3" xref="S3.Thmtheorem4.p1.1.1.m1.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem4.p1.1.1.m1.1b"><apply id="S3.Thmtheorem4.p1.1.1.m1.1.1.cmml" xref="S3.Thmtheorem4.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S3.Thmtheorem4.p1.1.1.m1.1.1.1.cmml" xref="S3.Thmtheorem4.p1.1.1.m1.1.1">superscript</csymbol><ci id="S3.Thmtheorem4.p1.1.1.m1.1.1.2.cmml" xref="S3.Thmtheorem4.p1.1.1.m1.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.Thmtheorem4.p1.1.1.m1.1.1.3.cmml" xref="S3.Thmtheorem4.p1.1.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem4.p1.1.1.m1.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete under <math id="S3.Thmtheorem4.p1.2.2.m2.1" class="ltx_Math" alttext="\mathsf{AC}^{0}" display="inline"><semantics id="S3.Thmtheorem4.p1.2.2.m2.1a"><msup id="S3.Thmtheorem4.p1.2.2.m2.1.1" xref="S3.Thmtheorem4.p1.2.2.m2.1.1.cmml"><mi id="S3.Thmtheorem4.p1.2.2.m2.1.1.2" xref="S3.Thmtheorem4.p1.2.2.m2.1.1.2.cmml">𝖠𝖢</mi><mn id="S3.Thmtheorem4.p1.2.2.m2.1.1.3" xref="S3.Thmtheorem4.p1.2.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S3.Thmtheorem4.p1.2.2.m2.1b"><apply id="S3.Thmtheorem4.p1.2.2.m2.1.1.cmml" xref="S3.Thmtheorem4.p1.2.2.m2.1.1"><csymbol cd="ambiguous" id="S3.Thmtheorem4.p1.2.2.m2.1.1.1.cmml" xref="S3.Thmtheorem4.p1.2.2.m2.1.1">superscript</csymbol><ci id="S3.Thmtheorem4.p1.2.2.m2.1.1.2.cmml" xref="S3.Thmtheorem4.p1.2.2.m2.1.1.2">𝖠𝖢</ci><cn type="integer" id="S3.Thmtheorem4.p1.2.2.m2.1.1.3.cmml" xref="S3.Thmtheorem4.p1.2.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.Thmtheorem4.p1.2.2.m2.1c">\mathsf{AC}^{0}</annotation></semantics></math> reductions.</span></p>
</div>
</div>
<div id="S3.SS2.p5" class="ltx_para">
<p id="S3.SS2.p5.1" class="ltx_p">Similar reductions can be constructed for evaluating Python or tracking entities in a dialog, as suggested in <a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a>.
For another example of a similar reduction used to prove <math id="S3.SS2.p5.1.m1.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S3.SS2.p5.1.m1.1a"><msup id="S3.SS2.p5.1.m1.1.1" xref="S3.SS2.p5.1.m1.1.1.cmml"><mi id="S3.SS2.p5.1.m1.1.1.2" xref="S3.SS2.p5.1.m1.1.1.2.cmml">𝖭𝖢</mi><mn id="S3.SS2.p5.1.m1.1.1.3" xref="S3.SS2.p5.1.m1.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S3.SS2.p5.1.m1.1b"><apply id="S3.SS2.p5.1.m1.1.1.cmml" xref="S3.SS2.p5.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p5.1.m1.1.1.1.cmml" xref="S3.SS2.p5.1.m1.1.1">superscript</csymbol><ci id="S3.SS2.p5.1.m1.1.1.2.cmml" xref="S3.SS2.p5.1.m1.1.1.2">𝖭𝖢</ci><cn type="integer" id="S3.SS2.p5.1.m1.1.1.3.cmml" xref="S3.SS2.p5.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p5.1.m1.1c">\mathsf{NC}^{1}</annotation></semantics></math>-completeness, we refer the reader to Theorem 3.2 of <cite class="ltx_cite ltx_citemacro_citet">Feng et al. (<a href="#bib.bib6" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>SSMs Can be Simulated in <math id="S4.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.1.m1.1b"><msup id="S4.1.m1.1.1" xref="S4.1.m1.1.1.cmml"><mi id="S4.1.m1.1.1.2" xref="S4.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.1.m1.1.1.3" xref="S4.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.1.m1.1c"><apply id="S4.1.m1.1.1.cmml" xref="S4.1.m1.1.1"><csymbol cd="ambiguous" id="S4.1.m1.1.1.1.cmml" xref="S4.1.m1.1.1">superscript</csymbol><ci id="S4.1.m1.1.1.2.cmml" xref="S4.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.1.m1.1.1.3.cmml" xref="S4.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.1.m1.1d">\mathsf{TC}^{0}</annotation></semantics></math>
</h2>

<div id="S4.p1" class="ltx_para">
<p id="S4.p1.3" class="ltx_p">In this section, we show that the convolutional form of an SSM can be simulated in <math id="S4.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.p1.1.m1.1a"><msup id="S4.p1.1.m1.1.1" xref="S4.p1.1.m1.1.1.cmml"><mi id="S4.p1.1.m1.1.1.2" xref="S4.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.p1.1.m1.1.1.3" xref="S4.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.p1.1.m1.1b"><apply id="S4.p1.1.m1.1.1.cmml" xref="S4.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.p1.1.m1.1.1.1.cmml" xref="S4.p1.1.m1.1.1">superscript</csymbol><ci id="S4.p1.1.m1.1.1.2.cmml" xref="S4.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.p1.1.m1.1.1.3.cmml" xref="S4.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>. Assuming the convolutional form of the model computes the same function as the recurrent form, this implies that SSMs, in whatever parameterization, cannot solve inherently sequential problems, despite their appearance of recurrence and statefulness. We first show containment in <math id="S4.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.p1.2.m2.1a"><msup id="S4.p1.2.m2.1.1" xref="S4.p1.2.m2.1.1.cmml"><mi id="S4.p1.2.m2.1.1.2" xref="S4.p1.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.p1.2.m2.1.1.3" xref="S4.p1.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.p1.2.m2.1b"><apply id="S4.p1.2.m2.1.1.cmml" xref="S4.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S4.p1.2.m2.1.1.1.cmml" xref="S4.p1.2.m2.1.1">superscript</csymbol><ci id="S4.p1.2.m2.1.1.2.cmml" xref="S4.p1.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.p1.2.m2.1.1.3.cmml" xref="S4.p1.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p1.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math> for the simple non-gated variant of generalized S4 models (<a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorem</span> <span class="ltx_text ltx_ref_tag">4.1</span></a>), and then show that the proof goes through for generalized S6 models as well (<a href="#S4.Thmtheorem2" title="Theorem 4.2. ‣ 4.2 Simulating S6 with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorem</span> <span class="ltx_text ltx_ref_tag">4.2</span></a>). The main idea in both proofs is that matrix powering can be computed in <math id="S4.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.p1.3.m3.1a"><msup id="S4.p1.3.m3.1.1" xref="S4.p1.3.m3.1.1.cmml"><mi id="S4.p1.3.m3.1.1.2" xref="S4.p1.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.p1.3.m3.1.1.3" xref="S4.p1.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.p1.3.m3.1b"><apply id="S4.p1.3.m3.1.1.cmml" xref="S4.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S4.p1.3.m3.1.1.1.cmml" xref="S4.p1.3.m3.1.1">superscript</csymbol><ci id="S4.p1.3.m3.1.1.2.cmml" xref="S4.p1.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.p1.3.m3.1.1.3.cmml" xref="S4.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p1.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math> <cite class="ltx_cite ltx_citemacro_citep">(Mereghetti &amp; Palano, <a href="#bib.bib23" title="" class="ltx_ref">2000</a>)</cite>, and computing the convolutional form of an SSM can essentially be reduced to matrix powering.</p>
</div>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Simulating Linear SSMs with a <math id="S4.SS1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.1.m1.1b"><msup id="S4.SS1.1.m1.1.1" xref="S4.SS1.1.m1.1.1.cmml"><mi id="S4.SS1.1.m1.1.1.2" xref="S4.SS1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.1.m1.1.1.3" xref="S4.SS1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.1.m1.1c"><apply id="S4.SS1.1.m1.1.1.cmml" xref="S4.SS1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS1.1.m1.1.1.1.cmml" xref="S4.SS1.1.m1.1.1">superscript</csymbol><ci id="S4.SS1.1.m1.1.1.2.cmml" xref="S4.SS1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.1.m1.1.1.3.cmml" xref="S4.SS1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.1.m1.1d">\mathsf{TC}^{0}</annotation></semantics></math> Circuit Family</h3>

<div id="S4.Thmtheorem1" class="ltx_theorem ltx_theorem_theorem">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S4.Thmtheorem1.1.1.1" class="ltx_text ltx_font_bold">Theorem 4.1</span></span><span id="S4.Thmtheorem1.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S4.Thmtheorem1.p1" class="ltx_para">
<p id="S4.Thmtheorem1.p1.4" class="ltx_p"><span id="S4.Thmtheorem1.p1.4.4" class="ltx_text ltx_font_italic">For any log-precision SSM <math id="S4.Thmtheorem1.p1.1.1.m1.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S4.Thmtheorem1.p1.1.1.m1.1a"><mi id="S4.Thmtheorem1.p1.1.1.m1.1.1" xref="S4.Thmtheorem1.p1.1.1.m1.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem1.p1.1.1.m1.1b"><ci id="S4.Thmtheorem1.p1.1.1.m1.1.1.cmml" xref="S4.Thmtheorem1.p1.1.1.m1.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem1.p1.1.1.m1.1c">M</annotation></semantics></math> with the S4 architecture, there exists an <math id="S4.Thmtheorem1.p1.2.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.Thmtheorem1.p1.2.2.m2.1a"><mi id="S4.Thmtheorem1.p1.2.2.m2.1.1" xref="S4.Thmtheorem1.p1.2.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem1.p1.2.2.m2.1b"><ci id="S4.Thmtheorem1.p1.2.2.m2.1.1.cmml" xref="S4.Thmtheorem1.p1.2.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem1.p1.2.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.Thmtheorem1.p1.3.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.Thmtheorem1.p1.3.3.m3.1a"><msup id="S4.Thmtheorem1.p1.3.3.m3.1.1" xref="S4.Thmtheorem1.p1.3.3.m3.1.1.cmml"><mi id="S4.Thmtheorem1.p1.3.3.m3.1.1.2" xref="S4.Thmtheorem1.p1.3.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.Thmtheorem1.p1.3.3.m3.1.1.3" xref="S4.Thmtheorem1.p1.3.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem1.p1.3.3.m3.1b"><apply id="S4.Thmtheorem1.p1.3.3.m3.1.1.cmml" xref="S4.Thmtheorem1.p1.3.3.m3.1.1"><csymbol cd="ambiguous" id="S4.Thmtheorem1.p1.3.3.m3.1.1.1.cmml" xref="S4.Thmtheorem1.p1.3.3.m3.1.1">superscript</csymbol><ci id="S4.Thmtheorem1.p1.3.3.m3.1.1.2.cmml" xref="S4.Thmtheorem1.p1.3.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.Thmtheorem1.p1.3.3.m3.1.1.3.cmml" xref="S4.Thmtheorem1.p1.3.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem1.p1.3.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit family that computes the same function as <math id="S4.Thmtheorem1.p1.4.4.m4.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S4.Thmtheorem1.p1.4.4.m4.1a"><mi id="S4.Thmtheorem1.p1.4.4.m4.1.1" xref="S4.Thmtheorem1.p1.4.4.m4.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem1.p1.4.4.m4.1b"><ci id="S4.Thmtheorem1.p1.4.4.m4.1.1.cmml" xref="S4.Thmtheorem1.p1.4.4.m4.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem1.p1.4.4.m4.1c">M</annotation></semantics></math>’s convolutional form.</span></p>
</div>
</div>
<div id="S4.SS1.3" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S4.SS1.2.p1" class="ltx_para">
<p id="S4.SS1.2.p1.2" class="ltx_p">It suffices to construct an <math id="S4.SS1.2.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.SS1.2.p1.1.m1.1a"><mi id="S4.SS1.2.p1.1.m1.1.1" xref="S4.SS1.2.p1.1.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.2.p1.1.m1.1b"><ci id="S4.SS1.2.p1.1.m1.1.1.cmml" xref="S4.SS1.2.p1.1.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.2.p1.1.m1.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.SS1.2.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.2.p1.2.m2.1a"><msup id="S4.SS1.2.p1.2.m2.1.1" xref="S4.SS1.2.p1.2.m2.1.1.cmml"><mi id="S4.SS1.2.p1.2.m2.1.1.2" xref="S4.SS1.2.p1.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.2.p1.2.m2.1.1.3" xref="S4.SS1.2.p1.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.2.p1.2.m2.1b"><apply id="S4.SS1.2.p1.2.m2.1.1.cmml" xref="S4.SS1.2.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.2.p1.2.m2.1.1.1.cmml" xref="S4.SS1.2.p1.2.m2.1.1">superscript</csymbol><ci id="S4.SS1.2.p1.2.m2.1.1.2.cmml" xref="S4.SS1.2.p1.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.2.p1.2.m2.1.1.3.cmml" xref="S4.SS1.2.p1.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.2.p1.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit family to simulate a single layer. Then, the full SSM can be simulate by generating this circuit multiple times, routing the output bits from one layer as the inputs to the next. This can be done with log-space overhead by simply storing a counter that tracks the index of the current input gates.</p>
</div>
<div id="S4.SS1.3.p2" class="ltx_para">
<p id="S4.SS1.3.p2.9" class="ltx_p">Recall that the S4 convolutional form Eqn. (<a href="#S2.E2" title="Equation 2 ‣ Definition 2.1 (Linear SSM layer; e.g. S4). ‣ 2.1 Architecture of State-Space Models ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>) is <math id="S4.SS1.3.p2.1.m1.1" class="ltx_Math" alttext="h_{i}=\sum_{j=1}^{i}A^{i-j}Bx_{j}" display="inline"><semantics id="S4.SS1.3.p2.1.m1.1a"><mrow id="S4.SS1.3.p2.1.m1.1.1" xref="S4.SS1.3.p2.1.m1.1.1.cmml"><msub id="S4.SS1.3.p2.1.m1.1.1.2" xref="S4.SS1.3.p2.1.m1.1.1.2.cmml"><mi id="S4.SS1.3.p2.1.m1.1.1.2.2" xref="S4.SS1.3.p2.1.m1.1.1.2.2.cmml">h</mi><mi id="S4.SS1.3.p2.1.m1.1.1.2.3" xref="S4.SS1.3.p2.1.m1.1.1.2.3.cmml">i</mi></msub><mo rspace="0.111em" id="S4.SS1.3.p2.1.m1.1.1.1" xref="S4.SS1.3.p2.1.m1.1.1.1.cmml">=</mo><mrow id="S4.SS1.3.p2.1.m1.1.1.3" xref="S4.SS1.3.p2.1.m1.1.1.3.cmml"><msubsup id="S4.SS1.3.p2.1.m1.1.1.3.1" xref="S4.SS1.3.p2.1.m1.1.1.3.1.cmml"><mo id="S4.SS1.3.p2.1.m1.1.1.3.1.2.2" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.2.cmml">∑</mo><mrow id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.cmml"><mi id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.2" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.2.cmml">j</mi><mo id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.1" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.1.cmml">=</mo><mn id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.3" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="S4.SS1.3.p2.1.m1.1.1.3.1.3" xref="S4.SS1.3.p2.1.m1.1.1.3.1.3.cmml">i</mi></msubsup><mrow id="S4.SS1.3.p2.1.m1.1.1.3.2" xref="S4.SS1.3.p2.1.m1.1.1.3.2.cmml"><msup id="S4.SS1.3.p2.1.m1.1.1.3.2.2" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.cmml"><mi id="S4.SS1.3.p2.1.m1.1.1.3.2.2.2" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.2.cmml">A</mi><mrow id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.cmml"><mi id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.2" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.2.cmml">i</mi><mo id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.1" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.1.cmml">−</mo><mi id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.3" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.3.cmml">j</mi></mrow></msup><mo lspace="0em" rspace="0em" id="S4.SS1.3.p2.1.m1.1.1.3.2.1" xref="S4.SS1.3.p2.1.m1.1.1.3.2.1.cmml">​</mo><mi id="S4.SS1.3.p2.1.m1.1.1.3.2.3" xref="S4.SS1.3.p2.1.m1.1.1.3.2.3.cmml">B</mi><mo lspace="0em" rspace="0em" id="S4.SS1.3.p2.1.m1.1.1.3.2.1a" xref="S4.SS1.3.p2.1.m1.1.1.3.2.1.cmml">​</mo><msub id="S4.SS1.3.p2.1.m1.1.1.3.2.4" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4.cmml"><mi id="S4.SS1.3.p2.1.m1.1.1.3.2.4.2" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4.2.cmml">x</mi><mi id="S4.SS1.3.p2.1.m1.1.1.3.2.4.3" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4.3.cmml">j</mi></msub></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.1.m1.1b"><apply id="S4.SS1.3.p2.1.m1.1.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1"><eq id="S4.SS1.3.p2.1.m1.1.1.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.1"></eq><apply id="S4.SS1.3.p2.1.m1.1.1.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.2"><csymbol cd="ambiguous" id="S4.SS1.3.p2.1.m1.1.1.2.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.2">subscript</csymbol><ci id="S4.SS1.3.p2.1.m1.1.1.2.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.2.2">ℎ</ci><ci id="S4.SS1.3.p2.1.m1.1.1.2.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.2.3">𝑖</ci></apply><apply id="S4.SS1.3.p2.1.m1.1.1.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3"><apply id="S4.SS1.3.p2.1.m1.1.1.3.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.1.m1.1.1.3.1.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1">superscript</csymbol><apply id="S4.SS1.3.p2.1.m1.1.1.3.1.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.1.m1.1.1.3.1.2.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1">subscript</csymbol><sum id="S4.SS1.3.p2.1.m1.1.1.3.1.2.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.2"></sum><apply id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3"><eq id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.1"></eq><ci id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.2">𝑗</ci><cn type="integer" id="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="S4.SS1.3.p2.1.m1.1.1.3.1.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.1.3">𝑖</ci></apply><apply id="S4.SS1.3.p2.1.m1.1.1.3.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2"><times id="S4.SS1.3.p2.1.m1.1.1.3.2.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.1"></times><apply id="S4.SS1.3.p2.1.m1.1.1.3.2.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2"><csymbol cd="ambiguous" id="S4.SS1.3.p2.1.m1.1.1.3.2.2.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2">superscript</csymbol><ci id="S4.SS1.3.p2.1.m1.1.1.3.2.2.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.2">𝐴</ci><apply id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3"><minus id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.1"></minus><ci id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.2">𝑖</ci><ci id="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.2.3.3">𝑗</ci></apply></apply><ci id="S4.SS1.3.p2.1.m1.1.1.3.2.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.3">𝐵</ci><apply id="S4.SS1.3.p2.1.m1.1.1.3.2.4.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4"><csymbol cd="ambiguous" id="S4.SS1.3.p2.1.m1.1.1.3.2.4.1.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4">subscript</csymbol><ci id="S4.SS1.3.p2.1.m1.1.1.3.2.4.2.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4.2">𝑥</ci><ci id="S4.SS1.3.p2.1.m1.1.1.3.2.4.3.cmml" xref="S4.SS1.3.p2.1.m1.1.1.3.2.4.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.1.m1.1c">h_{i}=\sum_{j=1}^{i}A^{i-j}Bx_{j}</annotation></semantics></math>. Crucially, we use the fact that matrix powering over floats is in <math id="S4.SS1.3.p2.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.SS1.3.p2.2.m2.1a"><mi id="S4.SS1.3.p2.2.m2.1.1" xref="S4.SS1.3.p2.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.2.m2.1b"><ci id="S4.SS1.3.p2.2.m2.1.1.cmml" xref="S4.SS1.3.p2.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.SS1.3.p2.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.3.p2.3.m3.1a"><msup id="S4.SS1.3.p2.3.m3.1.1" xref="S4.SS1.3.p2.3.m3.1.1.cmml"><mi id="S4.SS1.3.p2.3.m3.1.1.2" xref="S4.SS1.3.p2.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.3.p2.3.m3.1.1.3" xref="S4.SS1.3.p2.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.3.m3.1b"><apply id="S4.SS1.3.p2.3.m3.1.1.cmml" xref="S4.SS1.3.p2.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.3.m3.1.1.1.cmml" xref="S4.SS1.3.p2.3.m3.1.1">superscript</csymbol><ci id="S4.SS1.3.p2.3.m3.1.1.2.cmml" xref="S4.SS1.3.p2.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.3.p2.3.m3.1.1.3.cmml" xref="S4.SS1.3.p2.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math> (<a href="#S2.Thmtheorem4" title="Lemma 2.4 (Float matrix power). ‣ 2.2 Numeric Datatype ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">2.4</span></a>, extending <cite class="ltx_cite ltx_citemacro_citep">Mereghetti &amp; Palano, <a href="#bib.bib23" title="" class="ltx_ref">2000</a></cite>) to print, for each <math id="S4.SS1.3.p2.4.m4.2" class="ltx_Math" alttext="i,j" display="inline"><semantics id="S4.SS1.3.p2.4.m4.2a"><mrow id="S4.SS1.3.p2.4.m4.2.3.2" xref="S4.SS1.3.p2.4.m4.2.3.1.cmml"><mi id="S4.SS1.3.p2.4.m4.1.1" xref="S4.SS1.3.p2.4.m4.1.1.cmml">i</mi><mo id="S4.SS1.3.p2.4.m4.2.3.2.1" xref="S4.SS1.3.p2.4.m4.2.3.1.cmml">,</mo><mi id="S4.SS1.3.p2.4.m4.2.2" xref="S4.SS1.3.p2.4.m4.2.2.cmml">j</mi></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.4.m4.2b"><list id="S4.SS1.3.p2.4.m4.2.3.1.cmml" xref="S4.SS1.3.p2.4.m4.2.3.2"><ci id="S4.SS1.3.p2.4.m4.1.1.cmml" xref="S4.SS1.3.p2.4.m4.1.1">𝑖</ci><ci id="S4.SS1.3.p2.4.m4.2.2.cmml" xref="S4.SS1.3.p2.4.m4.2.2">𝑗</ci></list></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.4.m4.2c">i,j</annotation></semantics></math>, a <math id="S4.SS1.3.p2.5.m5.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.3.p2.5.m5.1a"><msup id="S4.SS1.3.p2.5.m5.1.1" xref="S4.SS1.3.p2.5.m5.1.1.cmml"><mi id="S4.SS1.3.p2.5.m5.1.1.2" xref="S4.SS1.3.p2.5.m5.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.3.p2.5.m5.1.1.3" xref="S4.SS1.3.p2.5.m5.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.5.m5.1b"><apply id="S4.SS1.3.p2.5.m5.1.1.cmml" xref="S4.SS1.3.p2.5.m5.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.5.m5.1.1.1.cmml" xref="S4.SS1.3.p2.5.m5.1.1">superscript</csymbol><ci id="S4.SS1.3.p2.5.m5.1.1.2.cmml" xref="S4.SS1.3.p2.5.m5.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.3.p2.5.m5.1.1.3.cmml" xref="S4.SS1.3.p2.5.m5.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.5.m5.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit family that computes <math id="S4.SS1.3.p2.6.m6.1" class="ltx_Math" alttext="\pi_{ij}\triangleq A^{i-j}" display="inline"><semantics id="S4.SS1.3.p2.6.m6.1a"><mrow id="S4.SS1.3.p2.6.m6.1.1" xref="S4.SS1.3.p2.6.m6.1.1.cmml"><msub id="S4.SS1.3.p2.6.m6.1.1.2" xref="S4.SS1.3.p2.6.m6.1.1.2.cmml"><mi id="S4.SS1.3.p2.6.m6.1.1.2.2" xref="S4.SS1.3.p2.6.m6.1.1.2.2.cmml">π</mi><mrow id="S4.SS1.3.p2.6.m6.1.1.2.3" xref="S4.SS1.3.p2.6.m6.1.1.2.3.cmml"><mi id="S4.SS1.3.p2.6.m6.1.1.2.3.2" xref="S4.SS1.3.p2.6.m6.1.1.2.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.SS1.3.p2.6.m6.1.1.2.3.1" xref="S4.SS1.3.p2.6.m6.1.1.2.3.1.cmml">​</mo><mi id="S4.SS1.3.p2.6.m6.1.1.2.3.3" xref="S4.SS1.3.p2.6.m6.1.1.2.3.3.cmml">j</mi></mrow></msub><mo id="S4.SS1.3.p2.6.m6.1.1.1" xref="S4.SS1.3.p2.6.m6.1.1.1.cmml">≜</mo><msup id="S4.SS1.3.p2.6.m6.1.1.3" xref="S4.SS1.3.p2.6.m6.1.1.3.cmml"><mi id="S4.SS1.3.p2.6.m6.1.1.3.2" xref="S4.SS1.3.p2.6.m6.1.1.3.2.cmml">A</mi><mrow id="S4.SS1.3.p2.6.m6.1.1.3.3" xref="S4.SS1.3.p2.6.m6.1.1.3.3.cmml"><mi id="S4.SS1.3.p2.6.m6.1.1.3.3.2" xref="S4.SS1.3.p2.6.m6.1.1.3.3.2.cmml">i</mi><mo id="S4.SS1.3.p2.6.m6.1.1.3.3.1" xref="S4.SS1.3.p2.6.m6.1.1.3.3.1.cmml">−</mo><mi id="S4.SS1.3.p2.6.m6.1.1.3.3.3" xref="S4.SS1.3.p2.6.m6.1.1.3.3.3.cmml">j</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.6.m6.1b"><apply id="S4.SS1.3.p2.6.m6.1.1.cmml" xref="S4.SS1.3.p2.6.m6.1.1"><ci id="S4.SS1.3.p2.6.m6.1.1.1.cmml" xref="S4.SS1.3.p2.6.m6.1.1.1">≜</ci><apply id="S4.SS1.3.p2.6.m6.1.1.2.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2"><csymbol cd="ambiguous" id="S4.SS1.3.p2.6.m6.1.1.2.1.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2">subscript</csymbol><ci id="S4.SS1.3.p2.6.m6.1.1.2.2.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2.2">𝜋</ci><apply id="S4.SS1.3.p2.6.m6.1.1.2.3.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2.3"><times id="S4.SS1.3.p2.6.m6.1.1.2.3.1.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2.3.1"></times><ci id="S4.SS1.3.p2.6.m6.1.1.2.3.2.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2.3.2">𝑖</ci><ci id="S4.SS1.3.p2.6.m6.1.1.2.3.3.cmml" xref="S4.SS1.3.p2.6.m6.1.1.2.3.3">𝑗</ci></apply></apply><apply id="S4.SS1.3.p2.6.m6.1.1.3.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3"><csymbol cd="ambiguous" id="S4.SS1.3.p2.6.m6.1.1.3.1.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3">superscript</csymbol><ci id="S4.SS1.3.p2.6.m6.1.1.3.2.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3.2">𝐴</ci><apply id="S4.SS1.3.p2.6.m6.1.1.3.3.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3.3"><minus id="S4.SS1.3.p2.6.m6.1.1.3.3.1.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3.3.1"></minus><ci id="S4.SS1.3.p2.6.m6.1.1.3.3.2.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3.3.2">𝑖</ci><ci id="S4.SS1.3.p2.6.m6.1.1.3.3.3.cmml" xref="S4.SS1.3.p2.6.m6.1.1.3.3.3">𝑗</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.6.m6.1c">\pi_{ij}\triangleq A^{i-j}</annotation></semantics></math>.
Next, we use the fact that fixed-arity arithmetic over floats is in <math id="S4.SS1.3.p2.7.m7.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.SS1.3.p2.7.m7.1a"><mi id="S4.SS1.3.p2.7.m7.1.1" xref="S4.SS1.3.p2.7.m7.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.7.m7.1b"><ci id="S4.SS1.3.p2.7.m7.1.1.cmml" xref="S4.SS1.3.p2.7.m7.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.7.m7.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.SS1.3.p2.8.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.3.p2.8.m8.1a"><msup id="S4.SS1.3.p2.8.m8.1.1" xref="S4.SS1.3.p2.8.m8.1.1.cmml"><mi id="S4.SS1.3.p2.8.m8.1.1.2" xref="S4.SS1.3.p2.8.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.3.p2.8.m8.1.1.3" xref="S4.SS1.3.p2.8.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.8.m8.1b"><apply id="S4.SS1.3.p2.8.m8.1.1.cmml" xref="S4.SS1.3.p2.8.m8.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.8.m8.1.1.1.cmml" xref="S4.SS1.3.p2.8.m8.1.1">superscript</csymbol><ci id="S4.SS1.3.p2.8.m8.1.1.2.cmml" xref="S4.SS1.3.p2.8.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.3.p2.8.m8.1.1.3.cmml" xref="S4.SS1.3.p2.8.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.8.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math> to print, for each <math id="S4.SS1.3.p2.9.m9.2" class="ltx_Math" alttext="i,j" display="inline"><semantics id="S4.SS1.3.p2.9.m9.2a"><mrow id="S4.SS1.3.p2.9.m9.2.3.2" xref="S4.SS1.3.p2.9.m9.2.3.1.cmml"><mi id="S4.SS1.3.p2.9.m9.1.1" xref="S4.SS1.3.p2.9.m9.1.1.cmml">i</mi><mo id="S4.SS1.3.p2.9.m9.2.3.2.1" xref="S4.SS1.3.p2.9.m9.2.3.1.cmml">,</mo><mi id="S4.SS1.3.p2.9.m9.2.2" xref="S4.SS1.3.p2.9.m9.2.2.cmml">j</mi></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.9.m9.2b"><list id="S4.SS1.3.p2.9.m9.2.3.1.cmml" xref="S4.SS1.3.p2.9.m9.2.3.2"><ci id="S4.SS1.3.p2.9.m9.1.1.cmml" xref="S4.SS1.3.p2.9.m9.1.1">𝑖</ci><ci id="S4.SS1.3.p2.9.m9.2.2.cmml" xref="S4.SS1.3.p2.9.m9.2.2">𝑗</ci></list></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.9.m9.2c">i,j</annotation></semantics></math>, a circuit that computes</p>
<table id="S4.Ex3" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S4.Ex3.m1.1" class="ltx_Math" alttext="\tau_{ij}\,\triangleq\,\pi_{ij}Bx_{j}\,=\,A^{i-j}Bx_{j}." display="block"><semantics id="S4.Ex3.m1.1a"><mrow id="S4.Ex3.m1.1.1.1" xref="S4.Ex3.m1.1.1.1.1.cmml"><mrow id="S4.Ex3.m1.1.1.1.1" xref="S4.Ex3.m1.1.1.1.1.cmml"><msub id="S4.Ex3.m1.1.1.1.1.2" xref="S4.Ex3.m1.1.1.1.1.2.cmml"><mi id="S4.Ex3.m1.1.1.1.1.2.2" xref="S4.Ex3.m1.1.1.1.1.2.2.cmml">τ</mi><mrow id="S4.Ex3.m1.1.1.1.1.2.3" xref="S4.Ex3.m1.1.1.1.1.2.3.cmml"><mi id="S4.Ex3.m1.1.1.1.1.2.3.2" xref="S4.Ex3.m1.1.1.1.1.2.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.Ex3.m1.1.1.1.1.2.3.1" xref="S4.Ex3.m1.1.1.1.1.2.3.1.cmml">​</mo><mi id="S4.Ex3.m1.1.1.1.1.2.3.3" xref="S4.Ex3.m1.1.1.1.1.2.3.3.cmml">j</mi></mrow></msub><mo lspace="0.448em" rspace="0.448em" id="S4.Ex3.m1.1.1.1.1.3" xref="S4.Ex3.m1.1.1.1.1.3.cmml">≜</mo><mrow id="S4.Ex3.m1.1.1.1.1.4" xref="S4.Ex3.m1.1.1.1.1.4.cmml"><msub id="S4.Ex3.m1.1.1.1.1.4.2" xref="S4.Ex3.m1.1.1.1.1.4.2.cmml"><mi id="S4.Ex3.m1.1.1.1.1.4.2.2" xref="S4.Ex3.m1.1.1.1.1.4.2.2.cmml">π</mi><mrow id="S4.Ex3.m1.1.1.1.1.4.2.3" xref="S4.Ex3.m1.1.1.1.1.4.2.3.cmml"><mi id="S4.Ex3.m1.1.1.1.1.4.2.3.2" xref="S4.Ex3.m1.1.1.1.1.4.2.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.Ex3.m1.1.1.1.1.4.2.3.1" xref="S4.Ex3.m1.1.1.1.1.4.2.3.1.cmml">​</mo><mi id="S4.Ex3.m1.1.1.1.1.4.2.3.3" xref="S4.Ex3.m1.1.1.1.1.4.2.3.3.cmml">j</mi></mrow></msub><mo lspace="0em" rspace="0em" id="S4.Ex3.m1.1.1.1.1.4.1" xref="S4.Ex3.m1.1.1.1.1.4.1.cmml">​</mo><mi id="S4.Ex3.m1.1.1.1.1.4.3" xref="S4.Ex3.m1.1.1.1.1.4.3.cmml">B</mi><mo lspace="0em" rspace="0em" id="S4.Ex3.m1.1.1.1.1.4.1a" xref="S4.Ex3.m1.1.1.1.1.4.1.cmml">​</mo><msub id="S4.Ex3.m1.1.1.1.1.4.4" xref="S4.Ex3.m1.1.1.1.1.4.4.cmml"><mi id="S4.Ex3.m1.1.1.1.1.4.4.2" xref="S4.Ex3.m1.1.1.1.1.4.4.2.cmml">x</mi><mi id="S4.Ex3.m1.1.1.1.1.4.4.3" xref="S4.Ex3.m1.1.1.1.1.4.4.3.cmml">j</mi></msub></mrow><mo rspace="0.448em" id="S4.Ex3.m1.1.1.1.1.5" xref="S4.Ex3.m1.1.1.1.1.5.cmml">=</mo><mrow id="S4.Ex3.m1.1.1.1.1.6" xref="S4.Ex3.m1.1.1.1.1.6.cmml"><msup id="S4.Ex3.m1.1.1.1.1.6.2" xref="S4.Ex3.m1.1.1.1.1.6.2.cmml"><mi id="S4.Ex3.m1.1.1.1.1.6.2.2" xref="S4.Ex3.m1.1.1.1.1.6.2.2.cmml">A</mi><mrow id="S4.Ex3.m1.1.1.1.1.6.2.3" xref="S4.Ex3.m1.1.1.1.1.6.2.3.cmml"><mi id="S4.Ex3.m1.1.1.1.1.6.2.3.2" xref="S4.Ex3.m1.1.1.1.1.6.2.3.2.cmml">i</mi><mo id="S4.Ex3.m1.1.1.1.1.6.2.3.1" xref="S4.Ex3.m1.1.1.1.1.6.2.3.1.cmml">−</mo><mi id="S4.Ex3.m1.1.1.1.1.6.2.3.3" xref="S4.Ex3.m1.1.1.1.1.6.2.3.3.cmml">j</mi></mrow></msup><mo lspace="0em" rspace="0em" id="S4.Ex3.m1.1.1.1.1.6.1" xref="S4.Ex3.m1.1.1.1.1.6.1.cmml">​</mo><mi id="S4.Ex3.m1.1.1.1.1.6.3" xref="S4.Ex3.m1.1.1.1.1.6.3.cmml">B</mi><mo lspace="0em" rspace="0em" id="S4.Ex3.m1.1.1.1.1.6.1a" xref="S4.Ex3.m1.1.1.1.1.6.1.cmml">​</mo><msub id="S4.Ex3.m1.1.1.1.1.6.4" xref="S4.Ex3.m1.1.1.1.1.6.4.cmml"><mi id="S4.Ex3.m1.1.1.1.1.6.4.2" xref="S4.Ex3.m1.1.1.1.1.6.4.2.cmml">x</mi><mi id="S4.Ex3.m1.1.1.1.1.6.4.3" xref="S4.Ex3.m1.1.1.1.1.6.4.3.cmml">j</mi></msub></mrow></mrow><mo lspace="0em" id="S4.Ex3.m1.1.1.1.2" xref="S4.Ex3.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.Ex3.m1.1b"><apply id="S4.Ex3.m1.1.1.1.1.cmml" xref="S4.Ex3.m1.1.1.1"><and id="S4.Ex3.m1.1.1.1.1a.cmml" xref="S4.Ex3.m1.1.1.1"></and><apply id="S4.Ex3.m1.1.1.1.1b.cmml" xref="S4.Ex3.m1.1.1.1"><ci id="S4.Ex3.m1.1.1.1.1.3.cmml" xref="S4.Ex3.m1.1.1.1.1.3">≜</ci><apply id="S4.Ex3.m1.1.1.1.1.2.cmml" xref="S4.Ex3.m1.1.1.1.1.2"><csymbol cd="ambiguous" id="S4.Ex3.m1.1.1.1.1.2.1.cmml" xref="S4.Ex3.m1.1.1.1.1.2">subscript</csymbol><ci id="S4.Ex3.m1.1.1.1.1.2.2.cmml" xref="S4.Ex3.m1.1.1.1.1.2.2">𝜏</ci><apply id="S4.Ex3.m1.1.1.1.1.2.3.cmml" xref="S4.Ex3.m1.1.1.1.1.2.3"><times id="S4.Ex3.m1.1.1.1.1.2.3.1.cmml" xref="S4.Ex3.m1.1.1.1.1.2.3.1"></times><ci id="S4.Ex3.m1.1.1.1.1.2.3.2.cmml" xref="S4.Ex3.m1.1.1.1.1.2.3.2">𝑖</ci><ci id="S4.Ex3.m1.1.1.1.1.2.3.3.cmml" xref="S4.Ex3.m1.1.1.1.1.2.3.3">𝑗</ci></apply></apply><apply id="S4.Ex3.m1.1.1.1.1.4.cmml" xref="S4.Ex3.m1.1.1.1.1.4"><times id="S4.Ex3.m1.1.1.1.1.4.1.cmml" xref="S4.Ex3.m1.1.1.1.1.4.1"></times><apply id="S4.Ex3.m1.1.1.1.1.4.2.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2"><csymbol cd="ambiguous" id="S4.Ex3.m1.1.1.1.1.4.2.1.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2">subscript</csymbol><ci id="S4.Ex3.m1.1.1.1.1.4.2.2.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2.2">𝜋</ci><apply id="S4.Ex3.m1.1.1.1.1.4.2.3.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2.3"><times id="S4.Ex3.m1.1.1.1.1.4.2.3.1.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2.3.1"></times><ci id="S4.Ex3.m1.1.1.1.1.4.2.3.2.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2.3.2">𝑖</ci><ci id="S4.Ex3.m1.1.1.1.1.4.2.3.3.cmml" xref="S4.Ex3.m1.1.1.1.1.4.2.3.3">𝑗</ci></apply></apply><ci id="S4.Ex3.m1.1.1.1.1.4.3.cmml" xref="S4.Ex3.m1.1.1.1.1.4.3">𝐵</ci><apply id="S4.Ex3.m1.1.1.1.1.4.4.cmml" xref="S4.Ex3.m1.1.1.1.1.4.4"><csymbol cd="ambiguous" id="S4.Ex3.m1.1.1.1.1.4.4.1.cmml" xref="S4.Ex3.m1.1.1.1.1.4.4">subscript</csymbol><ci id="S4.Ex3.m1.1.1.1.1.4.4.2.cmml" xref="S4.Ex3.m1.1.1.1.1.4.4.2">𝑥</ci><ci id="S4.Ex3.m1.1.1.1.1.4.4.3.cmml" xref="S4.Ex3.m1.1.1.1.1.4.4.3">𝑗</ci></apply></apply></apply><apply id="S4.Ex3.m1.1.1.1.1c.cmml" xref="S4.Ex3.m1.1.1.1"><eq id="S4.Ex3.m1.1.1.1.1.5.cmml" xref="S4.Ex3.m1.1.1.1.1.5"></eq><share href="#S4.Ex3.m1.1.1.1.1.4.cmml" id="S4.Ex3.m1.1.1.1.1d.cmml" xref="S4.Ex3.m1.1.1.1"></share><apply id="S4.Ex3.m1.1.1.1.1.6.cmml" xref="S4.Ex3.m1.1.1.1.1.6"><times id="S4.Ex3.m1.1.1.1.1.6.1.cmml" xref="S4.Ex3.m1.1.1.1.1.6.1"></times><apply id="S4.Ex3.m1.1.1.1.1.6.2.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2"><csymbol cd="ambiguous" id="S4.Ex3.m1.1.1.1.1.6.2.1.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2">superscript</csymbol><ci id="S4.Ex3.m1.1.1.1.1.6.2.2.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2.2">𝐴</ci><apply id="S4.Ex3.m1.1.1.1.1.6.2.3.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2.3"><minus id="S4.Ex3.m1.1.1.1.1.6.2.3.1.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2.3.1"></minus><ci id="S4.Ex3.m1.1.1.1.1.6.2.3.2.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2.3.2">𝑖</ci><ci id="S4.Ex3.m1.1.1.1.1.6.2.3.3.cmml" xref="S4.Ex3.m1.1.1.1.1.6.2.3.3">𝑗</ci></apply></apply><ci id="S4.Ex3.m1.1.1.1.1.6.3.cmml" xref="S4.Ex3.m1.1.1.1.1.6.3">𝐵</ci><apply id="S4.Ex3.m1.1.1.1.1.6.4.cmml" xref="S4.Ex3.m1.1.1.1.1.6.4"><csymbol cd="ambiguous" id="S4.Ex3.m1.1.1.1.1.6.4.1.cmml" xref="S4.Ex3.m1.1.1.1.1.6.4">subscript</csymbol><ci id="S4.Ex3.m1.1.1.1.1.6.4.2.cmml" xref="S4.Ex3.m1.1.1.1.1.6.4.2">𝑥</ci><ci id="S4.Ex3.m1.1.1.1.1.6.4.3.cmml" xref="S4.Ex3.m1.1.1.1.1.6.4.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Ex3.m1.1c">\tau_{ij}\,\triangleq\,\pi_{ij}Bx_{j}\,=\,A^{i-j}Bx_{j}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S4.SS1.3.p2.16" class="ltx_p">Since iterated addition over floats is in <math id="S4.SS1.3.p2.10.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.SS1.3.p2.10.m1.1a"><mi id="S4.SS1.3.p2.10.m1.1.1" xref="S4.SS1.3.p2.10.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.10.m1.1b"><ci id="S4.SS1.3.p2.10.m1.1.1.cmml" xref="S4.SS1.3.p2.10.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.10.m1.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.SS1.3.p2.11.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.3.p2.11.m2.1a"><msup id="S4.SS1.3.p2.11.m2.1.1" xref="S4.SS1.3.p2.11.m2.1.1.cmml"><mi id="S4.SS1.3.p2.11.m2.1.1.2" xref="S4.SS1.3.p2.11.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.3.p2.11.m2.1.1.3" xref="S4.SS1.3.p2.11.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.11.m2.1b"><apply id="S4.SS1.3.p2.11.m2.1.1.cmml" xref="S4.SS1.3.p2.11.m2.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.11.m2.1.1.1.cmml" xref="S4.SS1.3.p2.11.m2.1.1">superscript</csymbol><ci id="S4.SS1.3.p2.11.m2.1.1.2.cmml" xref="S4.SS1.3.p2.11.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.3.p2.11.m2.1.1.3.cmml" xref="S4.SS1.3.p2.11.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.11.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math> (<cite class="ltx_cite ltx_citemacro_citep">Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a></cite>, extending <cite class="ltx_cite ltx_citemacro_citep">Hesse, <a href="#bib.bib14" title="" class="ltx_ref">2001</a>; Chiu et al., <a href="#bib.bib4" title="" class="ltx_ref">2001</a></cite> for integers), we can compute <math id="S4.SS1.3.p2.12.m3.1" class="ltx_Math" alttext="h_{i}\triangleq\sum_{j=1}^{i}\tau_{ij}" display="inline"><semantics id="S4.SS1.3.p2.12.m3.1a"><mrow id="S4.SS1.3.p2.12.m3.1.1" xref="S4.SS1.3.p2.12.m3.1.1.cmml"><msub id="S4.SS1.3.p2.12.m3.1.1.2" xref="S4.SS1.3.p2.12.m3.1.1.2.cmml"><mi id="S4.SS1.3.p2.12.m3.1.1.2.2" xref="S4.SS1.3.p2.12.m3.1.1.2.2.cmml">h</mi><mi id="S4.SS1.3.p2.12.m3.1.1.2.3" xref="S4.SS1.3.p2.12.m3.1.1.2.3.cmml">i</mi></msub><mo rspace="0.111em" id="S4.SS1.3.p2.12.m3.1.1.1" xref="S4.SS1.3.p2.12.m3.1.1.1.cmml">≜</mo><mrow id="S4.SS1.3.p2.12.m3.1.1.3" xref="S4.SS1.3.p2.12.m3.1.1.3.cmml"><msubsup id="S4.SS1.3.p2.12.m3.1.1.3.1" xref="S4.SS1.3.p2.12.m3.1.1.3.1.cmml"><mo id="S4.SS1.3.p2.12.m3.1.1.3.1.2.2" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.2.cmml">∑</mo><mrow id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.cmml"><mi id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.2" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.2.cmml">j</mi><mo id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.1" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.1.cmml">=</mo><mn id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.3" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="S4.SS1.3.p2.12.m3.1.1.3.1.3" xref="S4.SS1.3.p2.12.m3.1.1.3.1.3.cmml">i</mi></msubsup><msub id="S4.SS1.3.p2.12.m3.1.1.3.2" xref="S4.SS1.3.p2.12.m3.1.1.3.2.cmml"><mi id="S4.SS1.3.p2.12.m3.1.1.3.2.2" xref="S4.SS1.3.p2.12.m3.1.1.3.2.2.cmml">τ</mi><mrow id="S4.SS1.3.p2.12.m3.1.1.3.2.3" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.cmml"><mi id="S4.SS1.3.p2.12.m3.1.1.3.2.3.2" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.SS1.3.p2.12.m3.1.1.3.2.3.1" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.1.cmml">​</mo><mi id="S4.SS1.3.p2.12.m3.1.1.3.2.3.3" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.3.cmml">j</mi></mrow></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.12.m3.1b"><apply id="S4.SS1.3.p2.12.m3.1.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1"><ci id="S4.SS1.3.p2.12.m3.1.1.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.1">≜</ci><apply id="S4.SS1.3.p2.12.m3.1.1.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.2"><csymbol cd="ambiguous" id="S4.SS1.3.p2.12.m3.1.1.2.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.2">subscript</csymbol><ci id="S4.SS1.3.p2.12.m3.1.1.2.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.2.2">ℎ</ci><ci id="S4.SS1.3.p2.12.m3.1.1.2.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.2.3">𝑖</ci></apply><apply id="S4.SS1.3.p2.12.m3.1.1.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3"><apply id="S4.SS1.3.p2.12.m3.1.1.3.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.12.m3.1.1.3.1.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1">superscript</csymbol><apply id="S4.SS1.3.p2.12.m3.1.1.3.1.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.12.m3.1.1.3.1.2.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1">subscript</csymbol><sum id="S4.SS1.3.p2.12.m3.1.1.3.1.2.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.2"></sum><apply id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3"><eq id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.1"></eq><ci id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.2">𝑗</ci><cn type="integer" id="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="S4.SS1.3.p2.12.m3.1.1.3.1.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.1.3">𝑖</ci></apply><apply id="S4.SS1.3.p2.12.m3.1.1.3.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2"><csymbol cd="ambiguous" id="S4.SS1.3.p2.12.m3.1.1.3.2.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2">subscript</csymbol><ci id="S4.SS1.3.p2.12.m3.1.1.3.2.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2.2">𝜏</ci><apply id="S4.SS1.3.p2.12.m3.1.1.3.2.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3"><times id="S4.SS1.3.p2.12.m3.1.1.3.2.3.1.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.1"></times><ci id="S4.SS1.3.p2.12.m3.1.1.3.2.3.2.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.2">𝑖</ci><ci id="S4.SS1.3.p2.12.m3.1.1.3.2.3.3.cmml" xref="S4.SS1.3.p2.12.m3.1.1.3.2.3.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.12.m3.1c">h_{i}\triangleq\sum_{j=1}^{i}\tau_{ij}</annotation></semantics></math>.
Finally, we compute <math id="S4.SS1.3.p2.13.m4.1" class="ltx_Math" alttext="y_{i}" display="inline"><semantics id="S4.SS1.3.p2.13.m4.1a"><msub id="S4.SS1.3.p2.13.m4.1.1" xref="S4.SS1.3.p2.13.m4.1.1.cmml"><mi id="S4.SS1.3.p2.13.m4.1.1.2" xref="S4.SS1.3.p2.13.m4.1.1.2.cmml">y</mi><mi id="S4.SS1.3.p2.13.m4.1.1.3" xref="S4.SS1.3.p2.13.m4.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.13.m4.1b"><apply id="S4.SS1.3.p2.13.m4.1.1.cmml" xref="S4.SS1.3.p2.13.m4.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.13.m4.1.1.1.cmml" xref="S4.SS1.3.p2.13.m4.1.1">subscript</csymbol><ci id="S4.SS1.3.p2.13.m4.1.1.2.cmml" xref="S4.SS1.3.p2.13.m4.1.1.2">𝑦</ci><ci id="S4.SS1.3.p2.13.m4.1.1.3.cmml" xref="S4.SS1.3.p2.13.m4.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.13.m4.1c">y_{i}</annotation></semantics></math> from <math id="S4.SS1.3.p2.14.m5.1" class="ltx_Math" alttext="x_{i}" display="inline"><semantics id="S4.SS1.3.p2.14.m5.1a"><msub id="S4.SS1.3.p2.14.m5.1.1" xref="S4.SS1.3.p2.14.m5.1.1.cmml"><mi id="S4.SS1.3.p2.14.m5.1.1.2" xref="S4.SS1.3.p2.14.m5.1.1.2.cmml">x</mi><mi id="S4.SS1.3.p2.14.m5.1.1.3" xref="S4.SS1.3.p2.14.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.14.m5.1b"><apply id="S4.SS1.3.p2.14.m5.1.1.cmml" xref="S4.SS1.3.p2.14.m5.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.14.m5.1.1.1.cmml" xref="S4.SS1.3.p2.14.m5.1.1">subscript</csymbol><ci id="S4.SS1.3.p2.14.m5.1.1.2.cmml" xref="S4.SS1.3.p2.14.m5.1.1.2">𝑥</ci><ci id="S4.SS1.3.p2.14.m5.1.1.3.cmml" xref="S4.SS1.3.p2.14.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.14.m5.1c">x_{i}</annotation></semantics></math> and <math id="S4.SS1.3.p2.15.m6.1" class="ltx_Math" alttext="h_{i}" display="inline"><semantics id="S4.SS1.3.p2.15.m6.1a"><msub id="S4.SS1.3.p2.15.m6.1.1" xref="S4.SS1.3.p2.15.m6.1.1.cmml"><mi id="S4.SS1.3.p2.15.m6.1.1.2" xref="S4.SS1.3.p2.15.m6.1.1.2.cmml">h</mi><mi id="S4.SS1.3.p2.15.m6.1.1.3" xref="S4.SS1.3.p2.15.m6.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.15.m6.1b"><apply id="S4.SS1.3.p2.15.m6.1.1.cmml" xref="S4.SS1.3.p2.15.m6.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.15.m6.1.1.1.cmml" xref="S4.SS1.3.p2.15.m6.1.1">subscript</csymbol><ci id="S4.SS1.3.p2.15.m6.1.1.2.cmml" xref="S4.SS1.3.p2.15.m6.1.1.2">ℎ</ci><ci id="S4.SS1.3.p2.15.m6.1.1.3.cmml" xref="S4.SS1.3.p2.15.m6.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.15.m6.1c">h_{i}</annotation></semantics></math> in <math id="S4.SS1.3.p2.16.m7.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS1.3.p2.16.m7.1a"><msup id="S4.SS1.3.p2.16.m7.1.1" xref="S4.SS1.3.p2.16.m7.1.1.cmml"><mi id="S4.SS1.3.p2.16.m7.1.1.2" xref="S4.SS1.3.p2.16.m7.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS1.3.p2.16.m7.1.1.3" xref="S4.SS1.3.p2.16.m7.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS1.3.p2.16.m7.1b"><apply id="S4.SS1.3.p2.16.m7.1.1.cmml" xref="S4.SS1.3.p2.16.m7.1.1"><csymbol cd="ambiguous" id="S4.SS1.3.p2.16.m7.1.1.1.cmml" xref="S4.SS1.3.p2.16.m7.1.1">superscript</csymbol><ci id="S4.SS1.3.p2.16.m7.1.1.2.cmml" xref="S4.SS1.3.p2.16.m7.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS1.3.p2.16.m7.1.1.3.cmml" xref="S4.SS1.3.p2.16.m7.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.3.p2.16.m7.1c">\mathsf{TC}^{0}</annotation></semantics></math> via simple arithmetic operations, completing the proof.
∎</p>
</div>
</div>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Simulating S6 with a <math id="S4.SS2.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS2.1.m1.1b"><msup id="S4.SS2.1.m1.1.1" xref="S4.SS2.1.m1.1.1.cmml"><mi id="S4.SS2.1.m1.1.1.2" xref="S4.SS2.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS2.1.m1.1.1.3" xref="S4.SS2.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS2.1.m1.1c"><apply id="S4.SS2.1.m1.1.1.cmml" xref="S4.SS2.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS2.1.m1.1.1.1.cmml" xref="S4.SS2.1.m1.1.1">superscript</csymbol><ci id="S4.SS2.1.m1.1.1.2.cmml" xref="S4.SS2.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS2.1.m1.1.1.3.cmml" xref="S4.SS2.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.1.m1.1d">\mathsf{TC}^{0}</annotation></semantics></math> Circuit Family</h3>

<div id="S4.Thmtheorem2" class="ltx_theorem ltx_theorem_theorem">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S4.Thmtheorem2.1.1.1" class="ltx_text ltx_font_bold">Theorem 4.2</span></span><span id="S4.Thmtheorem2.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S4.Thmtheorem2.p1" class="ltx_para">
<p id="S4.Thmtheorem2.p1.4" class="ltx_p"><span id="S4.Thmtheorem2.p1.4.4" class="ltx_text ltx_font_italic">For any log-precision SSM <math id="S4.Thmtheorem2.p1.1.1.m1.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S4.Thmtheorem2.p1.1.1.m1.1a"><mi id="S4.Thmtheorem2.p1.1.1.m1.1.1" xref="S4.Thmtheorem2.p1.1.1.m1.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem2.p1.1.1.m1.1b"><ci id="S4.Thmtheorem2.p1.1.1.m1.1.1.cmml" xref="S4.Thmtheorem2.p1.1.1.m1.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem2.p1.1.1.m1.1c">M</annotation></semantics></math> with the S6 architecture, there exists an <math id="S4.Thmtheorem2.p1.2.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.Thmtheorem2.p1.2.2.m2.1a"><mi id="S4.Thmtheorem2.p1.2.2.m2.1.1" xref="S4.Thmtheorem2.p1.2.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem2.p1.2.2.m2.1b"><ci id="S4.Thmtheorem2.p1.2.2.m2.1.1.cmml" xref="S4.Thmtheorem2.p1.2.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem2.p1.2.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.Thmtheorem2.p1.3.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.Thmtheorem2.p1.3.3.m3.1a"><msup id="S4.Thmtheorem2.p1.3.3.m3.1.1" xref="S4.Thmtheorem2.p1.3.3.m3.1.1.cmml"><mi id="S4.Thmtheorem2.p1.3.3.m3.1.1.2" xref="S4.Thmtheorem2.p1.3.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.Thmtheorem2.p1.3.3.m3.1.1.3" xref="S4.Thmtheorem2.p1.3.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem2.p1.3.3.m3.1b"><apply id="S4.Thmtheorem2.p1.3.3.m3.1.1.cmml" xref="S4.Thmtheorem2.p1.3.3.m3.1.1"><csymbol cd="ambiguous" id="S4.Thmtheorem2.p1.3.3.m3.1.1.1.cmml" xref="S4.Thmtheorem2.p1.3.3.m3.1.1">superscript</csymbol><ci id="S4.Thmtheorem2.p1.3.3.m3.1.1.2.cmml" xref="S4.Thmtheorem2.p1.3.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.Thmtheorem2.p1.3.3.m3.1.1.3.cmml" xref="S4.Thmtheorem2.p1.3.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem2.p1.3.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit family that computes the same function as <math id="S4.Thmtheorem2.p1.4.4.m4.1" class="ltx_Math" alttext="M" display="inline"><semantics id="S4.Thmtheorem2.p1.4.4.m4.1a"><mi id="S4.Thmtheorem2.p1.4.4.m4.1.1" xref="S4.Thmtheorem2.p1.4.4.m4.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem2.p1.4.4.m4.1b"><ci id="S4.Thmtheorem2.p1.4.4.m4.1.1.cmml" xref="S4.Thmtheorem2.p1.4.4.m4.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem2.p1.4.4.m4.1c">M</annotation></semantics></math>’s convolutional form.</span></p>
</div>
</div>
<div id="S4.SS2.2" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S4.SS2.2.p1" class="ltx_para">
<p id="S4.SS2.2.p1.9" class="ltx_p">The convolutional form of the S6 layer is given in Eqn. (<a href="#S2.E4" title="Equation 4 ‣ Definition 2.2 (S6 layer). ‣ 2.1 Architecture of State-Space Models ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>).
For each <math id="S4.SS2.2.p1.1.m1.1" class="ltx_Math" alttext="i" display="inline"><semantics id="S4.SS2.2.p1.1.m1.1a"><mi id="S4.SS2.2.p1.1.m1.1.1" xref="S4.SS2.2.p1.1.m1.1.1.cmml">i</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.1.m1.1b"><ci id="S4.SS2.2.p1.1.m1.1.1.cmml" xref="S4.SS2.2.p1.1.m1.1.1">𝑖</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.1.m1.1c">i</annotation></semantics></math> we print an <math id="S4.SS2.2.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.SS2.2.p1.2.m2.1a"><mi id="S4.SS2.2.p1.2.m2.1.1" xref="S4.SS2.2.p1.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.2.m2.1b"><ci id="S4.SS2.2.p1.2.m2.1.1.cmml" xref="S4.SS2.2.p1.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.SS2.2.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS2.2.p1.3.m3.1a"><msup id="S4.SS2.2.p1.3.m3.1.1" xref="S4.SS2.2.p1.3.m3.1.1.cmml"><mi id="S4.SS2.2.p1.3.m3.1.1.2" xref="S4.SS2.2.p1.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS2.2.p1.3.m3.1.1.3" xref="S4.SS2.2.p1.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.3.m3.1b"><apply id="S4.SS2.2.p1.3.m3.1.1.cmml" xref="S4.SS2.2.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.3.m3.1.1.1.cmml" xref="S4.SS2.2.p1.3.m3.1.1">superscript</csymbol><ci id="S4.SS2.2.p1.3.m3.1.1.2.cmml" xref="S4.SS2.2.p1.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS2.2.p1.3.m3.1.1.3.cmml" xref="S4.SS2.2.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit computing <math id="S4.SS2.2.p1.4.m4.2" class="ltx_Math" alttext="\delta_{i},B_{i}" display="inline"><semantics id="S4.SS2.2.p1.4.m4.2a"><mrow id="S4.SS2.2.p1.4.m4.2.2.2" xref="S4.SS2.2.p1.4.m4.2.2.3.cmml"><msub id="S4.SS2.2.p1.4.m4.1.1.1.1" xref="S4.SS2.2.p1.4.m4.1.1.1.1.cmml"><mi id="S4.SS2.2.p1.4.m4.1.1.1.1.2" xref="S4.SS2.2.p1.4.m4.1.1.1.1.2.cmml">δ</mi><mi id="S4.SS2.2.p1.4.m4.1.1.1.1.3" xref="S4.SS2.2.p1.4.m4.1.1.1.1.3.cmml">i</mi></msub><mo id="S4.SS2.2.p1.4.m4.2.2.2.3" xref="S4.SS2.2.p1.4.m4.2.2.3.cmml">,</mo><msub id="S4.SS2.2.p1.4.m4.2.2.2.2" xref="S4.SS2.2.p1.4.m4.2.2.2.2.cmml"><mi id="S4.SS2.2.p1.4.m4.2.2.2.2.2" xref="S4.SS2.2.p1.4.m4.2.2.2.2.2.cmml">B</mi><mi id="S4.SS2.2.p1.4.m4.2.2.2.2.3" xref="S4.SS2.2.p1.4.m4.2.2.2.2.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.4.m4.2b"><list id="S4.SS2.2.p1.4.m4.2.2.3.cmml" xref="S4.SS2.2.p1.4.m4.2.2.2"><apply id="S4.SS2.2.p1.4.m4.1.1.1.1.cmml" xref="S4.SS2.2.p1.4.m4.1.1.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.4.m4.1.1.1.1.1.cmml" xref="S4.SS2.2.p1.4.m4.1.1.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.4.m4.1.1.1.1.2.cmml" xref="S4.SS2.2.p1.4.m4.1.1.1.1.2">𝛿</ci><ci id="S4.SS2.2.p1.4.m4.1.1.1.1.3.cmml" xref="S4.SS2.2.p1.4.m4.1.1.1.1.3">𝑖</ci></apply><apply id="S4.SS2.2.p1.4.m4.2.2.2.2.cmml" xref="S4.SS2.2.p1.4.m4.2.2.2.2"><csymbol cd="ambiguous" id="S4.SS2.2.p1.4.m4.2.2.2.2.1.cmml" xref="S4.SS2.2.p1.4.m4.2.2.2.2">subscript</csymbol><ci id="S4.SS2.2.p1.4.m4.2.2.2.2.2.cmml" xref="S4.SS2.2.p1.4.m4.2.2.2.2.2">𝐵</ci><ci id="S4.SS2.2.p1.4.m4.2.2.2.2.3.cmml" xref="S4.SS2.2.p1.4.m4.2.2.2.2.3">𝑖</ci></apply></list></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.4.m4.2c">\delta_{i},B_{i}</annotation></semantics></math> and <math id="S4.SS2.2.p1.5.m5.1" class="ltx_Math" alttext="C_{i}" display="inline"><semantics id="S4.SS2.2.p1.5.m5.1a"><msub id="S4.SS2.2.p1.5.m5.1.1" xref="S4.SS2.2.p1.5.m5.1.1.cmml"><mi id="S4.SS2.2.p1.5.m5.1.1.2" xref="S4.SS2.2.p1.5.m5.1.1.2.cmml">C</mi><mi id="S4.SS2.2.p1.5.m5.1.1.3" xref="S4.SS2.2.p1.5.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.5.m5.1b"><apply id="S4.SS2.2.p1.5.m5.1.1.cmml" xref="S4.SS2.2.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.5.m5.1.1.1.cmml" xref="S4.SS2.2.p1.5.m5.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.5.m5.1.1.2.cmml" xref="S4.SS2.2.p1.5.m5.1.1.2">𝐶</ci><ci id="S4.SS2.2.p1.5.m5.1.1.3.cmml" xref="S4.SS2.2.p1.5.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.5.m5.1c">C_{i}</annotation></semantics></math>.
Since <math id="S4.SS2.2.p1.6.m6.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S4.SS2.2.p1.6.m6.1a"><mi id="S4.SS2.2.p1.6.m6.1.1" xref="S4.SS2.2.p1.6.m6.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.6.m6.1b"><ci id="S4.SS2.2.p1.6.m6.1.1.cmml" xref="S4.SS2.2.p1.6.m6.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.6.m6.1c">A</annotation></semantics></math> is diagonal, iterated matrix multiplication is reducible to iterated scalar multiplication, which is in <math id="S4.SS2.2.p1.7.m7.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S4.SS2.2.p1.7.m7.1a"><mi id="S4.SS2.2.p1.7.m7.1.1" xref="S4.SS2.2.p1.7.m7.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.7.m7.1b"><ci id="S4.SS2.2.p1.7.m7.1.1.cmml" xref="S4.SS2.2.p1.7.m7.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.7.m7.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S4.SS2.2.p1.8.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS2.2.p1.8.m8.1a"><msup id="S4.SS2.2.p1.8.m8.1.1" xref="S4.SS2.2.p1.8.m8.1.1.cmml"><mi id="S4.SS2.2.p1.8.m8.1.1.2" xref="S4.SS2.2.p1.8.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS2.2.p1.8.m8.1.1.3" xref="S4.SS2.2.p1.8.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.8.m8.1b"><apply id="S4.SS2.2.p1.8.m8.1.1.cmml" xref="S4.SS2.2.p1.8.m8.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.8.m8.1.1.1.cmml" xref="S4.SS2.2.p1.8.m8.1.1">superscript</csymbol><ci id="S4.SS2.2.p1.8.m8.1.1.2.cmml" xref="S4.SS2.2.p1.8.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS2.2.p1.8.m8.1.1.3.cmml" xref="S4.SS2.2.p1.8.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.8.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math> (<a href="#S2.Thmtheorem4" title="Lemma 2.4 (Float matrix power). ‣ 2.2 Numeric Datatype ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">2.4</span></a>).
so we can compute the following in <math id="S4.SS2.2.p1.9.m9.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS2.2.p1.9.m9.1a"><msup id="S4.SS2.2.p1.9.m9.1.1" xref="S4.SS2.2.p1.9.m9.1.1.cmml"><mi id="S4.SS2.2.p1.9.m9.1.1.2" xref="S4.SS2.2.p1.9.m9.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS2.2.p1.9.m9.1.1.3" xref="S4.SS2.2.p1.9.m9.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.9.m9.1b"><apply id="S4.SS2.2.p1.9.m9.1.1.cmml" xref="S4.SS2.2.p1.9.m9.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.9.m9.1.1.1.cmml" xref="S4.SS2.2.p1.9.m9.1.1">superscript</csymbol><ci id="S4.SS2.2.p1.9.m9.1.1.2.cmml" xref="S4.SS2.2.p1.9.m9.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS2.2.p1.9.m9.1.1.3.cmml" xref="S4.SS2.2.p1.9.m9.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.9.m9.1c">\mathsf{TC}^{0}</annotation></semantics></math>:</p>
<table id="S4.Ex4" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S4.Ex4.m1.2" class="ltx_Math" alttext="\pi_{ij}\triangleq\prod_{k=j+1}^{i}\exp(\delta_{k}A)." display="block"><semantics id="S4.Ex4.m1.2a"><mrow id="S4.Ex4.m1.2.2.1" xref="S4.Ex4.m1.2.2.1.1.cmml"><mrow id="S4.Ex4.m1.2.2.1.1" xref="S4.Ex4.m1.2.2.1.1.cmml"><msub id="S4.Ex4.m1.2.2.1.1.3" xref="S4.Ex4.m1.2.2.1.1.3.cmml"><mi id="S4.Ex4.m1.2.2.1.1.3.2" xref="S4.Ex4.m1.2.2.1.1.3.2.cmml">π</mi><mrow id="S4.Ex4.m1.2.2.1.1.3.3" xref="S4.Ex4.m1.2.2.1.1.3.3.cmml"><mi id="S4.Ex4.m1.2.2.1.1.3.3.2" xref="S4.Ex4.m1.2.2.1.1.3.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.Ex4.m1.2.2.1.1.3.3.1" xref="S4.Ex4.m1.2.2.1.1.3.3.1.cmml">​</mo><mi id="S4.Ex4.m1.2.2.1.1.3.3.3" xref="S4.Ex4.m1.2.2.1.1.3.3.3.cmml">j</mi></mrow></msub><mo rspace="0.111em" id="S4.Ex4.m1.2.2.1.1.2" xref="S4.Ex4.m1.2.2.1.1.2.cmml">≜</mo><mrow id="S4.Ex4.m1.2.2.1.1.1" xref="S4.Ex4.m1.2.2.1.1.1.cmml"><munderover id="S4.Ex4.m1.2.2.1.1.1.2" xref="S4.Ex4.m1.2.2.1.1.1.2.cmml"><mo movablelimits="false" id="S4.Ex4.m1.2.2.1.1.1.2.2.2" xref="S4.Ex4.m1.2.2.1.1.1.2.2.2.cmml">∏</mo><mrow id="S4.Ex4.m1.2.2.1.1.1.2.2.3" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.cmml"><mi id="S4.Ex4.m1.2.2.1.1.1.2.2.3.2" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.2.cmml">k</mi><mo id="S4.Ex4.m1.2.2.1.1.1.2.2.3.1" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.1.cmml">=</mo><mrow id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.cmml"><mi id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.2" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.2.cmml">j</mi><mo id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.1" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.1.cmml">+</mo><mn id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.3" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.3.cmml">1</mn></mrow></mrow><mi id="S4.Ex4.m1.2.2.1.1.1.2.3" xref="S4.Ex4.m1.2.2.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S4.Ex4.m1.2.2.1.1.1.1.1" xref="S4.Ex4.m1.2.2.1.1.1.1.2.cmml"><mi id="S4.Ex4.m1.1.1" xref="S4.Ex4.m1.1.1.cmml">exp</mi><mo id="S4.Ex4.m1.2.2.1.1.1.1.1a" xref="S4.Ex4.m1.2.2.1.1.1.1.2.cmml">⁡</mo><mrow id="S4.Ex4.m1.2.2.1.1.1.1.1.1" xref="S4.Ex4.m1.2.2.1.1.1.1.2.cmml"><mo stretchy="false" id="S4.Ex4.m1.2.2.1.1.1.1.1.1.2" xref="S4.Ex4.m1.2.2.1.1.1.1.2.cmml">(</mo><mrow id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.cmml"><msub id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.cmml"><mi id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.2" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.2.cmml">δ</mi><mi id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.3" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.3.cmml">k</mi></msub><mo lspace="0em" rspace="0em" id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.1" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.1.cmml">​</mo><mi id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.3" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.3.cmml">A</mi></mrow><mo stretchy="false" id="S4.Ex4.m1.2.2.1.1.1.1.1.1.3" xref="S4.Ex4.m1.2.2.1.1.1.1.2.cmml">)</mo></mrow></mrow></mrow></mrow><mo lspace="0em" id="S4.Ex4.m1.2.2.1.2" xref="S4.Ex4.m1.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.Ex4.m1.2b"><apply id="S4.Ex4.m1.2.2.1.1.cmml" xref="S4.Ex4.m1.2.2.1"><ci id="S4.Ex4.m1.2.2.1.1.2.cmml" xref="S4.Ex4.m1.2.2.1.1.2">≜</ci><apply id="S4.Ex4.m1.2.2.1.1.3.cmml" xref="S4.Ex4.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S4.Ex4.m1.2.2.1.1.3.1.cmml" xref="S4.Ex4.m1.2.2.1.1.3">subscript</csymbol><ci id="S4.Ex4.m1.2.2.1.1.3.2.cmml" xref="S4.Ex4.m1.2.2.1.1.3.2">𝜋</ci><apply id="S4.Ex4.m1.2.2.1.1.3.3.cmml" xref="S4.Ex4.m1.2.2.1.1.3.3"><times id="S4.Ex4.m1.2.2.1.1.3.3.1.cmml" xref="S4.Ex4.m1.2.2.1.1.3.3.1"></times><ci id="S4.Ex4.m1.2.2.1.1.3.3.2.cmml" xref="S4.Ex4.m1.2.2.1.1.3.3.2">𝑖</ci><ci id="S4.Ex4.m1.2.2.1.1.3.3.3.cmml" xref="S4.Ex4.m1.2.2.1.1.3.3.3">𝑗</ci></apply></apply><apply id="S4.Ex4.m1.2.2.1.1.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1"><apply id="S4.Ex4.m1.2.2.1.1.1.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2"><csymbol cd="ambiguous" id="S4.Ex4.m1.2.2.1.1.1.2.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2">superscript</csymbol><apply id="S4.Ex4.m1.2.2.1.1.1.2.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2"><csymbol cd="ambiguous" id="S4.Ex4.m1.2.2.1.1.1.2.2.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="S4.Ex4.m1.2.2.1.1.1.2.2.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.2">product</csymbol><apply id="S4.Ex4.m1.2.2.1.1.1.2.2.3.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3"><eq id="S4.Ex4.m1.2.2.1.1.1.2.2.3.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.1"></eq><ci id="S4.Ex4.m1.2.2.1.1.1.2.2.3.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.2">𝑘</ci><apply id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3"><plus id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.1"></plus><ci id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.2">𝑗</ci><cn type="integer" id="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.3.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.2.3.3.3">1</cn></apply></apply></apply><ci id="S4.Ex4.m1.2.2.1.1.1.2.3.cmml" xref="S4.Ex4.m1.2.2.1.1.1.2.3">𝑖</ci></apply><apply id="S4.Ex4.m1.2.2.1.1.1.1.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1"><exp id="S4.Ex4.m1.1.1.cmml" xref="S4.Ex4.m1.1.1"></exp><apply id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1"><times id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.1"></times><apply id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.1.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2">subscript</csymbol><ci id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.2.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.2">𝛿</ci><ci id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.3.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.2.3">𝑘</ci></apply><ci id="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.3.cmml" xref="S4.Ex4.m1.2.2.1.1.1.1.1.1.1.3">𝐴</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Ex4.m1.2c">\pi_{ij}\triangleq\prod_{k=j+1}^{i}\exp(\delta_{k}A).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S4.SS2.2.p1.17" class="ltx_p">We then conclude analogously to <a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorem</span> <span class="ltx_text ltx_ref_tag">4.1</span></a>, first computing <math id="S4.SS2.2.p1.10.m1.1" class="ltx_Math" alttext="\tau_{ij}" display="inline"><semantics id="S4.SS2.2.p1.10.m1.1a"><msub id="S4.SS2.2.p1.10.m1.1.1" xref="S4.SS2.2.p1.10.m1.1.1.cmml"><mi id="S4.SS2.2.p1.10.m1.1.1.2" xref="S4.SS2.2.p1.10.m1.1.1.2.cmml">τ</mi><mrow id="S4.SS2.2.p1.10.m1.1.1.3" xref="S4.SS2.2.p1.10.m1.1.1.3.cmml"><mi id="S4.SS2.2.p1.10.m1.1.1.3.2" xref="S4.SS2.2.p1.10.m1.1.1.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.SS2.2.p1.10.m1.1.1.3.1" xref="S4.SS2.2.p1.10.m1.1.1.3.1.cmml">​</mo><mi id="S4.SS2.2.p1.10.m1.1.1.3.3" xref="S4.SS2.2.p1.10.m1.1.1.3.3.cmml">j</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.10.m1.1b"><apply id="S4.SS2.2.p1.10.m1.1.1.cmml" xref="S4.SS2.2.p1.10.m1.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.10.m1.1.1.1.cmml" xref="S4.SS2.2.p1.10.m1.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.10.m1.1.1.2.cmml" xref="S4.SS2.2.p1.10.m1.1.1.2">𝜏</ci><apply id="S4.SS2.2.p1.10.m1.1.1.3.cmml" xref="S4.SS2.2.p1.10.m1.1.1.3"><times id="S4.SS2.2.p1.10.m1.1.1.3.1.cmml" xref="S4.SS2.2.p1.10.m1.1.1.3.1"></times><ci id="S4.SS2.2.p1.10.m1.1.1.3.2.cmml" xref="S4.SS2.2.p1.10.m1.1.1.3.2">𝑖</ci><ci id="S4.SS2.2.p1.10.m1.1.1.3.3.cmml" xref="S4.SS2.2.p1.10.m1.1.1.3.3">𝑗</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.10.m1.1c">\tau_{ij}</annotation></semantics></math> from <math id="S4.SS2.2.p1.11.m2.1" class="ltx_Math" alttext="\pi_{ij}" display="inline"><semantics id="S4.SS2.2.p1.11.m2.1a"><msub id="S4.SS2.2.p1.11.m2.1.1" xref="S4.SS2.2.p1.11.m2.1.1.cmml"><mi id="S4.SS2.2.p1.11.m2.1.1.2" xref="S4.SS2.2.p1.11.m2.1.1.2.cmml">π</mi><mrow id="S4.SS2.2.p1.11.m2.1.1.3" xref="S4.SS2.2.p1.11.m2.1.1.3.cmml"><mi id="S4.SS2.2.p1.11.m2.1.1.3.2" xref="S4.SS2.2.p1.11.m2.1.1.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.SS2.2.p1.11.m2.1.1.3.1" xref="S4.SS2.2.p1.11.m2.1.1.3.1.cmml">​</mo><mi id="S4.SS2.2.p1.11.m2.1.1.3.3" xref="S4.SS2.2.p1.11.m2.1.1.3.3.cmml">j</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.11.m2.1b"><apply id="S4.SS2.2.p1.11.m2.1.1.cmml" xref="S4.SS2.2.p1.11.m2.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.11.m2.1.1.1.cmml" xref="S4.SS2.2.p1.11.m2.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.11.m2.1.1.2.cmml" xref="S4.SS2.2.p1.11.m2.1.1.2">𝜋</ci><apply id="S4.SS2.2.p1.11.m2.1.1.3.cmml" xref="S4.SS2.2.p1.11.m2.1.1.3"><times id="S4.SS2.2.p1.11.m2.1.1.3.1.cmml" xref="S4.SS2.2.p1.11.m2.1.1.3.1"></times><ci id="S4.SS2.2.p1.11.m2.1.1.3.2.cmml" xref="S4.SS2.2.p1.11.m2.1.1.3.2">𝑖</ci><ci id="S4.SS2.2.p1.11.m2.1.1.3.3.cmml" xref="S4.SS2.2.p1.11.m2.1.1.3.3">𝑗</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.11.m2.1c">\pi_{ij}</annotation></semantics></math> and <math id="S4.SS2.2.p1.12.m3.1" class="ltx_Math" alttext="B_{i}" display="inline"><semantics id="S4.SS2.2.p1.12.m3.1a"><msub id="S4.SS2.2.p1.12.m3.1.1" xref="S4.SS2.2.p1.12.m3.1.1.cmml"><mi id="S4.SS2.2.p1.12.m3.1.1.2" xref="S4.SS2.2.p1.12.m3.1.1.2.cmml">B</mi><mi id="S4.SS2.2.p1.12.m3.1.1.3" xref="S4.SS2.2.p1.12.m3.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.12.m3.1b"><apply id="S4.SS2.2.p1.12.m3.1.1.cmml" xref="S4.SS2.2.p1.12.m3.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.12.m3.1.1.1.cmml" xref="S4.SS2.2.p1.12.m3.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.12.m3.1.1.2.cmml" xref="S4.SS2.2.p1.12.m3.1.1.2">𝐵</ci><ci id="S4.SS2.2.p1.12.m3.1.1.3.cmml" xref="S4.SS2.2.p1.12.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.12.m3.1c">B_{i}</annotation></semantics></math>, then computing <math id="S4.SS2.2.p1.13.m4.1" class="ltx_Math" alttext="h_{i}=\sum_{j=1}^{i}\tau_{ij}" display="inline"><semantics id="S4.SS2.2.p1.13.m4.1a"><mrow id="S4.SS2.2.p1.13.m4.1.1" xref="S4.SS2.2.p1.13.m4.1.1.cmml"><msub id="S4.SS2.2.p1.13.m4.1.1.2" xref="S4.SS2.2.p1.13.m4.1.1.2.cmml"><mi id="S4.SS2.2.p1.13.m4.1.1.2.2" xref="S4.SS2.2.p1.13.m4.1.1.2.2.cmml">h</mi><mi id="S4.SS2.2.p1.13.m4.1.1.2.3" xref="S4.SS2.2.p1.13.m4.1.1.2.3.cmml">i</mi></msub><mo rspace="0.111em" id="S4.SS2.2.p1.13.m4.1.1.1" xref="S4.SS2.2.p1.13.m4.1.1.1.cmml">=</mo><mrow id="S4.SS2.2.p1.13.m4.1.1.3" xref="S4.SS2.2.p1.13.m4.1.1.3.cmml"><msubsup id="S4.SS2.2.p1.13.m4.1.1.3.1" xref="S4.SS2.2.p1.13.m4.1.1.3.1.cmml"><mo id="S4.SS2.2.p1.13.m4.1.1.3.1.2.2" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.2.cmml">∑</mo><mrow id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.cmml"><mi id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.2" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.2.cmml">j</mi><mo id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.1" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.1.cmml">=</mo><mn id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.3" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="S4.SS2.2.p1.13.m4.1.1.3.1.3" xref="S4.SS2.2.p1.13.m4.1.1.3.1.3.cmml">i</mi></msubsup><msub id="S4.SS2.2.p1.13.m4.1.1.3.2" xref="S4.SS2.2.p1.13.m4.1.1.3.2.cmml"><mi id="S4.SS2.2.p1.13.m4.1.1.3.2.2" xref="S4.SS2.2.p1.13.m4.1.1.3.2.2.cmml">τ</mi><mrow id="S4.SS2.2.p1.13.m4.1.1.3.2.3" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.cmml"><mi id="S4.SS2.2.p1.13.m4.1.1.3.2.3.2" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.2.cmml">i</mi><mo lspace="0em" rspace="0em" id="S4.SS2.2.p1.13.m4.1.1.3.2.3.1" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.1.cmml">​</mo><mi id="S4.SS2.2.p1.13.m4.1.1.3.2.3.3" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.3.cmml">j</mi></mrow></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.13.m4.1b"><apply id="S4.SS2.2.p1.13.m4.1.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1"><eq id="S4.SS2.2.p1.13.m4.1.1.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.1"></eq><apply id="S4.SS2.2.p1.13.m4.1.1.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.2"><csymbol cd="ambiguous" id="S4.SS2.2.p1.13.m4.1.1.2.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.2">subscript</csymbol><ci id="S4.SS2.2.p1.13.m4.1.1.2.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.2.2">ℎ</ci><ci id="S4.SS2.2.p1.13.m4.1.1.2.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.2.3">𝑖</ci></apply><apply id="S4.SS2.2.p1.13.m4.1.1.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3"><apply id="S4.SS2.2.p1.13.m4.1.1.3.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.13.m4.1.1.3.1.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1">superscript</csymbol><apply id="S4.SS2.2.p1.13.m4.1.1.3.1.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.13.m4.1.1.3.1.2.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1">subscript</csymbol><sum id="S4.SS2.2.p1.13.m4.1.1.3.1.2.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.2"></sum><apply id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3"><eq id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.1"></eq><ci id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.2">𝑗</ci><cn type="integer" id="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="S4.SS2.2.p1.13.m4.1.1.3.1.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.1.3">𝑖</ci></apply><apply id="S4.SS2.2.p1.13.m4.1.1.3.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2"><csymbol cd="ambiguous" id="S4.SS2.2.p1.13.m4.1.1.3.2.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2">subscript</csymbol><ci id="S4.SS2.2.p1.13.m4.1.1.3.2.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2.2">𝜏</ci><apply id="S4.SS2.2.p1.13.m4.1.1.3.2.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3"><times id="S4.SS2.2.p1.13.m4.1.1.3.2.3.1.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.1"></times><ci id="S4.SS2.2.p1.13.m4.1.1.3.2.3.2.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.2">𝑖</ci><ci id="S4.SS2.2.p1.13.m4.1.1.3.2.3.3.cmml" xref="S4.SS2.2.p1.13.m4.1.1.3.2.3.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.13.m4.1c">h_{i}=\sum_{j=1}^{i}\tau_{ij}</annotation></semantics></math>, and finally computing <math id="S4.SS2.2.p1.14.m5.1" class="ltx_Math" alttext="y_{i}" display="inline"><semantics id="S4.SS2.2.p1.14.m5.1a"><msub id="S4.SS2.2.p1.14.m5.1.1" xref="S4.SS2.2.p1.14.m5.1.1.cmml"><mi id="S4.SS2.2.p1.14.m5.1.1.2" xref="S4.SS2.2.p1.14.m5.1.1.2.cmml">y</mi><mi id="S4.SS2.2.p1.14.m5.1.1.3" xref="S4.SS2.2.p1.14.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.14.m5.1b"><apply id="S4.SS2.2.p1.14.m5.1.1.cmml" xref="S4.SS2.2.p1.14.m5.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.14.m5.1.1.1.cmml" xref="S4.SS2.2.p1.14.m5.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.14.m5.1.1.2.cmml" xref="S4.SS2.2.p1.14.m5.1.1.2">𝑦</ci><ci id="S4.SS2.2.p1.14.m5.1.1.3.cmml" xref="S4.SS2.2.p1.14.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.14.m5.1c">y_{i}</annotation></semantics></math> from <math id="S4.SS2.2.p1.15.m6.1" class="ltx_Math" alttext="x_{i},h_{i}," display="inline"><semantics id="S4.SS2.2.p1.15.m6.1a"><mrow id="S4.SS2.2.p1.15.m6.1.1.1"><mrow id="S4.SS2.2.p1.15.m6.1.1.1.1.2" xref="S4.SS2.2.p1.15.m6.1.1.1.1.3.cmml"><msub id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.cmml"><mi id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.2" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.2.cmml">x</mi><mi id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.3" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.3.cmml">i</mi></msub><mo id="S4.SS2.2.p1.15.m6.1.1.1.1.2.3" xref="S4.SS2.2.p1.15.m6.1.1.1.1.3.cmml">,</mo><msub id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.cmml"><mi id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.2" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.2.cmml">h</mi><mi id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.3" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.3.cmml">i</mi></msub></mrow><mo id="S4.SS2.2.p1.15.m6.1.1.1.2">,</mo></mrow><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.15.m6.1b"><list id="S4.SS2.2.p1.15.m6.1.1.1.1.3.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2"><apply id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.1.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.2.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.2">𝑥</ci><ci id="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.3.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2"><csymbol cd="ambiguous" id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.1.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2">subscript</csymbol><ci id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.2.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.2">ℎ</ci><ci id="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.3.cmml" xref="S4.SS2.2.p1.15.m6.1.1.1.1.2.2.3">𝑖</ci></apply></list></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.15.m6.1c">x_{i},h_{i},</annotation></semantics></math> and <math id="S4.SS2.2.p1.16.m7.1" class="ltx_Math" alttext="C_{i}" display="inline"><semantics id="S4.SS2.2.p1.16.m7.1a"><msub id="S4.SS2.2.p1.16.m7.1.1" xref="S4.SS2.2.p1.16.m7.1.1.cmml"><mi id="S4.SS2.2.p1.16.m7.1.1.2" xref="S4.SS2.2.p1.16.m7.1.1.2.cmml">C</mi><mi id="S4.SS2.2.p1.16.m7.1.1.3" xref="S4.SS2.2.p1.16.m7.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.16.m7.1b"><apply id="S4.SS2.2.p1.16.m7.1.1.cmml" xref="S4.SS2.2.p1.16.m7.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.16.m7.1.1.1.cmml" xref="S4.SS2.2.p1.16.m7.1.1">subscript</csymbol><ci id="S4.SS2.2.p1.16.m7.1.1.2.cmml" xref="S4.SS2.2.p1.16.m7.1.1.2">𝐶</ci><ci id="S4.SS2.2.p1.16.m7.1.1.3.cmml" xref="S4.SS2.2.p1.16.m7.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.16.m7.1c">C_{i}</annotation></semantics></math> in <math id="S4.SS2.2.p1.17.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS2.2.p1.17.m8.1a"><msup id="S4.SS2.2.p1.17.m8.1.1" xref="S4.SS2.2.p1.17.m8.1.1.cmml"><mi id="S4.SS2.2.p1.17.m8.1.1.2" xref="S4.SS2.2.p1.17.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS2.2.p1.17.m8.1.1.3" xref="S4.SS2.2.p1.17.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS2.2.p1.17.m8.1b"><apply id="S4.SS2.2.p1.17.m8.1.1.cmml" xref="S4.SS2.2.p1.17.m8.1.1"><csymbol cd="ambiguous" id="S4.SS2.2.p1.17.m8.1.1.1.cmml" xref="S4.SS2.2.p1.17.m8.1.1">superscript</csymbol><ci id="S4.SS2.2.p1.17.m8.1.1.2.cmml" xref="S4.SS2.2.p1.17.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS2.2.p1.17.m8.1.1.3.cmml" xref="S4.SS2.2.p1.17.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.2.p1.17.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math>.
∎</p>
</div>
</div>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>Discussion</h3>

<div id="S4.SS3.p1" class="ltx_para">
<p id="S4.SS3.p1.3" class="ltx_p"><a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorems</span> <span class="ltx_text ltx_ref_tag">4.1</span></a> and <a href="#S4.Thmtheorem2" title="Theorem 4.2. ‣ 4.2 Simulating S6 with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.2</span></a> establish that SSMs, like transformers, can only express solutions to problems in the class <math id="S4.SS3.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S4.SS3.p1.1.m1.1a"><msup id="S4.SS3.p1.1.m1.1.1" xref="S4.SS3.p1.1.m1.1.1.cmml"><mi id="S4.SS3.p1.1.m1.1.1.2" xref="S4.SS3.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S4.SS3.p1.1.m1.1.1.3" xref="S4.SS3.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.1.m1.1b"><apply id="S4.SS3.p1.1.m1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.1.m1.1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1">superscript</csymbol><ci id="S4.SS3.p1.1.m1.1.1.2.cmml" xref="S4.SS3.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S4.SS3.p1.1.m1.1.1.3.cmml" xref="S4.SS3.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>. This means that SSMs cannot solve <math id="S4.SS3.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S4.SS3.p1.2.m2.1a"><msup id="S4.SS3.p1.2.m2.1.1" xref="S4.SS3.p1.2.m2.1.1.cmml"><mi id="S4.SS3.p1.2.m2.1.1.2" xref="S4.SS3.p1.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S4.SS3.p1.2.m2.1.1.3" xref="S4.SS3.p1.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.2.m2.1b"><apply id="S4.SS3.p1.2.m2.1.1.cmml" xref="S4.SS3.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.2.m2.1.1.1.cmml" xref="S4.SS3.p1.2.m2.1.1">superscript</csymbol><ci id="S4.SS3.p1.2.m2.1.1.2.cmml" xref="S4.SS3.p1.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S4.SS3.p1.2.m2.1.1.3.cmml" xref="S4.SS3.p1.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math>-hard problems like evaluating boolean formulas or graph connectivity. In particular, it shows that they are limited as far as their state tracking capabilities as they are unable to compose permutations (solve the <math id="S4.SS3.p1.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S4.SS3.p1.3.m3.1a"><msub id="S4.SS3.p1.3.m3.1.1" xref="S4.SS3.p1.3.m3.1.1.cmml"><mi id="S4.SS3.p1.3.m3.1.1.2" xref="S4.SS3.p1.3.m3.1.1.2.cmml">S</mi><mn id="S4.SS3.p1.3.m3.1.1.3" xref="S4.SS3.p1.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.3.m3.1b"><apply id="S4.SS3.p1.3.m3.1.1.cmml" xref="S4.SS3.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S4.SS3.p1.3.m3.1.1.1.cmml" xref="S4.SS3.p1.3.m3.1.1">subscript</csymbol><ci id="S4.SS3.p1.3.m3.1.1.2.cmml" xref="S4.SS3.p1.3.m3.1.1.2">𝑆</ci><cn type="integer" id="S4.SS3.p1.3.m3.1.1.3.cmml" xref="S4.SS3.p1.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.3.m3.1c">S_{5}</annotation></semantics></math> word problem):</p>
</div>
<div id="S4.Thmtheorem3" class="ltx_theorem ltx_theorem_corollary">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S4.Thmtheorem3.1.1.1" class="ltx_text ltx_font_bold">Corollary 4.3</span></span><span id="S4.Thmtheorem3.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S4.Thmtheorem3.p1" class="ltx_para">
<p id="S4.Thmtheorem3.p1.3" class="ltx_p"><span id="S4.Thmtheorem3.p1.3.3" class="ltx_text ltx_font_italic">Assuming <math id="S4.Thmtheorem3.p1.1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}\neq\mathsf{NC}^{1}" display="inline"><semantics id="S4.Thmtheorem3.p1.1.1.m1.1a"><mrow id="S4.Thmtheorem3.p1.1.1.m1.1.1" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.cmml"><msup id="S4.Thmtheorem3.p1.1.1.m1.1.1.2" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2.cmml"><mi id="S4.Thmtheorem3.p1.1.1.m1.1.1.2.2" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2.2.cmml">𝖳𝖢</mi><mn id="S4.Thmtheorem3.p1.1.1.m1.1.1.2.3" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2.3.cmml">0</mn></msup><mo id="S4.Thmtheorem3.p1.1.1.m1.1.1.1" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.1.cmml">≠</mo><msup id="S4.Thmtheorem3.p1.1.1.m1.1.1.3" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3.cmml"><mi id="S4.Thmtheorem3.p1.1.1.m1.1.1.3.2" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3.2.cmml">𝖭𝖢</mi><mn id="S4.Thmtheorem3.p1.1.1.m1.1.1.3.3" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3.3.cmml">1</mn></msup></mrow><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem3.p1.1.1.m1.1b"><apply id="S4.Thmtheorem3.p1.1.1.m1.1.1.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1"><neq id="S4.Thmtheorem3.p1.1.1.m1.1.1.1.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.1"></neq><apply id="S4.Thmtheorem3.p1.1.1.m1.1.1.2.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2"><csymbol cd="ambiguous" id="S4.Thmtheorem3.p1.1.1.m1.1.1.2.1.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2">superscript</csymbol><ci id="S4.Thmtheorem3.p1.1.1.m1.1.1.2.2.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2.2">𝖳𝖢</ci><cn type="integer" id="S4.Thmtheorem3.p1.1.1.m1.1.1.2.3.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.2.3">0</cn></apply><apply id="S4.Thmtheorem3.p1.1.1.m1.1.1.3.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3"><csymbol cd="ambiguous" id="S4.Thmtheorem3.p1.1.1.m1.1.1.3.1.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3">superscript</csymbol><ci id="S4.Thmtheorem3.p1.1.1.m1.1.1.3.2.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3.2">𝖭𝖢</ci><cn type="integer" id="S4.Thmtheorem3.p1.1.1.m1.1.1.3.3.cmml" xref="S4.Thmtheorem3.p1.1.1.m1.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem3.p1.1.1.m1.1c">\mathsf{TC}^{0}\neq\mathsf{NC}^{1}</annotation></semantics></math>, no log-precision SSM with the S4 or S6 architecture can solve the word problem for <math id="S4.Thmtheorem3.p1.2.2.m2.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S4.Thmtheorem3.p1.2.2.m2.1a"><msub id="S4.Thmtheorem3.p1.2.2.m2.1.1" xref="S4.Thmtheorem3.p1.2.2.m2.1.1.cmml"><mi id="S4.Thmtheorem3.p1.2.2.m2.1.1.2" xref="S4.Thmtheorem3.p1.2.2.m2.1.1.2.cmml">S</mi><mn id="S4.Thmtheorem3.p1.2.2.m2.1.1.3" xref="S4.Thmtheorem3.p1.2.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem3.p1.2.2.m2.1b"><apply id="S4.Thmtheorem3.p1.2.2.m2.1.1.cmml" xref="S4.Thmtheorem3.p1.2.2.m2.1.1"><csymbol cd="ambiguous" id="S4.Thmtheorem3.p1.2.2.m2.1.1.1.cmml" xref="S4.Thmtheorem3.p1.2.2.m2.1.1">subscript</csymbol><ci id="S4.Thmtheorem3.p1.2.2.m2.1.1.2.cmml" xref="S4.Thmtheorem3.p1.2.2.m2.1.1.2">𝑆</ci><cn type="integer" id="S4.Thmtheorem3.p1.2.2.m2.1.1.3.cmml" xref="S4.Thmtheorem3.p1.2.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem3.p1.2.2.m2.1c">S_{5}</annotation></semantics></math> or any other <math id="S4.Thmtheorem3.p1.3.3.m3.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S4.Thmtheorem3.p1.3.3.m3.1a"><msup id="S4.Thmtheorem3.p1.3.3.m3.1.1" xref="S4.Thmtheorem3.p1.3.3.m3.1.1.cmml"><mi id="S4.Thmtheorem3.p1.3.3.m3.1.1.2" xref="S4.Thmtheorem3.p1.3.3.m3.1.1.2.cmml">𝖭𝖢</mi><mn id="S4.Thmtheorem3.p1.3.3.m3.1.1.3" xref="S4.Thmtheorem3.p1.3.3.m3.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S4.Thmtheorem3.p1.3.3.m3.1b"><apply id="S4.Thmtheorem3.p1.3.3.m3.1.1.cmml" xref="S4.Thmtheorem3.p1.3.3.m3.1.1"><csymbol cd="ambiguous" id="S4.Thmtheorem3.p1.3.3.m3.1.1.1.cmml" xref="S4.Thmtheorem3.p1.3.3.m3.1.1">superscript</csymbol><ci id="S4.Thmtheorem3.p1.3.3.m3.1.1.2.cmml" xref="S4.Thmtheorem3.p1.3.3.m3.1.1.2">𝖭𝖢</ci><cn type="integer" id="S4.Thmtheorem3.p1.3.3.m3.1.1.3.cmml" xref="S4.Thmtheorem3.p1.3.3.m3.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.Thmtheorem3.p1.3.3.m3.1c">\mathsf{NC}^{1}</annotation></semantics></math>-hard problem.</span></p>
</div>
</div>
<div id="S4.SS3.p2" class="ltx_para">
<p id="S4.SS3.p2.1" class="ltx_p">In contrast, RNNs can easily express <math id="S4.SS3.p2.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S4.SS3.p2.1.m1.1a"><msub id="S4.SS3.p2.1.m1.1.1" xref="S4.SS3.p2.1.m1.1.1.cmml"><mi id="S4.SS3.p2.1.m1.1.1.2" xref="S4.SS3.p2.1.m1.1.1.2.cmml">S</mi><mn id="S4.SS3.p2.1.m1.1.1.3" xref="S4.SS3.p2.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.1.m1.1b"><apply id="S4.SS3.p2.1.m1.1.1.cmml" xref="S4.SS3.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S4.SS3.p2.1.m1.1.1.1.cmml" xref="S4.SS3.p2.1.m1.1.1">subscript</csymbol><ci id="S4.SS3.p2.1.m1.1.1.2.cmml" xref="S4.SS3.p2.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S4.SS3.p2.1.m1.1.1.3.cmml" xref="S4.SS3.p2.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.1.m1.1c">S_{5}</annotation></semantics></math> via standard constructions that encode finite-state transitions into an RNN <cite class="ltx_cite ltx_citemacro_citep">(Minsky, <a href="#bib.bib27" title="" class="ltx_ref">1954</a>; Merrill, <a href="#bib.bib24" title="" class="ltx_ref">2019</a>)</cite>. This shows that SSMs cannot express some kinds of state tracking and recurrence that RNNs can. This tempers the claim from <cite class="ltx_cite ltx_citemacro_citet">Gu et al. (<a href="#bib.bib9" title="" class="ltx_ref">2021</a>, Lemma 3.2)</cite> that SSMs have the expressive power to simulate RNNs, which relied on the assumption that SSMs can have <em id="S4.SS3.p2.1.1" class="ltx_emph ltx_font_italic">infinite depth</em>. In a more realistic setting with a bounded number of layers, our results show SSMs cannot express many state-tracking problems, including those which can be solved by fixed-depth RNNs.</p>
</div>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Extending the Expressive Power of SSMs</h2>

<div id="S5.p1" class="ltx_para">
<p id="S5.p1.4" class="ltx_p">We have shown that S4 and S6, despite their seemingly “stateful” design, cannot express problems outside <math id="S5.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S5.p1.1.m1.1a"><msup id="S5.p1.1.m1.1.1" xref="S5.p1.1.m1.1.1.cmml"><mi id="S5.p1.1.m1.1.1.2" xref="S5.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S5.p1.1.m1.1.1.3" xref="S5.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S5.p1.1.m1.1b"><apply id="S5.p1.1.m1.1.1.cmml" xref="S5.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.p1.1.m1.1.1.1.cmml" xref="S5.p1.1.m1.1.1">superscript</csymbol><ci id="S5.p1.1.m1.1.1.2.cmml" xref="S5.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S5.p1.1.m1.1.1.3.cmml" xref="S5.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>, which includes state tracking like <math id="S5.p1.2.m2.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.p1.2.m2.1a"><msub id="S5.p1.2.m2.1.1" xref="S5.p1.2.m2.1.1.cmml"><mi id="S5.p1.2.m2.1.1.2" xref="S5.p1.2.m2.1.1.2.cmml">S</mi><mn id="S5.p1.2.m2.1.1.3" xref="S5.p1.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.p1.2.m2.1b"><apply id="S5.p1.2.m2.1.1.cmml" xref="S5.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S5.p1.2.m2.1.1.1.cmml" xref="S5.p1.2.m2.1.1">subscript</csymbol><ci id="S5.p1.2.m2.1.1.2.cmml" xref="S5.p1.2.m2.1.1.2">𝑆</ci><cn type="integer" id="S5.p1.2.m2.1.1.3.cmml" xref="S5.p1.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.2.m2.1c">S_{5}</annotation></semantics></math>. We show how SSMs can be extended to close the gap in expressive power with RNNs, allowing them to express <math id="S5.p1.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.p1.3.m3.1a"><msub id="S5.p1.3.m3.1.1" xref="S5.p1.3.m3.1.1.cmml"><mi id="S5.p1.3.m3.1.1.2" xref="S5.p1.3.m3.1.1.2.cmml">S</mi><mn id="S5.p1.3.m3.1.1.3" xref="S5.p1.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.p1.3.m3.1b"><apply id="S5.p1.3.m3.1.1.cmml" xref="S5.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S5.p1.3.m3.1.1.1.cmml" xref="S5.p1.3.m3.1.1">subscript</csymbol><ci id="S5.p1.3.m3.1.1.2.cmml" xref="S5.p1.3.m3.1.1.2">𝑆</ci><cn type="integer" id="S5.p1.3.m3.1.1.3.cmml" xref="S5.p1.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.3.m3.1c">S_{5}</annotation></semantics></math>.
Two simple extensions can bring about this increase in expressive power: adding a nonlinearity to make the SSM more like an RNN or allowing the <math id="S5.p1.4.m4.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S5.p1.4.m4.1a"><mi id="S5.p1.4.m4.1.1" xref="S5.p1.4.m4.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S5.p1.4.m4.1b"><ci id="S5.p1.4.m4.1.1.cmml" xref="S5.p1.4.m4.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.4.m4.1c">A</annotation></semantics></math> matrix to be input-dependent to make the SSM more like a weighted finite automaton (WFA; <cite class="ltx_cite ltx_citemacro_citep">Mohri, <a href="#bib.bib28" title="" class="ltx_ref">2009</a></cite>).</p>
</div>
<section id="S5.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.1 </span>Via Nonlinearities</h3>

<div id="S5.SS1.p1" class="ltx_para">
<p id="S5.SS1.p1.2" class="ltx_p">Concretely, an <em id="S5.SS1.p1.2.1" class="ltx_emph ltx_font_italic">RNN-SSM layer</em> with a step activation function can be defined as follows:</p>
<table id="S5.E5" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S5.E5.m1.1" class="ltx_Math" alttext="h_{i}=\mathrm{sgn}\left(Ah_{i-1}+Bx_{i}\right)." display="block"><semantics id="S5.E5.m1.1a"><mrow id="S5.E5.m1.1.1.1" xref="S5.E5.m1.1.1.1.1.cmml"><mrow id="S5.E5.m1.1.1.1.1" xref="S5.E5.m1.1.1.1.1.cmml"><msub id="S5.E5.m1.1.1.1.1.3" xref="S5.E5.m1.1.1.1.1.3.cmml"><mi id="S5.E5.m1.1.1.1.1.3.2" xref="S5.E5.m1.1.1.1.1.3.2.cmml">h</mi><mi id="S5.E5.m1.1.1.1.1.3.3" xref="S5.E5.m1.1.1.1.1.3.3.cmml">i</mi></msub><mo id="S5.E5.m1.1.1.1.1.2" xref="S5.E5.m1.1.1.1.1.2.cmml">=</mo><mrow id="S5.E5.m1.1.1.1.1.1" xref="S5.E5.m1.1.1.1.1.1.cmml"><mi id="S5.E5.m1.1.1.1.1.1.3" xref="S5.E5.m1.1.1.1.1.1.3.cmml">sgn</mi><mo lspace="0em" rspace="0em" id="S5.E5.m1.1.1.1.1.1.2" xref="S5.E5.m1.1.1.1.1.1.2.cmml">​</mo><mrow id="S5.E5.m1.1.1.1.1.1.1.1" xref="S5.E5.m1.1.1.1.1.1.1.1.1.cmml"><mo id="S5.E5.m1.1.1.1.1.1.1.1.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.cmml">(</mo><mrow id="S5.E5.m1.1.1.1.1.1.1.1.1" xref="S5.E5.m1.1.1.1.1.1.1.1.1.cmml"><mrow id="S5.E5.m1.1.1.1.1.1.1.1.1.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.cmml"><mi id="S5.E5.m1.1.1.1.1.1.1.1.1.2.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.2.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.E5.m1.1.1.1.1.1.1.1.1.2.1" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.1.cmml">​</mo><msub id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.cmml"><mi id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.2.cmml">h</mi><mrow id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.cmml"><mi id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.2.cmml">i</mi><mo id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.1" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.1.cmml">−</mo><mn id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.3.cmml">1</mn></mrow></msub></mrow><mo id="S5.E5.m1.1.1.1.1.1.1.1.1.1" xref="S5.E5.m1.1.1.1.1.1.1.1.1.1.cmml">+</mo><mrow id="S5.E5.m1.1.1.1.1.1.1.1.1.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.cmml"><mi id="S5.E5.m1.1.1.1.1.1.1.1.1.3.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.2.cmml">B</mi><mo lspace="0em" rspace="0em" id="S5.E5.m1.1.1.1.1.1.1.1.1.3.1" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.1.cmml">​</mo><msub id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.cmml"><mi id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.2" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.2.cmml">x</mi><mi id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.3.cmml">i</mi></msub></mrow></mrow><mo id="S5.E5.m1.1.1.1.1.1.1.1.3" xref="S5.E5.m1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="S5.E5.m1.1.1.1.2" xref="S5.E5.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.E5.m1.1b"><apply id="S5.E5.m1.1.1.1.1.cmml" xref="S5.E5.m1.1.1.1"><eq id="S5.E5.m1.1.1.1.1.2.cmml" xref="S5.E5.m1.1.1.1.1.2"></eq><apply id="S5.E5.m1.1.1.1.1.3.cmml" xref="S5.E5.m1.1.1.1.1.3"><csymbol cd="ambiguous" id="S5.E5.m1.1.1.1.1.3.1.cmml" xref="S5.E5.m1.1.1.1.1.3">subscript</csymbol><ci id="S5.E5.m1.1.1.1.1.3.2.cmml" xref="S5.E5.m1.1.1.1.1.3.2">ℎ</ci><ci id="S5.E5.m1.1.1.1.1.3.3.cmml" xref="S5.E5.m1.1.1.1.1.3.3">𝑖</ci></apply><apply id="S5.E5.m1.1.1.1.1.1.cmml" xref="S5.E5.m1.1.1.1.1.1"><times id="S5.E5.m1.1.1.1.1.1.2.cmml" xref="S5.E5.m1.1.1.1.1.1.2"></times><ci id="S5.E5.m1.1.1.1.1.1.3.cmml" xref="S5.E5.m1.1.1.1.1.1.3">sgn</ci><apply id="S5.E5.m1.1.1.1.1.1.1.1.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1"><plus id="S5.E5.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.1"></plus><apply id="S5.E5.m1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2"><times id="S5.E5.m1.1.1.1.1.1.1.1.1.2.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.1"></times><ci id="S5.E5.m1.1.1.1.1.1.1.1.1.2.2.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.2">𝐴</ci><apply id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3"><csymbol cd="ambiguous" id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3">subscript</csymbol><ci id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.2.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.2">ℎ</ci><apply id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3"><minus id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.1"></minus><ci id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.2.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.2">𝑖</ci><cn type="integer" id="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.3.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.2.3.3.3">1</cn></apply></apply></apply><apply id="S5.E5.m1.1.1.1.1.1.1.1.1.3.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3"><times id="S5.E5.m1.1.1.1.1.1.1.1.1.3.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.1"></times><ci id="S5.E5.m1.1.1.1.1.1.1.1.1.3.2.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.2">𝐵</ci><apply id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3"><csymbol cd="ambiguous" id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.1.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3">subscript</csymbol><ci id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.2.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.2">𝑥</ci><ci id="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.3.cmml" xref="S5.E5.m1.1.1.1.1.1.1.1.1.3.3.3">𝑖</ci></apply></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.E5.m1.1c">h_{i}=\mathrm{sgn}\left(Ah_{i-1}+Bx_{i}\right).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(5)</span></td>
</tr></tbody>
</table>
<p id="S5.SS1.p1.1" class="ltx_p">After this change, the SSM no longer has a straightforward convolutional form. However, its recurrent form is effectively an RNN, and can therefore solve <math id="S5.SS1.p1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.SS1.p1.1.m1.1a"><msub id="S5.SS1.p1.1.m1.1.1" xref="S5.SS1.p1.1.m1.1.1.cmml"><mi id="S5.SS1.p1.1.m1.1.1.2" xref="S5.SS1.p1.1.m1.1.1.2.cmml">S</mi><mn id="S5.SS1.p1.1.m1.1.1.3" xref="S5.SS1.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.SS1.p1.1.m1.1b"><apply id="S5.SS1.p1.1.m1.1.1.cmml" xref="S5.SS1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS1.p1.1.m1.1.1.1.cmml" xref="S5.SS1.p1.1.m1.1.1">subscript</csymbol><ci id="S5.SS1.p1.1.m1.1.1.2.cmml" xref="S5.SS1.p1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S5.SS1.p1.1.m1.1.1.3.cmml" xref="S5.SS1.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p1.1.m1.1c">S_{5}</annotation></semantics></math>:</p>
</div>
<div id="S5.Thmtheorem1" class="ltx_theorem ltx_theorem_theorem">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S5.Thmtheorem1.1.1.1" class="ltx_text ltx_font_bold">Theorem 5.1</span></span><span id="S5.Thmtheorem1.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S5.Thmtheorem1.p1" class="ltx_para">
<p id="S5.Thmtheorem1.p1.3" class="ltx_p"><span id="S5.Thmtheorem1.p1.3.3" class="ltx_text ltx_font_italic">For any regular language <math id="S5.Thmtheorem1.p1.1.1.m1.1" class="ltx_Math" alttext="L" display="inline"><semantics id="S5.Thmtheorem1.p1.1.1.m1.1a"><mi id="S5.Thmtheorem1.p1.1.1.m1.1.1" xref="S5.Thmtheorem1.p1.1.1.m1.1.1.cmml">L</mi><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem1.p1.1.1.m1.1b"><ci id="S5.Thmtheorem1.p1.1.1.m1.1.1.cmml" xref="S5.Thmtheorem1.p1.1.1.m1.1.1">𝐿</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem1.p1.1.1.m1.1c">L</annotation></semantics></math> (including the word problem for <math id="S5.Thmtheorem1.p1.2.2.m2.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.Thmtheorem1.p1.2.2.m2.1a"><msub id="S5.Thmtheorem1.p1.2.2.m2.1.1" xref="S5.Thmtheorem1.p1.2.2.m2.1.1.cmml"><mi id="S5.Thmtheorem1.p1.2.2.m2.1.1.2" xref="S5.Thmtheorem1.p1.2.2.m2.1.1.2.cmml">S</mi><mn id="S5.Thmtheorem1.p1.2.2.m2.1.1.3" xref="S5.Thmtheorem1.p1.2.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem1.p1.2.2.m2.1b"><apply id="S5.Thmtheorem1.p1.2.2.m2.1.1.cmml" xref="S5.Thmtheorem1.p1.2.2.m2.1.1"><csymbol cd="ambiguous" id="S5.Thmtheorem1.p1.2.2.m2.1.1.1.cmml" xref="S5.Thmtheorem1.p1.2.2.m2.1.1">subscript</csymbol><ci id="S5.Thmtheorem1.p1.2.2.m2.1.1.2.cmml" xref="S5.Thmtheorem1.p1.2.2.m2.1.1.2">𝑆</ci><cn type="integer" id="S5.Thmtheorem1.p1.2.2.m2.1.1.3.cmml" xref="S5.Thmtheorem1.p1.2.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem1.p1.2.2.m2.1c">S_{5}</annotation></semantics></math>), there exists a one-layer log-precision RNN-SSM that recognizes <math id="S5.Thmtheorem1.p1.3.3.m3.1" class="ltx_Math" alttext="L" display="inline"><semantics id="S5.Thmtheorem1.p1.3.3.m3.1a"><mi id="S5.Thmtheorem1.p1.3.3.m3.1.1" xref="S5.Thmtheorem1.p1.3.3.m3.1.1.cmml">L</mi><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem1.p1.3.3.m3.1b"><ci id="S5.Thmtheorem1.p1.3.3.m3.1.1.cmml" xref="S5.Thmtheorem1.p1.3.3.m3.1.1">𝐿</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem1.p1.3.3.m3.1c">L</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="S5.SS1.1" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S5.SS1.1.p1" class="ltx_para">
<p id="S5.SS1.1.p1.1" class="ltx_p">The standard constructions for simulating automata with RNNs <cite class="ltx_cite ltx_citemacro_citep">(cf. Minsky, <a href="#bib.bib27" title="" class="ltx_ref">1954</a>; Merrill, <a href="#bib.bib24" title="" class="ltx_ref">2019</a>)</cite> apply.
∎</p>
</div>
</div>
<div id="S5.SS1.p2" class="ltx_para">
<p id="S5.SS1.p2.1" class="ltx_p">Note that adding nonlinearities to the output of an SSM layer (as in Mamba) is not the same thing as an RNN-SSM. Rather, an RNN-SSM has nonlinearities applied after each recurrent update to the state.</p>
</div>
</section>
<section id="S5.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.2 </span>Via Input-Dependent Transition Matrices</h3>

<div id="S5.SS2.p1" class="ltx_para">
<p id="S5.SS2.p1.2" class="ltx_p">Another completely different way to get greater expressive power is to let <math id="S5.SS2.p1.1.m1.1" class="ltx_Math" alttext="A" display="inline"><semantics id="S5.SS2.p1.1.m1.1a"><mi id="S5.SS2.p1.1.m1.1.1" xref="S5.SS2.p1.1.m1.1.1.cmml">A</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.p1.1.m1.1b"><ci id="S5.SS2.p1.1.m1.1.1.cmml" xref="S5.SS2.p1.1.m1.1.1">𝐴</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p1.1.m1.1c">A</annotation></semantics></math> matrix to be input-dependent.
To illustate this, we define and analyze the <em id="S5.SS2.p1.2.1" class="ltx_emph ltx_font_italic">WFA-SSM layer</em>. Let <math id="S5.SS2.p1.2.m2.2" class="ltx_Math" alttext="A(x_{i})=I+s_{A}(x_{i})" display="inline"><semantics id="S5.SS2.p1.2.m2.2a"><mrow id="S5.SS2.p1.2.m2.2.2" xref="S5.SS2.p1.2.m2.2.2.cmml"><mrow id="S5.SS2.p1.2.m2.1.1.1" xref="S5.SS2.p1.2.m2.1.1.1.cmml"><mi id="S5.SS2.p1.2.m2.1.1.1.3" xref="S5.SS2.p1.2.m2.1.1.1.3.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.SS2.p1.2.m2.1.1.1.2" xref="S5.SS2.p1.2.m2.1.1.1.2.cmml">​</mo><mrow id="S5.SS2.p1.2.m2.1.1.1.1.1" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.SS2.p1.2.m2.1.1.1.1.1.2" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.cmml">(</mo><msub id="S5.SS2.p1.2.m2.1.1.1.1.1.1" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.cmml"><mi id="S5.SS2.p1.2.m2.1.1.1.1.1.1.2" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.2.cmml">x</mi><mi id="S5.SS2.p1.2.m2.1.1.1.1.1.1.3" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="S5.SS2.p1.2.m2.1.1.1.1.1.3" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S5.SS2.p1.2.m2.2.2.3" xref="S5.SS2.p1.2.m2.2.2.3.cmml">=</mo><mrow id="S5.SS2.p1.2.m2.2.2.2" xref="S5.SS2.p1.2.m2.2.2.2.cmml"><mi id="S5.SS2.p1.2.m2.2.2.2.3" xref="S5.SS2.p1.2.m2.2.2.2.3.cmml">I</mi><mo id="S5.SS2.p1.2.m2.2.2.2.2" xref="S5.SS2.p1.2.m2.2.2.2.2.cmml">+</mo><mrow id="S5.SS2.p1.2.m2.2.2.2.1" xref="S5.SS2.p1.2.m2.2.2.2.1.cmml"><msub id="S5.SS2.p1.2.m2.2.2.2.1.3" xref="S5.SS2.p1.2.m2.2.2.2.1.3.cmml"><mi id="S5.SS2.p1.2.m2.2.2.2.1.3.2" xref="S5.SS2.p1.2.m2.2.2.2.1.3.2.cmml">s</mi><mi id="S5.SS2.p1.2.m2.2.2.2.1.3.3" xref="S5.SS2.p1.2.m2.2.2.2.1.3.3.cmml">A</mi></msub><mo lspace="0em" rspace="0em" id="S5.SS2.p1.2.m2.2.2.2.1.2" xref="S5.SS2.p1.2.m2.2.2.2.1.2.cmml">​</mo><mrow id="S5.SS2.p1.2.m2.2.2.2.1.1.1" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.cmml"><mo stretchy="false" id="S5.SS2.p1.2.m2.2.2.2.1.1.1.2" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.cmml">(</mo><msub id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.cmml"><mi id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.2" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.2.cmml">x</mi><mi id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.3" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="S5.SS2.p1.2.m2.2.2.2.1.1.1.3" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.cmml">)</mo></mrow></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.p1.2.m2.2b"><apply id="S5.SS2.p1.2.m2.2.2.cmml" xref="S5.SS2.p1.2.m2.2.2"><eq id="S5.SS2.p1.2.m2.2.2.3.cmml" xref="S5.SS2.p1.2.m2.2.2.3"></eq><apply id="S5.SS2.p1.2.m2.1.1.1.cmml" xref="S5.SS2.p1.2.m2.1.1.1"><times id="S5.SS2.p1.2.m2.1.1.1.2.cmml" xref="S5.SS2.p1.2.m2.1.1.1.2"></times><ci id="S5.SS2.p1.2.m2.1.1.1.3.cmml" xref="S5.SS2.p1.2.m2.1.1.1.3">𝐴</ci><apply id="S5.SS2.p1.2.m2.1.1.1.1.1.1.cmml" xref="S5.SS2.p1.2.m2.1.1.1.1.1"><csymbol cd="ambiguous" id="S5.SS2.p1.2.m2.1.1.1.1.1.1.1.cmml" xref="S5.SS2.p1.2.m2.1.1.1.1.1">subscript</csymbol><ci id="S5.SS2.p1.2.m2.1.1.1.1.1.1.2.cmml" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.2">𝑥</ci><ci id="S5.SS2.p1.2.m2.1.1.1.1.1.1.3.cmml" xref="S5.SS2.p1.2.m2.1.1.1.1.1.1.3">𝑖</ci></apply></apply><apply id="S5.SS2.p1.2.m2.2.2.2.cmml" xref="S5.SS2.p1.2.m2.2.2.2"><plus id="S5.SS2.p1.2.m2.2.2.2.2.cmml" xref="S5.SS2.p1.2.m2.2.2.2.2"></plus><ci id="S5.SS2.p1.2.m2.2.2.2.3.cmml" xref="S5.SS2.p1.2.m2.2.2.2.3">𝐼</ci><apply id="S5.SS2.p1.2.m2.2.2.2.1.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1"><times id="S5.SS2.p1.2.m2.2.2.2.1.2.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.2"></times><apply id="S5.SS2.p1.2.m2.2.2.2.1.3.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.3"><csymbol cd="ambiguous" id="S5.SS2.p1.2.m2.2.2.2.1.3.1.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.3">subscript</csymbol><ci id="S5.SS2.p1.2.m2.2.2.2.1.3.2.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.3.2">𝑠</ci><ci id="S5.SS2.p1.2.m2.2.2.2.1.3.3.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.3.3">𝐴</ci></apply><apply id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1"><csymbol cd="ambiguous" id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.1.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1">subscript</csymbol><ci id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.2.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.2">𝑥</ci><ci id="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.3.cmml" xref="S5.SS2.p1.2.m2.2.2.2.1.1.1.1.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p1.2.m2.2c">A(x_{i})=I+s_{A}(x_{i})</annotation></semantics></math>. The recurrent form becomes:</p>
<table id="S5.E6" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S5.E6.m1.1" class="ltx_Math" alttext="h_{i}=A(x_{i})h_{i-1}+Bx_{i}." display="block"><semantics id="S5.E6.m1.1a"><mrow id="S5.E6.m1.1.1.1" xref="S5.E6.m1.1.1.1.1.cmml"><mrow id="S5.E6.m1.1.1.1.1" xref="S5.E6.m1.1.1.1.1.cmml"><msub id="S5.E6.m1.1.1.1.1.3" xref="S5.E6.m1.1.1.1.1.3.cmml"><mi id="S5.E6.m1.1.1.1.1.3.2" xref="S5.E6.m1.1.1.1.1.3.2.cmml">h</mi><mi id="S5.E6.m1.1.1.1.1.3.3" xref="S5.E6.m1.1.1.1.1.3.3.cmml">i</mi></msub><mo id="S5.E6.m1.1.1.1.1.2" xref="S5.E6.m1.1.1.1.1.2.cmml">=</mo><mrow id="S5.E6.m1.1.1.1.1.1" xref="S5.E6.m1.1.1.1.1.1.cmml"><mrow id="S5.E6.m1.1.1.1.1.1.1" xref="S5.E6.m1.1.1.1.1.1.1.cmml"><mi id="S5.E6.m1.1.1.1.1.1.1.3" xref="S5.E6.m1.1.1.1.1.1.1.3.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.E6.m1.1.1.1.1.1.1.2" xref="S5.E6.m1.1.1.1.1.1.1.2.cmml">​</mo><mrow id="S5.E6.m1.1.1.1.1.1.1.1.1" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.E6.m1.1.1.1.1.1.1.1.1.2" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.cmml">(</mo><msub id="S5.E6.m1.1.1.1.1.1.1.1.1.1" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S5.E6.m1.1.1.1.1.1.1.1.1.1.2" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mi id="S5.E6.m1.1.1.1.1.1.1.1.1.1.3" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="S5.E6.m1.1.1.1.1.1.1.1.1.3" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow><mo lspace="0em" rspace="0em" id="S5.E6.m1.1.1.1.1.1.1.2a" xref="S5.E6.m1.1.1.1.1.1.1.2.cmml">​</mo><msub id="S5.E6.m1.1.1.1.1.1.1.4" xref="S5.E6.m1.1.1.1.1.1.1.4.cmml"><mi id="S5.E6.m1.1.1.1.1.1.1.4.2" xref="S5.E6.m1.1.1.1.1.1.1.4.2.cmml">h</mi><mrow id="S5.E6.m1.1.1.1.1.1.1.4.3" xref="S5.E6.m1.1.1.1.1.1.1.4.3.cmml"><mi id="S5.E6.m1.1.1.1.1.1.1.4.3.2" xref="S5.E6.m1.1.1.1.1.1.1.4.3.2.cmml">i</mi><mo id="S5.E6.m1.1.1.1.1.1.1.4.3.1" xref="S5.E6.m1.1.1.1.1.1.1.4.3.1.cmml">−</mo><mn id="S5.E6.m1.1.1.1.1.1.1.4.3.3" xref="S5.E6.m1.1.1.1.1.1.1.4.3.3.cmml">1</mn></mrow></msub></mrow><mo id="S5.E6.m1.1.1.1.1.1.2" xref="S5.E6.m1.1.1.1.1.1.2.cmml">+</mo><mrow id="S5.E6.m1.1.1.1.1.1.3" xref="S5.E6.m1.1.1.1.1.1.3.cmml"><mi id="S5.E6.m1.1.1.1.1.1.3.2" xref="S5.E6.m1.1.1.1.1.1.3.2.cmml">B</mi><mo lspace="0em" rspace="0em" id="S5.E6.m1.1.1.1.1.1.3.1" xref="S5.E6.m1.1.1.1.1.1.3.1.cmml">​</mo><msub id="S5.E6.m1.1.1.1.1.1.3.3" xref="S5.E6.m1.1.1.1.1.1.3.3.cmml"><mi id="S5.E6.m1.1.1.1.1.1.3.3.2" xref="S5.E6.m1.1.1.1.1.1.3.3.2.cmml">x</mi><mi id="S5.E6.m1.1.1.1.1.1.3.3.3" xref="S5.E6.m1.1.1.1.1.1.3.3.3.cmml">i</mi></msub></mrow></mrow></mrow><mo lspace="0em" id="S5.E6.m1.1.1.1.2" xref="S5.E6.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.E6.m1.1b"><apply id="S5.E6.m1.1.1.1.1.cmml" xref="S5.E6.m1.1.1.1"><eq id="S5.E6.m1.1.1.1.1.2.cmml" xref="S5.E6.m1.1.1.1.1.2"></eq><apply id="S5.E6.m1.1.1.1.1.3.cmml" xref="S5.E6.m1.1.1.1.1.3"><csymbol cd="ambiguous" id="S5.E6.m1.1.1.1.1.3.1.cmml" xref="S5.E6.m1.1.1.1.1.3">subscript</csymbol><ci id="S5.E6.m1.1.1.1.1.3.2.cmml" xref="S5.E6.m1.1.1.1.1.3.2">ℎ</ci><ci id="S5.E6.m1.1.1.1.1.3.3.cmml" xref="S5.E6.m1.1.1.1.1.3.3">𝑖</ci></apply><apply id="S5.E6.m1.1.1.1.1.1.cmml" xref="S5.E6.m1.1.1.1.1.1"><plus id="S5.E6.m1.1.1.1.1.1.2.cmml" xref="S5.E6.m1.1.1.1.1.1.2"></plus><apply id="S5.E6.m1.1.1.1.1.1.1.cmml" xref="S5.E6.m1.1.1.1.1.1.1"><times id="S5.E6.m1.1.1.1.1.1.1.2.cmml" xref="S5.E6.m1.1.1.1.1.1.1.2"></times><ci id="S5.E6.m1.1.1.1.1.1.1.3.cmml" xref="S5.E6.m1.1.1.1.1.1.1.3">𝐴</ci><apply id="S5.E6.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E6.m1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S5.E6.m1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E6.m1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S5.E6.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.2">𝑥</ci><ci id="S5.E6.m1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S5.E6.m1.1.1.1.1.1.1.1.1.1.3">𝑖</ci></apply><apply id="S5.E6.m1.1.1.1.1.1.1.4.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4"><csymbol cd="ambiguous" id="S5.E6.m1.1.1.1.1.1.1.4.1.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4">subscript</csymbol><ci id="S5.E6.m1.1.1.1.1.1.1.4.2.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4.2">ℎ</ci><apply id="S5.E6.m1.1.1.1.1.1.1.4.3.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4.3"><minus id="S5.E6.m1.1.1.1.1.1.1.4.3.1.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4.3.1"></minus><ci id="S5.E6.m1.1.1.1.1.1.1.4.3.2.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4.3.2">𝑖</ci><cn type="integer" id="S5.E6.m1.1.1.1.1.1.1.4.3.3.cmml" xref="S5.E6.m1.1.1.1.1.1.1.4.3.3">1</cn></apply></apply></apply><apply id="S5.E6.m1.1.1.1.1.1.3.cmml" xref="S5.E6.m1.1.1.1.1.1.3"><times id="S5.E6.m1.1.1.1.1.1.3.1.cmml" xref="S5.E6.m1.1.1.1.1.1.3.1"></times><ci id="S5.E6.m1.1.1.1.1.1.3.2.cmml" xref="S5.E6.m1.1.1.1.1.1.3.2">𝐵</ci><apply id="S5.E6.m1.1.1.1.1.1.3.3.cmml" xref="S5.E6.m1.1.1.1.1.1.3.3"><csymbol cd="ambiguous" id="S5.E6.m1.1.1.1.1.1.3.3.1.cmml" xref="S5.E6.m1.1.1.1.1.1.3.3">subscript</csymbol><ci id="S5.E6.m1.1.1.1.1.1.3.3.2.cmml" xref="S5.E6.m1.1.1.1.1.1.3.3.2">𝑥</ci><ci id="S5.E6.m1.1.1.1.1.1.3.3.3.cmml" xref="S5.E6.m1.1.1.1.1.1.3.3.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.E6.m1.1c">h_{i}=A(x_{i})h_{i-1}+Bx_{i}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(6)</span></td>
</tr></tbody>
</table>
<p id="S5.SS2.p1.3" class="ltx_p">This means the convolutional form computes an <em id="S5.SS2.p1.3.1" class="ltx_emph ltx_font_italic">iterated product</em> of a sequence of matrices rather than powering a matrix as for S4 and S6 (cf. <a href="#S2.SS1" title="2.1 Architecture of State-Space Models ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">2.1</span></a>):</p>
<table id="S5.E7" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S5.E7.m1.1" class="ltx_Math" alttext="h_{i}=\sum_{j=1}^{i}\left(\prod_{k=j+1}^{i}A(x_{k})\right)Bx_{j}." display="block"><semantics id="S5.E7.m1.1a"><mrow id="S5.E7.m1.1.1.1" xref="S5.E7.m1.1.1.1.1.cmml"><mrow id="S5.E7.m1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.cmml"><msub id="S5.E7.m1.1.1.1.1.3" xref="S5.E7.m1.1.1.1.1.3.cmml"><mi id="S5.E7.m1.1.1.1.1.3.2" xref="S5.E7.m1.1.1.1.1.3.2.cmml">h</mi><mi id="S5.E7.m1.1.1.1.1.3.3" xref="S5.E7.m1.1.1.1.1.3.3.cmml">i</mi></msub><mo rspace="0.111em" id="S5.E7.m1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.2.cmml">=</mo><mrow id="S5.E7.m1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.cmml"><munderover id="S5.E7.m1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.2.cmml"><mo movablelimits="false" rspace="0em" id="S5.E7.m1.1.1.1.1.1.2.2.2" xref="S5.E7.m1.1.1.1.1.1.2.2.2.cmml">∑</mo><mrow id="S5.E7.m1.1.1.1.1.1.2.2.3" xref="S5.E7.m1.1.1.1.1.1.2.2.3.cmml"><mi id="S5.E7.m1.1.1.1.1.1.2.2.3.2" xref="S5.E7.m1.1.1.1.1.1.2.2.3.2.cmml">j</mi><mo id="S5.E7.m1.1.1.1.1.1.2.2.3.1" xref="S5.E7.m1.1.1.1.1.1.2.2.3.1.cmml">=</mo><mn id="S5.E7.m1.1.1.1.1.1.2.2.3.3" xref="S5.E7.m1.1.1.1.1.1.2.2.3.3.cmml">1</mn></mrow><mi id="S5.E7.m1.1.1.1.1.1.2.3" xref="S5.E7.m1.1.1.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S5.E7.m1.1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.1.cmml"><mrow id="S5.E7.m1.1.1.1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.cmml"><mo id="S5.E7.m1.1.1.1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.cmml">(</mo><mrow id="S5.E7.m1.1.1.1.1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.cmml"><munderover id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.cmml"><mo lspace="0em" movablelimits="false" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.2.cmml">∏</mo><mrow id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.cmml"><mi id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.2.cmml">k</mi><mo id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.1.cmml">=</mo><mrow id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.cmml"><mi id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.2.cmml">j</mi><mo id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.1.cmml">+</mo><mn id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.3.cmml">1</mn></mrow></mrow><mi id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.3.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.2.cmml">​</mo><mrow id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml">(</mo><msub id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mi id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.3.cmml">k</mi></msub><mo stretchy="false" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo id="S5.E7.m1.1.1.1.1.1.1.1.1.3" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow><mo lspace="0em" rspace="0em" id="S5.E7.m1.1.1.1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.1.1.2.cmml">​</mo><mi id="S5.E7.m1.1.1.1.1.1.1.3" xref="S5.E7.m1.1.1.1.1.1.1.3.cmml">B</mi><mo lspace="0em" rspace="0em" id="S5.E7.m1.1.1.1.1.1.1.2a" xref="S5.E7.m1.1.1.1.1.1.1.2.cmml">​</mo><msub id="S5.E7.m1.1.1.1.1.1.1.4" xref="S5.E7.m1.1.1.1.1.1.1.4.cmml"><mi id="S5.E7.m1.1.1.1.1.1.1.4.2" xref="S5.E7.m1.1.1.1.1.1.1.4.2.cmml">x</mi><mi id="S5.E7.m1.1.1.1.1.1.1.4.3" xref="S5.E7.m1.1.1.1.1.1.1.4.3.cmml">j</mi></msub></mrow></mrow></mrow><mo lspace="0em" id="S5.E7.m1.1.1.1.2" xref="S5.E7.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.E7.m1.1b"><apply id="S5.E7.m1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1"><eq id="S5.E7.m1.1.1.1.1.2.cmml" xref="S5.E7.m1.1.1.1.1.2"></eq><apply id="S5.E7.m1.1.1.1.1.3.cmml" xref="S5.E7.m1.1.1.1.1.3"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.3.1.cmml" xref="S5.E7.m1.1.1.1.1.3">subscript</csymbol><ci id="S5.E7.m1.1.1.1.1.3.2.cmml" xref="S5.E7.m1.1.1.1.1.3.2">ℎ</ci><ci id="S5.E7.m1.1.1.1.1.3.3.cmml" xref="S5.E7.m1.1.1.1.1.3.3">𝑖</ci></apply><apply id="S5.E7.m1.1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1.1.1"><apply id="S5.E7.m1.1.1.1.1.1.2.cmml" xref="S5.E7.m1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.1.2.1.cmml" xref="S5.E7.m1.1.1.1.1.1.2">superscript</csymbol><apply id="S5.E7.m1.1.1.1.1.1.2.2.cmml" xref="S5.E7.m1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.1.2.2.1.cmml" xref="S5.E7.m1.1.1.1.1.1.2">subscript</csymbol><sum id="S5.E7.m1.1.1.1.1.1.2.2.2.cmml" xref="S5.E7.m1.1.1.1.1.1.2.2.2"></sum><apply id="S5.E7.m1.1.1.1.1.1.2.2.3.cmml" xref="S5.E7.m1.1.1.1.1.1.2.2.3"><eq id="S5.E7.m1.1.1.1.1.1.2.2.3.1.cmml" xref="S5.E7.m1.1.1.1.1.1.2.2.3.1"></eq><ci id="S5.E7.m1.1.1.1.1.1.2.2.3.2.cmml" xref="S5.E7.m1.1.1.1.1.1.2.2.3.2">𝑗</ci><cn type="integer" id="S5.E7.m1.1.1.1.1.1.2.2.3.3.cmml" xref="S5.E7.m1.1.1.1.1.1.2.2.3.3">1</cn></apply></apply><ci id="S5.E7.m1.1.1.1.1.1.2.3.cmml" xref="S5.E7.m1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S5.E7.m1.1.1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1"><times id="S5.E7.m1.1.1.1.1.1.1.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.2"></times><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1"><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2">superscript</csymbol><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.2">product</csymbol><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3"><eq id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.1"></eq><ci id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.2">𝑘</ci><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3"><plus id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.1"></plus><ci id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.2">𝑗</ci><cn type="integer" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.2.3.3.3">1</cn></apply></apply></apply><ci id="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1"><times id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.2"></times><ci id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.3">𝐴</ci><apply id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.2">𝑥</ci><ci id="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.3">𝑘</ci></apply></apply></apply><ci id="S5.E7.m1.1.1.1.1.1.1.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.3">𝐵</ci><apply id="S5.E7.m1.1.1.1.1.1.1.4.cmml" xref="S5.E7.m1.1.1.1.1.1.1.4"><csymbol cd="ambiguous" id="S5.E7.m1.1.1.1.1.1.1.4.1.cmml" xref="S5.E7.m1.1.1.1.1.1.1.4">subscript</csymbol><ci id="S5.E7.m1.1.1.1.1.1.1.4.2.cmml" xref="S5.E7.m1.1.1.1.1.1.1.4.2">𝑥</ci><ci id="S5.E7.m1.1.1.1.1.1.1.4.3.cmml" xref="S5.E7.m1.1.1.1.1.1.1.4.3">𝑗</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.E7.m1.1c">h_{i}=\sum_{j=1}^{i}\left(\prod_{k=j+1}^{i}A(x_{k})\right)Bx_{j}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(7)</span></td>
</tr></tbody>
</table>
</div>
<div id="S5.SS2.p2" class="ltx_para">
<p id="S5.SS2.p2.2" class="ltx_p">Unlike matrix powers, iterated matrix products cannot, in general, be computed in <math id="S5.SS2.p2.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S5.SS2.p2.1.m1.1a"><msup id="S5.SS2.p2.1.m1.1.1" xref="S5.SS2.p2.1.m1.1.1.cmml"><mi id="S5.SS2.p2.1.m1.1.1.2" xref="S5.SS2.p2.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S5.SS2.p2.1.m1.1.1.3" xref="S5.SS2.p2.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S5.SS2.p2.1.m1.1b"><apply id="S5.SS2.p2.1.m1.1.1.cmml" xref="S5.SS2.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS2.p2.1.m1.1.1.1.cmml" xref="S5.SS2.p2.1.m1.1.1">superscript</csymbol><ci id="S5.SS2.p2.1.m1.1.1.2.cmml" xref="S5.SS2.p2.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S5.SS2.p2.1.m1.1.1.3.cmml" xref="S5.SS2.p2.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p2.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math> <cite class="ltx_cite ltx_citemacro_citep">(Mereghetti &amp; Palano, <a href="#bib.bib23" title="" class="ltx_ref">2000</a>)</cite>.
This means that the argument from <a href="#S4.Thmtheorem1" title="Theorem 4.1. ‣ 4.1 Simulating Linear SSMs with a 𝖳𝖢⁰ Circuit Family ‣ 4 SSMs Can be Simulated in 𝖳𝖢⁰ ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorem</span> <span class="ltx_text ltx_ref_tag">4.1</span></a> will not go through for WFA-SSMs. In fact, we can show that WFA-SSMs gains expressive power beyond <math id="S5.SS2.p2.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S5.SS2.p2.2.m2.1a"><msup id="S5.SS2.p2.2.m2.1.1" xref="S5.SS2.p2.2.m2.1.1.cmml"><mi id="S5.SS2.p2.2.m2.1.1.2" xref="S5.SS2.p2.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S5.SS2.p2.2.m2.1.1.3" xref="S5.SS2.p2.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S5.SS2.p2.2.m2.1b"><apply id="S5.SS2.p2.2.m2.1.1.cmml" xref="S5.SS2.p2.2.m2.1.1"><csymbol cd="ambiguous" id="S5.SS2.p2.2.m2.1.1.1.cmml" xref="S5.SS2.p2.2.m2.1.1">superscript</csymbol><ci id="S5.SS2.p2.2.m2.1.1.2.cmml" xref="S5.SS2.p2.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S5.SS2.p2.2.m2.1.1.3.cmml" xref="S5.SS2.p2.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p2.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math>:</p>
</div>
<div id="S5.Thmtheorem2" class="ltx_theorem ltx_theorem_theorem">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S5.Thmtheorem2.1.1.1" class="ltx_text ltx_font_bold">Theorem 5.2</span></span><span id="S5.Thmtheorem2.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S5.Thmtheorem2.p1" class="ltx_para">
<p id="S5.Thmtheorem2.p1.5" class="ltx_p"><span id="S5.Thmtheorem2.p1.5.5" class="ltx_text ltx_font_italic">For any regular language <math id="S5.Thmtheorem2.p1.1.1.m1.1" class="ltx_Math" alttext="L" display="inline"><semantics id="S5.Thmtheorem2.p1.1.1.m1.1a"><mi id="S5.Thmtheorem2.p1.1.1.m1.1.1" xref="S5.Thmtheorem2.p1.1.1.m1.1.1.cmml">L</mi><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem2.p1.1.1.m1.1b"><ci id="S5.Thmtheorem2.p1.1.1.m1.1.1.cmml" xref="S5.Thmtheorem2.p1.1.1.m1.1.1">𝐿</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem2.p1.1.1.m1.1c">L</annotation></semantics></math> over vocabulary <math id="S5.Thmtheorem2.p1.2.2.m2.1" class="ltx_Math" alttext="\Sigma" display="inline"><semantics id="S5.Thmtheorem2.p1.2.2.m2.1a"><mi mathvariant="normal" id="S5.Thmtheorem2.p1.2.2.m2.1.1" xref="S5.Thmtheorem2.p1.2.2.m2.1.1.cmml">Σ</mi><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem2.p1.2.2.m2.1b"><ci id="S5.Thmtheorem2.p1.2.2.m2.1.1.cmml" xref="S5.Thmtheorem2.p1.2.2.m2.1.1">Σ</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem2.p1.2.2.m2.1c">\Sigma</annotation></semantics></math> (including the word problem for <math id="S5.Thmtheorem2.p1.3.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.Thmtheorem2.p1.3.3.m3.1a"><msub id="S5.Thmtheorem2.p1.3.3.m3.1.1" xref="S5.Thmtheorem2.p1.3.3.m3.1.1.cmml"><mi id="S5.Thmtheorem2.p1.3.3.m3.1.1.2" xref="S5.Thmtheorem2.p1.3.3.m3.1.1.2.cmml">S</mi><mn id="S5.Thmtheorem2.p1.3.3.m3.1.1.3" xref="S5.Thmtheorem2.p1.3.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem2.p1.3.3.m3.1b"><apply id="S5.Thmtheorem2.p1.3.3.m3.1.1.cmml" xref="S5.Thmtheorem2.p1.3.3.m3.1.1"><csymbol cd="ambiguous" id="S5.Thmtheorem2.p1.3.3.m3.1.1.1.cmml" xref="S5.Thmtheorem2.p1.3.3.m3.1.1">subscript</csymbol><ci id="S5.Thmtheorem2.p1.3.3.m3.1.1.2.cmml" xref="S5.Thmtheorem2.p1.3.3.m3.1.1.2">𝑆</ci><cn type="integer" id="S5.Thmtheorem2.p1.3.3.m3.1.1.3.cmml" xref="S5.Thmtheorem2.p1.3.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem2.p1.3.3.m3.1c">S_{5}</annotation></semantics></math>), there exists a one-layer log-precision WFA-SSM that recognizes <math id="S5.Thmtheorem2.p1.4.4.m4.1" class="ltx_Math" alttext="\$L" display="inline"><semantics id="S5.Thmtheorem2.p1.4.4.m4.1a"><mrow id="S5.Thmtheorem2.p1.4.4.m4.1.1" xref="S5.Thmtheorem2.p1.4.4.m4.1.1.cmml"><mo rspace="0.167em" id="S5.Thmtheorem2.p1.4.4.m4.1.1.1" xref="S5.Thmtheorem2.p1.4.4.m4.1.1.1.cmml">$</mo><mi id="S5.Thmtheorem2.p1.4.4.m4.1.1.2" xref="S5.Thmtheorem2.p1.4.4.m4.1.1.2.cmml">L</mi></mrow><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem2.p1.4.4.m4.1b"><apply id="S5.Thmtheorem2.p1.4.4.m4.1.1.cmml" xref="S5.Thmtheorem2.p1.4.4.m4.1.1"><csymbol cd="latexml" id="S5.Thmtheorem2.p1.4.4.m4.1.1.1.cmml" xref="S5.Thmtheorem2.p1.4.4.m4.1.1.1">currency-dollar</csymbol><ci id="S5.Thmtheorem2.p1.4.4.m4.1.1.2.cmml" xref="S5.Thmtheorem2.p1.4.4.m4.1.1.2">𝐿</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem2.p1.4.4.m4.1c">\$L</annotation></semantics></math>, where <math id="S5.Thmtheorem2.p1.5.5.m5.1" class="ltx_Math" alttext="\$\not\in\Sigma" display="inline"><semantics id="S5.Thmtheorem2.p1.5.5.m5.1a"><mrow id="S5.Thmtheorem2.p1.5.5.m5.1.1" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.cmml"><mo rspace="0.1389em" id="S5.Thmtheorem2.p1.5.5.m5.1.1.2" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.2.cmml">$</mo><mo lspace="0.1389em" id="S5.Thmtheorem2.p1.5.5.m5.1.1.1" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.1.cmml">∉</mo><mi mathvariant="normal" id="S5.Thmtheorem2.p1.5.5.m5.1.1.3" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.3.cmml">Σ</mi></mrow><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem2.p1.5.5.m5.1b"><apply id="S5.Thmtheorem2.p1.5.5.m5.1.1.cmml" xref="S5.Thmtheorem2.p1.5.5.m5.1.1"><notin id="S5.Thmtheorem2.p1.5.5.m5.1.1.1.cmml" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.1"></notin><csymbol cd="latexml" id="S5.Thmtheorem2.p1.5.5.m5.1.1.2.cmml" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.2">currency-dollar</csymbol><ci id="S5.Thmtheorem2.p1.5.5.m5.1.1.3.cmml" xref="S5.Thmtheorem2.p1.5.5.m5.1.1.3">Σ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem2.p1.5.5.m5.1c">\$\not\in\Sigma</annotation></semantics></math> is a special beginning-of-string symbol.</span></p>
</div>
</div>
<div id="S5.SS2.3" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="S5.SS2.1.p1" class="ltx_para">
<p id="S5.SS2.1.p1.4" class="ltx_p">It suffices to show that an WFA-SSM can simulate a deterministic finite automaton (DFA). We do this via a transition monoid construction. For any <math id="S5.SS2.1.p1.1.m1.1" class="ltx_Math" alttext="w\in\Sigma^{*}" display="inline"><semantics id="S5.SS2.1.p1.1.m1.1a"><mrow id="S5.SS2.1.p1.1.m1.1.1" xref="S5.SS2.1.p1.1.m1.1.1.cmml"><mi id="S5.SS2.1.p1.1.m1.1.1.2" xref="S5.SS2.1.p1.1.m1.1.1.2.cmml">w</mi><mo id="S5.SS2.1.p1.1.m1.1.1.1" xref="S5.SS2.1.p1.1.m1.1.1.1.cmml">∈</mo><msup id="S5.SS2.1.p1.1.m1.1.1.3" xref="S5.SS2.1.p1.1.m1.1.1.3.cmml"><mi mathvariant="normal" id="S5.SS2.1.p1.1.m1.1.1.3.2" xref="S5.SS2.1.p1.1.m1.1.1.3.2.cmml">Σ</mi><mo id="S5.SS2.1.p1.1.m1.1.1.3.3" xref="S5.SS2.1.p1.1.m1.1.1.3.3.cmml">∗</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.1.p1.1.m1.1b"><apply id="S5.SS2.1.p1.1.m1.1.1.cmml" xref="S5.SS2.1.p1.1.m1.1.1"><in id="S5.SS2.1.p1.1.m1.1.1.1.cmml" xref="S5.SS2.1.p1.1.m1.1.1.1"></in><ci id="S5.SS2.1.p1.1.m1.1.1.2.cmml" xref="S5.SS2.1.p1.1.m1.1.1.2">𝑤</ci><apply id="S5.SS2.1.p1.1.m1.1.1.3.cmml" xref="S5.SS2.1.p1.1.m1.1.1.3"><csymbol cd="ambiguous" id="S5.SS2.1.p1.1.m1.1.1.3.1.cmml" xref="S5.SS2.1.p1.1.m1.1.1.3">superscript</csymbol><ci id="S5.SS2.1.p1.1.m1.1.1.3.2.cmml" xref="S5.SS2.1.p1.1.m1.1.1.3.2">Σ</ci><times id="S5.SS2.1.p1.1.m1.1.1.3.3.cmml" xref="S5.SS2.1.p1.1.m1.1.1.3.3"></times></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.1.p1.1.m1.1c">w\in\Sigma^{*}</annotation></semantics></math>, let <math id="S5.SS2.1.p1.2.m2.1" class="ltx_Math" alttext="\delta_{w}:Q\to Q" display="inline"><semantics id="S5.SS2.1.p1.2.m2.1a"><mrow id="S5.SS2.1.p1.2.m2.1.1" xref="S5.SS2.1.p1.2.m2.1.1.cmml"><msub id="S5.SS2.1.p1.2.m2.1.1.2" xref="S5.SS2.1.p1.2.m2.1.1.2.cmml"><mi id="S5.SS2.1.p1.2.m2.1.1.2.2" xref="S5.SS2.1.p1.2.m2.1.1.2.2.cmml">δ</mi><mi id="S5.SS2.1.p1.2.m2.1.1.2.3" xref="S5.SS2.1.p1.2.m2.1.1.2.3.cmml">w</mi></msub><mo lspace="0.278em" rspace="0.278em" id="S5.SS2.1.p1.2.m2.1.1.1" xref="S5.SS2.1.p1.2.m2.1.1.1.cmml">:</mo><mrow id="S5.SS2.1.p1.2.m2.1.1.3" xref="S5.SS2.1.p1.2.m2.1.1.3.cmml"><mi id="S5.SS2.1.p1.2.m2.1.1.3.2" xref="S5.SS2.1.p1.2.m2.1.1.3.2.cmml">Q</mi><mo stretchy="false" id="S5.SS2.1.p1.2.m2.1.1.3.1" xref="S5.SS2.1.p1.2.m2.1.1.3.1.cmml">→</mo><mi id="S5.SS2.1.p1.2.m2.1.1.3.3" xref="S5.SS2.1.p1.2.m2.1.1.3.3.cmml">Q</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.1.p1.2.m2.1b"><apply id="S5.SS2.1.p1.2.m2.1.1.cmml" xref="S5.SS2.1.p1.2.m2.1.1"><ci id="S5.SS2.1.p1.2.m2.1.1.1.cmml" xref="S5.SS2.1.p1.2.m2.1.1.1">:</ci><apply id="S5.SS2.1.p1.2.m2.1.1.2.cmml" xref="S5.SS2.1.p1.2.m2.1.1.2"><csymbol cd="ambiguous" id="S5.SS2.1.p1.2.m2.1.1.2.1.cmml" xref="S5.SS2.1.p1.2.m2.1.1.2">subscript</csymbol><ci id="S5.SS2.1.p1.2.m2.1.1.2.2.cmml" xref="S5.SS2.1.p1.2.m2.1.1.2.2">𝛿</ci><ci id="S5.SS2.1.p1.2.m2.1.1.2.3.cmml" xref="S5.SS2.1.p1.2.m2.1.1.2.3">𝑤</ci></apply><apply id="S5.SS2.1.p1.2.m2.1.1.3.cmml" xref="S5.SS2.1.p1.2.m2.1.1.3"><ci id="S5.SS2.1.p1.2.m2.1.1.3.1.cmml" xref="S5.SS2.1.p1.2.m2.1.1.3.1">→</ci><ci id="S5.SS2.1.p1.2.m2.1.1.3.2.cmml" xref="S5.SS2.1.p1.2.m2.1.1.3.2">𝑄</ci><ci id="S5.SS2.1.p1.2.m2.1.1.3.3.cmml" xref="S5.SS2.1.p1.2.m2.1.1.3.3">𝑄</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.1.p1.2.m2.1c">\delta_{w}:Q\to Q</annotation></semantics></math> be the function mapping a state to its eventual destination state after <math id="S5.SS2.1.p1.3.m3.1" class="ltx_Math" alttext="w" display="inline"><semantics id="S5.SS2.1.p1.3.m3.1a"><mi id="S5.SS2.1.p1.3.m3.1.1" xref="S5.SS2.1.p1.3.m3.1.1.cmml">w</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.1.p1.3.m3.1b"><ci id="S5.SS2.1.p1.3.m3.1.1.cmml" xref="S5.SS2.1.p1.3.m3.1.1">𝑤</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.1.p1.3.m3.1c">w</annotation></semantics></math> is read from that state. For any DFA, this set of functions forms a finite monoid (the <em id="S5.SS2.1.p1.4.1" class="ltx_emph ltx_font_italic">transition monoid</em>) under composition,
following from the Myhill-Nerode theorem <cite class="ltx_cite ltx_citemacro_citep">(Hopcroft et al., <a href="#bib.bib17" title="" class="ltx_ref">2001</a>)</cite>.
Further, each monoid element <math id="S5.SS2.1.p1.4.m4.1" class="ltx_Math" alttext="\delta_{w}" display="inline"><semantics id="S5.SS2.1.p1.4.m4.1a"><msub id="S5.SS2.1.p1.4.m4.1.1" xref="S5.SS2.1.p1.4.m4.1.1.cmml"><mi id="S5.SS2.1.p1.4.m4.1.1.2" xref="S5.SS2.1.p1.4.m4.1.1.2.cmml">δ</mi><mi id="S5.SS2.1.p1.4.m4.1.1.3" xref="S5.SS2.1.p1.4.m4.1.1.3.cmml">w</mi></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.1.p1.4.m4.1b"><apply id="S5.SS2.1.p1.4.m4.1.1.cmml" xref="S5.SS2.1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S5.SS2.1.p1.4.m4.1.1.1.cmml" xref="S5.SS2.1.p1.4.m4.1.1">subscript</csymbol><ci id="S5.SS2.1.p1.4.m4.1.1.2.cmml" xref="S5.SS2.1.p1.4.m4.1.1.2">𝛿</ci><ci id="S5.SS2.1.p1.4.m4.1.1.3.cmml" xref="S5.SS2.1.p1.4.m4.1.1.3">𝑤</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.1.p1.4.m4.1c">\delta_{w}</annotation></semantics></math> can be represented as a boolean transition matrix, making matrix multiplication isomorphic to monoid composition.</p>
</div>
<div id="S5.SS2.2.p2" class="ltx_para">
<p id="S5.SS2.2.p2.1" class="ltx_p">Computing the transition monoid of a DFA allows recognizing valid words: compute the monoid element for a word by multiplying the elements for its tokens and then check whether the initial state maps to an accepting state.
In fact, a standard way to solve monoid word problems (e.g., for <math id="S5.SS2.2.p2.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.SS2.2.p2.1.m1.1a"><msub id="S5.SS2.2.p2.1.m1.1.1" xref="S5.SS2.2.p2.1.m1.1.1.cmml"><mi id="S5.SS2.2.p2.1.m1.1.1.2" xref="S5.SS2.2.p2.1.m1.1.1.2.cmml">S</mi><mn id="S5.SS2.2.p2.1.m1.1.1.3" xref="S5.SS2.2.p2.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.2.p2.1.m1.1b"><apply id="S5.SS2.2.p2.1.m1.1.1.cmml" xref="S5.SS2.2.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS2.2.p2.1.m1.1.1.1.cmml" xref="S5.SS2.2.p2.1.m1.1.1">subscript</csymbol><ci id="S5.SS2.2.p2.1.m1.1.1.2.cmml" xref="S5.SS2.2.p2.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S5.SS2.2.p2.1.m1.1.1.3.cmml" xref="S5.SS2.2.p2.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.2.p2.1.m1.1c">S_{5}</annotation></semantics></math>) with a DFA is simply to construct a DFA whose transition monoid is the monoid of interest.</p>
</div>
<div id="S5.SS2.3.p3" class="ltx_para">
<p id="S5.SS2.3.p3.11" class="ltx_p">Fix a DFA and its transition monoid <math id="S5.SS2.3.p3.1.m1.1" class="ltx_Math" alttext="\delta" display="inline"><semantics id="S5.SS2.3.p3.1.m1.1a"><mi id="S5.SS2.3.p3.1.m1.1.1" xref="S5.SS2.3.p3.1.m1.1.1.cmml">δ</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.1.m1.1b"><ci id="S5.SS2.3.p3.1.m1.1.1.cmml" xref="S5.SS2.3.p3.1.m1.1.1">𝛿</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.1.m1.1c">\delta</annotation></semantics></math>.
To complete the proof, we show that there exists an SSM that, for all <math id="S5.SS2.3.p3.2.m2.1" class="ltx_Math" alttext="w\in\Sigma^{*}" display="inline"><semantics id="S5.SS2.3.p3.2.m2.1a"><mrow id="S5.SS2.3.p3.2.m2.1.1" xref="S5.SS2.3.p3.2.m2.1.1.cmml"><mi id="S5.SS2.3.p3.2.m2.1.1.2" xref="S5.SS2.3.p3.2.m2.1.1.2.cmml">w</mi><mo id="S5.SS2.3.p3.2.m2.1.1.1" xref="S5.SS2.3.p3.2.m2.1.1.1.cmml">∈</mo><msup id="S5.SS2.3.p3.2.m2.1.1.3" xref="S5.SS2.3.p3.2.m2.1.1.3.cmml"><mi mathvariant="normal" id="S5.SS2.3.p3.2.m2.1.1.3.2" xref="S5.SS2.3.p3.2.m2.1.1.3.2.cmml">Σ</mi><mo id="S5.SS2.3.p3.2.m2.1.1.3.3" xref="S5.SS2.3.p3.2.m2.1.1.3.3.cmml">∗</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.2.m2.1b"><apply id="S5.SS2.3.p3.2.m2.1.1.cmml" xref="S5.SS2.3.p3.2.m2.1.1"><in id="S5.SS2.3.p3.2.m2.1.1.1.cmml" xref="S5.SS2.3.p3.2.m2.1.1.1"></in><ci id="S5.SS2.3.p3.2.m2.1.1.2.cmml" xref="S5.SS2.3.p3.2.m2.1.1.2">𝑤</ci><apply id="S5.SS2.3.p3.2.m2.1.1.3.cmml" xref="S5.SS2.3.p3.2.m2.1.1.3"><csymbol cd="ambiguous" id="S5.SS2.3.p3.2.m2.1.1.3.1.cmml" xref="S5.SS2.3.p3.2.m2.1.1.3">superscript</csymbol><ci id="S5.SS2.3.p3.2.m2.1.1.3.2.cmml" xref="S5.SS2.3.p3.2.m2.1.1.3.2">Σ</ci><times id="S5.SS2.3.p3.2.m2.1.1.3.3.cmml" xref="S5.SS2.3.p3.2.m2.1.1.3.3"></times></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.2.m2.1c">w\in\Sigma^{*}</annotation></semantics></math>, computes <math id="S5.SS2.3.p3.3.m3.1" class="ltx_Math" alttext="\delta_{w}" display="inline"><semantics id="S5.SS2.3.p3.3.m3.1a"><msub id="S5.SS2.3.p3.3.m3.1.1" xref="S5.SS2.3.p3.3.m3.1.1.cmml"><mi id="S5.SS2.3.p3.3.m3.1.1.2" xref="S5.SS2.3.p3.3.m3.1.1.2.cmml">δ</mi><mi id="S5.SS2.3.p3.3.m3.1.1.3" xref="S5.SS2.3.p3.3.m3.1.1.3.cmml">w</mi></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.3.m3.1b"><apply id="S5.SS2.3.p3.3.m3.1.1.cmml" xref="S5.SS2.3.p3.3.m3.1.1"><csymbol cd="ambiguous" id="S5.SS2.3.p3.3.m3.1.1.1.cmml" xref="S5.SS2.3.p3.3.m3.1.1">subscript</csymbol><ci id="S5.SS2.3.p3.3.m3.1.1.2.cmml" xref="S5.SS2.3.p3.3.m3.1.1.2">𝛿</ci><ci id="S5.SS2.3.p3.3.m3.1.1.3.cmml" xref="S5.SS2.3.p3.3.m3.1.1.3">𝑤</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.3.m3.1c">\delta_{w}</annotation></semantics></math> given input <math id="S5.SS2.3.p3.4.m4.1" class="ltx_Math" alttext="x=\$w" display="inline"><semantics id="S5.SS2.3.p3.4.m4.1a"><mrow id="S5.SS2.3.p3.4.m4.1.1" xref="S5.SS2.3.p3.4.m4.1.1.cmml"><mi id="S5.SS2.3.p3.4.m4.1.1.2" xref="S5.SS2.3.p3.4.m4.1.1.2.cmml">x</mi><mo rspace="0.1389em" id="S5.SS2.3.p3.4.m4.1.1.1" xref="S5.SS2.3.p3.4.m4.1.1.1.cmml">=</mo><mrow id="S5.SS2.3.p3.4.m4.1.1.3" xref="S5.SS2.3.p3.4.m4.1.1.3.cmml"><mo lspace="0.1389em" rspace="0.167em" id="S5.SS2.3.p3.4.m4.1.1.3.1" xref="S5.SS2.3.p3.4.m4.1.1.3.1.cmml">$</mo><mi id="S5.SS2.3.p3.4.m4.1.1.3.2" xref="S5.SS2.3.p3.4.m4.1.1.3.2.cmml">w</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.4.m4.1b"><apply id="S5.SS2.3.p3.4.m4.1.1.cmml" xref="S5.SS2.3.p3.4.m4.1.1"><eq id="S5.SS2.3.p3.4.m4.1.1.1.cmml" xref="S5.SS2.3.p3.4.m4.1.1.1"></eq><ci id="S5.SS2.3.p3.4.m4.1.1.2.cmml" xref="S5.SS2.3.p3.4.m4.1.1.2">𝑥</ci><apply id="S5.SS2.3.p3.4.m4.1.1.3.cmml" xref="S5.SS2.3.p3.4.m4.1.1.3"><csymbol cd="latexml" id="S5.SS2.3.p3.4.m4.1.1.3.1.cmml" xref="S5.SS2.3.p3.4.m4.1.1.3.1">currency-dollar</csymbol><ci id="S5.SS2.3.p3.4.m4.1.1.3.2.cmml" xref="S5.SS2.3.p3.4.m4.1.1.3.2">𝑤</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.4.m4.1c">x=\$w</annotation></semantics></math>. We view indices in <math id="S5.SS2.3.p3.5.m5.1" class="ltx_Math" alttext="h_{i}" display="inline"><semantics id="S5.SS2.3.p3.5.m5.1a"><msub id="S5.SS2.3.p3.5.m5.1.1" xref="S5.SS2.3.p3.5.m5.1.1.cmml"><mi id="S5.SS2.3.p3.5.m5.1.1.2" xref="S5.SS2.3.p3.5.m5.1.1.2.cmml">h</mi><mi id="S5.SS2.3.p3.5.m5.1.1.3" xref="S5.SS2.3.p3.5.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.5.m5.1b"><apply id="S5.SS2.3.p3.5.m5.1.1.cmml" xref="S5.SS2.3.p3.5.m5.1.1"><csymbol cd="ambiguous" id="S5.SS2.3.p3.5.m5.1.1.1.cmml" xref="S5.SS2.3.p3.5.m5.1.1">subscript</csymbol><ci id="S5.SS2.3.p3.5.m5.1.1.2.cmml" xref="S5.SS2.3.p3.5.m5.1.1.2">ℎ</ci><ci id="S5.SS2.3.p3.5.m5.1.1.3.cmml" xref="S5.SS2.3.p3.5.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.5.m5.1c">h_{i}</annotation></semantics></math> as states and define <math id="S5.SS2.3.p3.6.m6.1" class="ltx_Math" alttext="B\$" display="inline"><semantics id="S5.SS2.3.p3.6.m6.1a"><mrow id="S5.SS2.3.p3.6.m6.1.1" xref="S5.SS2.3.p3.6.m6.1.1.cmml"><mi id="S5.SS2.3.p3.6.m6.1.1.2" xref="S5.SS2.3.p3.6.m6.1.1.2.cmml">B</mi><mo lspace="0.167em" rspace="0em" id="S5.SS2.3.p3.6.m6.1.1.1" xref="S5.SS2.3.p3.6.m6.1.1.1.cmml">​</mo><mo id="S5.SS2.3.p3.6.m6.1.1.3" xref="S5.SS2.3.p3.6.m6.1.1.3.cmml">$</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.6.m6.1b"><apply id="S5.SS2.3.p3.6.m6.1.1.cmml" xref="S5.SS2.3.p3.6.m6.1.1"><times id="S5.SS2.3.p3.6.m6.1.1.1.cmml" xref="S5.SS2.3.p3.6.m6.1.1.1"></times><ci id="S5.SS2.3.p3.6.m6.1.1.2.cmml" xref="S5.SS2.3.p3.6.m6.1.1.2">𝐵</ci><csymbol cd="latexml" id="S5.SS2.3.p3.6.m6.1.1.3.cmml" xref="S5.SS2.3.p3.6.m6.1.1.3">currency-dollar</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.6.m6.1c">B\$</annotation></semantics></math> as <math id="S5.SS2.3.p3.7.m7.1" class="ltx_Math" alttext="1" display="inline"><semantics id="S5.SS2.3.p3.7.m7.1a"><mn id="S5.SS2.3.p3.7.m7.1.1" xref="S5.SS2.3.p3.7.m7.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.7.m7.1b"><cn type="integer" id="S5.SS2.3.p3.7.m7.1.1.cmml" xref="S5.SS2.3.p3.7.m7.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.7.m7.1c">1</annotation></semantics></math> at each accepting state, and <math id="S5.SS2.3.p3.8.m8.1" class="ltx_Math" alttext="0" display="inline"><semantics id="S5.SS2.3.p3.8.m8.1a"><mn id="S5.SS2.3.p3.8.m8.1.1" xref="S5.SS2.3.p3.8.m8.1.1.cmml">0</mn><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.8.m8.1b"><cn type="integer" id="S5.SS2.3.p3.8.m8.1.1.cmml" xref="S5.SS2.3.p3.8.m8.1.1">0</cn></annotation-xml></semantics></math> elsewhere. For all other <math id="S5.SS2.3.p3.9.m9.1" class="ltx_Math" alttext="\sigma" display="inline"><semantics id="S5.SS2.3.p3.9.m9.1a"><mi id="S5.SS2.3.p3.9.m9.1.1" xref="S5.SS2.3.p3.9.m9.1.1.cmml">σ</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.9.m9.1b"><ci id="S5.SS2.3.p3.9.m9.1.1.cmml" xref="S5.SS2.3.p3.9.m9.1.1">𝜎</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.9.m9.1c">\sigma</annotation></semantics></math>, we let <math id="S5.SS2.3.p3.10.m10.1" class="ltx_Math" alttext="B\sigma=\vec{0}" display="inline"><semantics id="S5.SS2.3.p3.10.m10.1a"><mrow id="S5.SS2.3.p3.10.m10.1.1" xref="S5.SS2.3.p3.10.m10.1.1.cmml"><mrow id="S5.SS2.3.p3.10.m10.1.1.2" xref="S5.SS2.3.p3.10.m10.1.1.2.cmml"><mi id="S5.SS2.3.p3.10.m10.1.1.2.2" xref="S5.SS2.3.p3.10.m10.1.1.2.2.cmml">B</mi><mo lspace="0em" rspace="0em" id="S5.SS2.3.p3.10.m10.1.1.2.1" xref="S5.SS2.3.p3.10.m10.1.1.2.1.cmml">​</mo><mi id="S5.SS2.3.p3.10.m10.1.1.2.3" xref="S5.SS2.3.p3.10.m10.1.1.2.3.cmml">σ</mi></mrow><mo id="S5.SS2.3.p3.10.m10.1.1.1" xref="S5.SS2.3.p3.10.m10.1.1.1.cmml">=</mo><mover accent="true" id="S5.SS2.3.p3.10.m10.1.1.3" xref="S5.SS2.3.p3.10.m10.1.1.3.cmml"><mn id="S5.SS2.3.p3.10.m10.1.1.3.2" xref="S5.SS2.3.p3.10.m10.1.1.3.2.cmml">0</mn><mo stretchy="false" id="S5.SS2.3.p3.10.m10.1.1.3.1" xref="S5.SS2.3.p3.10.m10.1.1.3.1.cmml">→</mo></mover></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.10.m10.1b"><apply id="S5.SS2.3.p3.10.m10.1.1.cmml" xref="S5.SS2.3.p3.10.m10.1.1"><eq id="S5.SS2.3.p3.10.m10.1.1.1.cmml" xref="S5.SS2.3.p3.10.m10.1.1.1"></eq><apply id="S5.SS2.3.p3.10.m10.1.1.2.cmml" xref="S5.SS2.3.p3.10.m10.1.1.2"><times id="S5.SS2.3.p3.10.m10.1.1.2.1.cmml" xref="S5.SS2.3.p3.10.m10.1.1.2.1"></times><ci id="S5.SS2.3.p3.10.m10.1.1.2.2.cmml" xref="S5.SS2.3.p3.10.m10.1.1.2.2">𝐵</ci><ci id="S5.SS2.3.p3.10.m10.1.1.2.3.cmml" xref="S5.SS2.3.p3.10.m10.1.1.2.3">𝜎</ci></apply><apply id="S5.SS2.3.p3.10.m10.1.1.3.cmml" xref="S5.SS2.3.p3.10.m10.1.1.3"><ci id="S5.SS2.3.p3.10.m10.1.1.3.1.cmml" xref="S5.SS2.3.p3.10.m10.1.1.3.1">→</ci><cn type="integer" id="S5.SS2.3.p3.10.m10.1.1.3.2.cmml" xref="S5.SS2.3.p3.10.m10.1.1.3.2">0</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.10.m10.1c">B\sigma=\vec{0}</annotation></semantics></math>.
This reduces the convolutional form of <math id="S5.SS2.3.p3.11.m11.1" class="ltx_Math" alttext="h_{i}" display="inline"><semantics id="S5.SS2.3.p3.11.m11.1a"><msub id="S5.SS2.3.p3.11.m11.1.1" xref="S5.SS2.3.p3.11.m11.1.1.cmml"><mi id="S5.SS2.3.p3.11.m11.1.1.2" xref="S5.SS2.3.p3.11.m11.1.1.2.cmml">h</mi><mi id="S5.SS2.3.p3.11.m11.1.1.3" xref="S5.SS2.3.p3.11.m11.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.11.m11.1b"><apply id="S5.SS2.3.p3.11.m11.1.1.cmml" xref="S5.SS2.3.p3.11.m11.1.1"><csymbol cd="ambiguous" id="S5.SS2.3.p3.11.m11.1.1.1.cmml" xref="S5.SS2.3.p3.11.m11.1.1">subscript</csymbol><ci id="S5.SS2.3.p3.11.m11.1.1.2.cmml" xref="S5.SS2.3.p3.11.m11.1.1.2">ℎ</ci><ci id="S5.SS2.3.p3.11.m11.1.1.3.cmml" xref="S5.SS2.3.p3.11.m11.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.11.m11.1c">h_{i}</annotation></semantics></math> to have a single term:</p>
<table id="S5.Ex5" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S5.Ex5.m1.1" class="ltx_Math" alttext="h_{i}=\prod_{k=2}^{i}A(x_{k})." display="block"><semantics id="S5.Ex5.m1.1a"><mrow id="S5.Ex5.m1.1.1.1" xref="S5.Ex5.m1.1.1.1.1.cmml"><mrow id="S5.Ex5.m1.1.1.1.1" xref="S5.Ex5.m1.1.1.1.1.cmml"><msub id="S5.Ex5.m1.1.1.1.1.3" xref="S5.Ex5.m1.1.1.1.1.3.cmml"><mi id="S5.Ex5.m1.1.1.1.1.3.2" xref="S5.Ex5.m1.1.1.1.1.3.2.cmml">h</mi><mi id="S5.Ex5.m1.1.1.1.1.3.3" xref="S5.Ex5.m1.1.1.1.1.3.3.cmml">i</mi></msub><mo rspace="0.111em" id="S5.Ex5.m1.1.1.1.1.2" xref="S5.Ex5.m1.1.1.1.1.2.cmml">=</mo><mrow id="S5.Ex5.m1.1.1.1.1.1" xref="S5.Ex5.m1.1.1.1.1.1.cmml"><munderover id="S5.Ex5.m1.1.1.1.1.1.2" xref="S5.Ex5.m1.1.1.1.1.1.2.cmml"><mo movablelimits="false" id="S5.Ex5.m1.1.1.1.1.1.2.2.2" xref="S5.Ex5.m1.1.1.1.1.1.2.2.2.cmml">∏</mo><mrow id="S5.Ex5.m1.1.1.1.1.1.2.2.3" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.cmml"><mi id="S5.Ex5.m1.1.1.1.1.1.2.2.3.2" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.2.cmml">k</mi><mo id="S5.Ex5.m1.1.1.1.1.1.2.2.3.1" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.1.cmml">=</mo><mn id="S5.Ex5.m1.1.1.1.1.1.2.2.3.3" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.3.cmml">2</mn></mrow><mi id="S5.Ex5.m1.1.1.1.1.1.2.3" xref="S5.Ex5.m1.1.1.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S5.Ex5.m1.1.1.1.1.1.1" xref="S5.Ex5.m1.1.1.1.1.1.1.cmml"><mi id="S5.Ex5.m1.1.1.1.1.1.1.3" xref="S5.Ex5.m1.1.1.1.1.1.1.3.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.Ex5.m1.1.1.1.1.1.1.2" xref="S5.Ex5.m1.1.1.1.1.1.1.2.cmml">​</mo><mrow id="S5.Ex5.m1.1.1.1.1.1.1.1.1" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.Ex5.m1.1.1.1.1.1.1.1.1.2" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.cmml">(</mo><msub id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.2" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mi id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.3" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.3.cmml">k</mi></msub><mo stretchy="false" id="S5.Ex5.m1.1.1.1.1.1.1.1.1.3" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow></mrow><mo lspace="0em" id="S5.Ex5.m1.1.1.1.2" xref="S5.Ex5.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.Ex5.m1.1b"><apply id="S5.Ex5.m1.1.1.1.1.cmml" xref="S5.Ex5.m1.1.1.1"><eq id="S5.Ex5.m1.1.1.1.1.2.cmml" xref="S5.Ex5.m1.1.1.1.1.2"></eq><apply id="S5.Ex5.m1.1.1.1.1.3.cmml" xref="S5.Ex5.m1.1.1.1.1.3"><csymbol cd="ambiguous" id="S5.Ex5.m1.1.1.1.1.3.1.cmml" xref="S5.Ex5.m1.1.1.1.1.3">subscript</csymbol><ci id="S5.Ex5.m1.1.1.1.1.3.2.cmml" xref="S5.Ex5.m1.1.1.1.1.3.2">ℎ</ci><ci id="S5.Ex5.m1.1.1.1.1.3.3.cmml" xref="S5.Ex5.m1.1.1.1.1.3.3">𝑖</ci></apply><apply id="S5.Ex5.m1.1.1.1.1.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1"><apply id="S5.Ex5.m1.1.1.1.1.1.2.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.Ex5.m1.1.1.1.1.1.2.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2">superscript</csymbol><apply id="S5.Ex5.m1.1.1.1.1.1.2.2.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.Ex5.m1.1.1.1.1.1.2.2.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="S5.Ex5.m1.1.1.1.1.1.2.2.2.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2.2.2">product</csymbol><apply id="S5.Ex5.m1.1.1.1.1.1.2.2.3.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3"><eq id="S5.Ex5.m1.1.1.1.1.1.2.2.3.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.1"></eq><ci id="S5.Ex5.m1.1.1.1.1.1.2.2.3.2.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.2">𝑘</ci><cn type="integer" id="S5.Ex5.m1.1.1.1.1.1.2.2.3.3.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2.2.3.3">2</cn></apply></apply><ci id="S5.Ex5.m1.1.1.1.1.1.2.3.cmml" xref="S5.Ex5.m1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S5.Ex5.m1.1.1.1.1.1.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1"><times id="S5.Ex5.m1.1.1.1.1.1.1.2.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1.2"></times><ci id="S5.Ex5.m1.1.1.1.1.1.1.3.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1.3">𝐴</ci><apply id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.2">𝑥</ci><ci id="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S5.Ex5.m1.1.1.1.1.1.1.1.1.1.3">𝑘</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Ex5.m1.1c">h_{i}=\prod_{k=2}^{i}A(x_{k}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S5.SS2.3.p3.13" class="ltx_p">Now, let <math id="S5.SS2.3.p3.12.m1.1" class="ltx_Math" alttext="A(\sigma)=\delta_{\sigma}" display="inline"><semantics id="S5.SS2.3.p3.12.m1.1a"><mrow id="S5.SS2.3.p3.12.m1.1.2" xref="S5.SS2.3.p3.12.m1.1.2.cmml"><mrow id="S5.SS2.3.p3.12.m1.1.2.2" xref="S5.SS2.3.p3.12.m1.1.2.2.cmml"><mi id="S5.SS2.3.p3.12.m1.1.2.2.2" xref="S5.SS2.3.p3.12.m1.1.2.2.2.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.SS2.3.p3.12.m1.1.2.2.1" xref="S5.SS2.3.p3.12.m1.1.2.2.1.cmml">​</mo><mrow id="S5.SS2.3.p3.12.m1.1.2.2.3.2" xref="S5.SS2.3.p3.12.m1.1.2.2.cmml"><mo stretchy="false" id="S5.SS2.3.p3.12.m1.1.2.2.3.2.1" xref="S5.SS2.3.p3.12.m1.1.2.2.cmml">(</mo><mi id="S5.SS2.3.p3.12.m1.1.1" xref="S5.SS2.3.p3.12.m1.1.1.cmml">σ</mi><mo stretchy="false" id="S5.SS2.3.p3.12.m1.1.2.2.3.2.2" xref="S5.SS2.3.p3.12.m1.1.2.2.cmml">)</mo></mrow></mrow><mo id="S5.SS2.3.p3.12.m1.1.2.1" xref="S5.SS2.3.p3.12.m1.1.2.1.cmml">=</mo><msub id="S5.SS2.3.p3.12.m1.1.2.3" xref="S5.SS2.3.p3.12.m1.1.2.3.cmml"><mi id="S5.SS2.3.p3.12.m1.1.2.3.2" xref="S5.SS2.3.p3.12.m1.1.2.3.2.cmml">δ</mi><mi id="S5.SS2.3.p3.12.m1.1.2.3.3" xref="S5.SS2.3.p3.12.m1.1.2.3.3.cmml">σ</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.12.m1.1b"><apply id="S5.SS2.3.p3.12.m1.1.2.cmml" xref="S5.SS2.3.p3.12.m1.1.2"><eq id="S5.SS2.3.p3.12.m1.1.2.1.cmml" xref="S5.SS2.3.p3.12.m1.1.2.1"></eq><apply id="S5.SS2.3.p3.12.m1.1.2.2.cmml" xref="S5.SS2.3.p3.12.m1.1.2.2"><times id="S5.SS2.3.p3.12.m1.1.2.2.1.cmml" xref="S5.SS2.3.p3.12.m1.1.2.2.1"></times><ci id="S5.SS2.3.p3.12.m1.1.2.2.2.cmml" xref="S5.SS2.3.p3.12.m1.1.2.2.2">𝐴</ci><ci id="S5.SS2.3.p3.12.m1.1.1.cmml" xref="S5.SS2.3.p3.12.m1.1.1">𝜎</ci></apply><apply id="S5.SS2.3.p3.12.m1.1.2.3.cmml" xref="S5.SS2.3.p3.12.m1.1.2.3"><csymbol cd="ambiguous" id="S5.SS2.3.p3.12.m1.1.2.3.1.cmml" xref="S5.SS2.3.p3.12.m1.1.2.3">subscript</csymbol><ci id="S5.SS2.3.p3.12.m1.1.2.3.2.cmml" xref="S5.SS2.3.p3.12.m1.1.2.3.2">𝛿</ci><ci id="S5.SS2.3.p3.12.m1.1.2.3.3.cmml" xref="S5.SS2.3.p3.12.m1.1.2.3.3">𝜎</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.12.m1.1c">A(\sigma)=\delta_{\sigma}</annotation></semantics></math> for <math id="S5.SS2.3.p3.13.m2.1" class="ltx_Math" alttext="\sigma\in\Sigma" display="inline"><semantics id="S5.SS2.3.p3.13.m2.1a"><mrow id="S5.SS2.3.p3.13.m2.1.1" xref="S5.SS2.3.p3.13.m2.1.1.cmml"><mi id="S5.SS2.3.p3.13.m2.1.1.2" xref="S5.SS2.3.p3.13.m2.1.1.2.cmml">σ</mi><mo id="S5.SS2.3.p3.13.m2.1.1.1" xref="S5.SS2.3.p3.13.m2.1.1.1.cmml">∈</mo><mi mathvariant="normal" id="S5.SS2.3.p3.13.m2.1.1.3" xref="S5.SS2.3.p3.13.m2.1.1.3.cmml">Σ</mi></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.13.m2.1b"><apply id="S5.SS2.3.p3.13.m2.1.1.cmml" xref="S5.SS2.3.p3.13.m2.1.1"><in id="S5.SS2.3.p3.13.m2.1.1.1.cmml" xref="S5.SS2.3.p3.13.m2.1.1.1"></in><ci id="S5.SS2.3.p3.13.m2.1.1.2.cmml" xref="S5.SS2.3.p3.13.m2.1.1.2">𝜎</ci><ci id="S5.SS2.3.p3.13.m2.1.1.3.cmml" xref="S5.SS2.3.p3.13.m2.1.1.3">Σ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.13.m2.1c">\sigma\in\Sigma</annotation></semantics></math>. It follows that</p>
<table id="S5.Ex6" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S5.Ex6.m1.1" class="ltx_Math" alttext="h_{i}=\prod_{k=2}^{i}A(x_{k})=\prod_{k=2}^{i}\delta_{x_{k}}." display="block"><semantics id="S5.Ex6.m1.1a"><mrow id="S5.Ex6.m1.1.1.1" xref="S5.Ex6.m1.1.1.1.1.cmml"><mrow id="S5.Ex6.m1.1.1.1.1" xref="S5.Ex6.m1.1.1.1.1.cmml"><msub id="S5.Ex6.m1.1.1.1.1.3" xref="S5.Ex6.m1.1.1.1.1.3.cmml"><mi id="S5.Ex6.m1.1.1.1.1.3.2" xref="S5.Ex6.m1.1.1.1.1.3.2.cmml">h</mi><mi id="S5.Ex6.m1.1.1.1.1.3.3" xref="S5.Ex6.m1.1.1.1.1.3.3.cmml">i</mi></msub><mo rspace="0.111em" id="S5.Ex6.m1.1.1.1.1.4" xref="S5.Ex6.m1.1.1.1.1.4.cmml">=</mo><mrow id="S5.Ex6.m1.1.1.1.1.1" xref="S5.Ex6.m1.1.1.1.1.1.cmml"><munderover id="S5.Ex6.m1.1.1.1.1.1.2" xref="S5.Ex6.m1.1.1.1.1.1.2.cmml"><mo movablelimits="false" id="S5.Ex6.m1.1.1.1.1.1.2.2.2" xref="S5.Ex6.m1.1.1.1.1.1.2.2.2.cmml">∏</mo><mrow id="S5.Ex6.m1.1.1.1.1.1.2.2.3" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.cmml"><mi id="S5.Ex6.m1.1.1.1.1.1.2.2.3.2" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.2.cmml">k</mi><mo id="S5.Ex6.m1.1.1.1.1.1.2.2.3.1" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.1.cmml">=</mo><mn id="S5.Ex6.m1.1.1.1.1.1.2.2.3.3" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.3.cmml">2</mn></mrow><mi id="S5.Ex6.m1.1.1.1.1.1.2.3" xref="S5.Ex6.m1.1.1.1.1.1.2.3.cmml">i</mi></munderover><mrow id="S5.Ex6.m1.1.1.1.1.1.1" xref="S5.Ex6.m1.1.1.1.1.1.1.cmml"><mi id="S5.Ex6.m1.1.1.1.1.1.1.3" xref="S5.Ex6.m1.1.1.1.1.1.1.3.cmml">A</mi><mo lspace="0em" rspace="0em" id="S5.Ex6.m1.1.1.1.1.1.1.2" xref="S5.Ex6.m1.1.1.1.1.1.1.2.cmml">​</mo><mrow id="S5.Ex6.m1.1.1.1.1.1.1.1.1" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S5.Ex6.m1.1.1.1.1.1.1.1.1.2" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.cmml">(</mo><msub id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.2" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mi id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.3" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.3.cmml">k</mi></msub><mo stretchy="false" id="S5.Ex6.m1.1.1.1.1.1.1.1.1.3" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo rspace="0.111em" id="S5.Ex6.m1.1.1.1.1.5" xref="S5.Ex6.m1.1.1.1.1.5.cmml">=</mo><mrow id="S5.Ex6.m1.1.1.1.1.6" xref="S5.Ex6.m1.1.1.1.1.6.cmml"><munderover id="S5.Ex6.m1.1.1.1.1.6.1" xref="S5.Ex6.m1.1.1.1.1.6.1.cmml"><mo movablelimits="false" id="S5.Ex6.m1.1.1.1.1.6.1.2.2" xref="S5.Ex6.m1.1.1.1.1.6.1.2.2.cmml">∏</mo><mrow id="S5.Ex6.m1.1.1.1.1.6.1.2.3" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.cmml"><mi id="S5.Ex6.m1.1.1.1.1.6.1.2.3.2" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.2.cmml">k</mi><mo id="S5.Ex6.m1.1.1.1.1.6.1.2.3.1" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.1.cmml">=</mo><mn id="S5.Ex6.m1.1.1.1.1.6.1.2.3.3" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.3.cmml">2</mn></mrow><mi id="S5.Ex6.m1.1.1.1.1.6.1.3" xref="S5.Ex6.m1.1.1.1.1.6.1.3.cmml">i</mi></munderover><msub id="S5.Ex6.m1.1.1.1.1.6.2" xref="S5.Ex6.m1.1.1.1.1.6.2.cmml"><mi id="S5.Ex6.m1.1.1.1.1.6.2.2" xref="S5.Ex6.m1.1.1.1.1.6.2.2.cmml">δ</mi><msub id="S5.Ex6.m1.1.1.1.1.6.2.3" xref="S5.Ex6.m1.1.1.1.1.6.2.3.cmml"><mi id="S5.Ex6.m1.1.1.1.1.6.2.3.2" xref="S5.Ex6.m1.1.1.1.1.6.2.3.2.cmml">x</mi><mi id="S5.Ex6.m1.1.1.1.1.6.2.3.3" xref="S5.Ex6.m1.1.1.1.1.6.2.3.3.cmml">k</mi></msub></msub></mrow></mrow><mo lspace="0em" id="S5.Ex6.m1.1.1.1.2" xref="S5.Ex6.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S5.Ex6.m1.1b"><apply id="S5.Ex6.m1.1.1.1.1.cmml" xref="S5.Ex6.m1.1.1.1"><and id="S5.Ex6.m1.1.1.1.1a.cmml" xref="S5.Ex6.m1.1.1.1"></and><apply id="S5.Ex6.m1.1.1.1.1b.cmml" xref="S5.Ex6.m1.1.1.1"><eq id="S5.Ex6.m1.1.1.1.1.4.cmml" xref="S5.Ex6.m1.1.1.1.1.4"></eq><apply id="S5.Ex6.m1.1.1.1.1.3.cmml" xref="S5.Ex6.m1.1.1.1.1.3"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.3.1.cmml" xref="S5.Ex6.m1.1.1.1.1.3">subscript</csymbol><ci id="S5.Ex6.m1.1.1.1.1.3.2.cmml" xref="S5.Ex6.m1.1.1.1.1.3.2">ℎ</ci><ci id="S5.Ex6.m1.1.1.1.1.3.3.cmml" xref="S5.Ex6.m1.1.1.1.1.3.3">𝑖</ci></apply><apply id="S5.Ex6.m1.1.1.1.1.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1"><apply id="S5.Ex6.m1.1.1.1.1.1.2.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.1.2.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2">superscript</csymbol><apply id="S5.Ex6.m1.1.1.1.1.1.2.2.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.1.2.2.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="S5.Ex6.m1.1.1.1.1.1.2.2.2.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2.2.2">product</csymbol><apply id="S5.Ex6.m1.1.1.1.1.1.2.2.3.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3"><eq id="S5.Ex6.m1.1.1.1.1.1.2.2.3.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.1"></eq><ci id="S5.Ex6.m1.1.1.1.1.1.2.2.3.2.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.2">𝑘</ci><cn type="integer" id="S5.Ex6.m1.1.1.1.1.1.2.2.3.3.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2.2.3.3">2</cn></apply></apply><ci id="S5.Ex6.m1.1.1.1.1.1.2.3.cmml" xref="S5.Ex6.m1.1.1.1.1.1.2.3">𝑖</ci></apply><apply id="S5.Ex6.m1.1.1.1.1.1.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1"><times id="S5.Ex6.m1.1.1.1.1.1.1.2.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1.2"></times><ci id="S5.Ex6.m1.1.1.1.1.1.1.3.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1.3">𝐴</ci><apply id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.2">𝑥</ci><ci id="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S5.Ex6.m1.1.1.1.1.1.1.1.1.1.3">𝑘</ci></apply></apply></apply></apply><apply id="S5.Ex6.m1.1.1.1.1c.cmml" xref="S5.Ex6.m1.1.1.1"><eq id="S5.Ex6.m1.1.1.1.1.5.cmml" xref="S5.Ex6.m1.1.1.1.1.5"></eq><share href="#S5.Ex6.m1.1.1.1.1.1.cmml" id="S5.Ex6.m1.1.1.1.1d.cmml" xref="S5.Ex6.m1.1.1.1"></share><apply id="S5.Ex6.m1.1.1.1.1.6.cmml" xref="S5.Ex6.m1.1.1.1.1.6"><apply id="S5.Ex6.m1.1.1.1.1.6.1.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.6.1.1.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1">superscript</csymbol><apply id="S5.Ex6.m1.1.1.1.1.6.1.2.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.6.1.2.1.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1">subscript</csymbol><csymbol cd="latexml" id="S5.Ex6.m1.1.1.1.1.6.1.2.2.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1.2.2">product</csymbol><apply id="S5.Ex6.m1.1.1.1.1.6.1.2.3.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3"><eq id="S5.Ex6.m1.1.1.1.1.6.1.2.3.1.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.1"></eq><ci id="S5.Ex6.m1.1.1.1.1.6.1.2.3.2.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.2">𝑘</ci><cn type="integer" id="S5.Ex6.m1.1.1.1.1.6.1.2.3.3.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1.2.3.3">2</cn></apply></apply><ci id="S5.Ex6.m1.1.1.1.1.6.1.3.cmml" xref="S5.Ex6.m1.1.1.1.1.6.1.3">𝑖</ci></apply><apply id="S5.Ex6.m1.1.1.1.1.6.2.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.6.2.1.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2">subscript</csymbol><ci id="S5.Ex6.m1.1.1.1.1.6.2.2.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2.2">𝛿</ci><apply id="S5.Ex6.m1.1.1.1.1.6.2.3.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2.3"><csymbol cd="ambiguous" id="S5.Ex6.m1.1.1.1.1.6.2.3.1.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2.3">subscript</csymbol><ci id="S5.Ex6.m1.1.1.1.1.6.2.3.2.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2.3.2">𝑥</ci><ci id="S5.Ex6.m1.1.1.1.1.6.2.3.3.cmml" xref="S5.Ex6.m1.1.1.1.1.6.2.3.3">𝑘</ci></apply></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Ex6.m1.1c">h_{i}=\prod_{k=2}^{i}A(x_{k})=\prod_{k=2}^{i}\delta_{x_{k}}.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S5.SS2.3.p3.16" class="ltx_p">Since <math id="S5.SS2.3.p3.14.m1.1" class="ltx_Math" alttext="x=\$w" display="inline"><semantics id="S5.SS2.3.p3.14.m1.1a"><mrow id="S5.SS2.3.p3.14.m1.1.1" xref="S5.SS2.3.p3.14.m1.1.1.cmml"><mi id="S5.SS2.3.p3.14.m1.1.1.2" xref="S5.SS2.3.p3.14.m1.1.1.2.cmml">x</mi><mo rspace="0.1389em" id="S5.SS2.3.p3.14.m1.1.1.1" xref="S5.SS2.3.p3.14.m1.1.1.1.cmml">=</mo><mrow id="S5.SS2.3.p3.14.m1.1.1.3" xref="S5.SS2.3.p3.14.m1.1.1.3.cmml"><mo lspace="0.1389em" rspace="0.167em" id="S5.SS2.3.p3.14.m1.1.1.3.1" xref="S5.SS2.3.p3.14.m1.1.1.3.1.cmml">$</mo><mi id="S5.SS2.3.p3.14.m1.1.1.3.2" xref="S5.SS2.3.p3.14.m1.1.1.3.2.cmml">w</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.14.m1.1b"><apply id="S5.SS2.3.p3.14.m1.1.1.cmml" xref="S5.SS2.3.p3.14.m1.1.1"><eq id="S5.SS2.3.p3.14.m1.1.1.1.cmml" xref="S5.SS2.3.p3.14.m1.1.1.1"></eq><ci id="S5.SS2.3.p3.14.m1.1.1.2.cmml" xref="S5.SS2.3.p3.14.m1.1.1.2">𝑥</ci><apply id="S5.SS2.3.p3.14.m1.1.1.3.cmml" xref="S5.SS2.3.p3.14.m1.1.1.3"><csymbol cd="latexml" id="S5.SS2.3.p3.14.m1.1.1.3.1.cmml" xref="S5.SS2.3.p3.14.m1.1.1.3.1">currency-dollar</csymbol><ci id="S5.SS2.3.p3.14.m1.1.1.3.2.cmml" xref="S5.SS2.3.p3.14.m1.1.1.3.2">𝑤</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.14.m1.1c">x=\$w</annotation></semantics></math>, we conclude that <math id="S5.SS2.3.p3.15.m2.1" class="ltx_Math" alttext="h_{\left\lvert x\right\rvert}" display="inline"><semantics id="S5.SS2.3.p3.15.m2.1a"><msub id="S5.SS2.3.p3.15.m2.1.2" xref="S5.SS2.3.p3.15.m2.1.2.cmml"><mi id="S5.SS2.3.p3.15.m2.1.2.2" xref="S5.SS2.3.p3.15.m2.1.2.2.cmml">h</mi><mrow id="S5.SS2.3.p3.15.m2.1.1.1.3" xref="S5.SS2.3.p3.15.m2.1.1.1.2.cmml"><mo id="S5.SS2.3.p3.15.m2.1.1.1.3.1" xref="S5.SS2.3.p3.15.m2.1.1.1.2.1.cmml">|</mo><mi id="S5.SS2.3.p3.15.m2.1.1.1.1" xref="S5.SS2.3.p3.15.m2.1.1.1.1.cmml">x</mi><mo id="S5.SS2.3.p3.15.m2.1.1.1.3.2" xref="S5.SS2.3.p3.15.m2.1.1.1.2.1.cmml">|</mo></mrow></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.15.m2.1b"><apply id="S5.SS2.3.p3.15.m2.1.2.cmml" xref="S5.SS2.3.p3.15.m2.1.2"><csymbol cd="ambiguous" id="S5.SS2.3.p3.15.m2.1.2.1.cmml" xref="S5.SS2.3.p3.15.m2.1.2">subscript</csymbol><ci id="S5.SS2.3.p3.15.m2.1.2.2.cmml" xref="S5.SS2.3.p3.15.m2.1.2.2">ℎ</ci><apply id="S5.SS2.3.p3.15.m2.1.1.1.2.cmml" xref="S5.SS2.3.p3.15.m2.1.1.1.3"><abs id="S5.SS2.3.p3.15.m2.1.1.1.2.1.cmml" xref="S5.SS2.3.p3.15.m2.1.1.1.3.1"></abs><ci id="S5.SS2.3.p3.15.m2.1.1.1.1.cmml" xref="S5.SS2.3.p3.15.m2.1.1.1.1">𝑥</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.15.m2.1c">h_{\left\lvert x\right\rvert}</annotation></semantics></math> is <math id="S5.SS2.3.p3.16.m3.1" class="ltx_Math" alttext="\delta_{w}" display="inline"><semantics id="S5.SS2.3.p3.16.m3.1a"><msub id="S5.SS2.3.p3.16.m3.1.1" xref="S5.SS2.3.p3.16.m3.1.1.cmml"><mi id="S5.SS2.3.p3.16.m3.1.1.2" xref="S5.SS2.3.p3.16.m3.1.1.2.cmml">δ</mi><mi id="S5.SS2.3.p3.16.m3.1.1.3" xref="S5.SS2.3.p3.16.m3.1.1.3.cmml">w</mi></msub><annotation-xml encoding="MathML-Content" id="S5.SS2.3.p3.16.m3.1b"><apply id="S5.SS2.3.p3.16.m3.1.1.cmml" xref="S5.SS2.3.p3.16.m3.1.1"><csymbol cd="ambiguous" id="S5.SS2.3.p3.16.m3.1.1.1.cmml" xref="S5.SS2.3.p3.16.m3.1.1">subscript</csymbol><ci id="S5.SS2.3.p3.16.m3.1.1.2.cmml" xref="S5.SS2.3.p3.16.m3.1.1.2">𝛿</ci><ci id="S5.SS2.3.p3.16.m3.1.1.3.cmml" xref="S5.SS2.3.p3.16.m3.1.1.3">𝑤</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.3.p3.16.m3.1c">\delta_{w}</annotation></semantics></math>.
∎</p>
</div>
</div>
</section>
<section id="S5.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.3 </span>Discussion</h3>

<div id="S5.SS3.p1" class="ltx_para">
<p id="S5.SS3.p1.1" class="ltx_p"><a href="#S5.Thmtheorem1" title="Theorem 5.1. ‣ 5.1 Via Nonlinearities ‣ 5 Extending the Expressive Power of SSMs ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Theorems</span> <span class="ltx_text ltx_ref_tag">5.1</span></a> and <a href="#S5.Thmtheorem2" title="Theorem 5.2. ‣ 5.2 Via Input-Dependent Transition Matrices ‣ 5 Extending the Expressive Power of SSMs ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5.2</span></a> show that two minimal extensions of the SSM enable expressive power outside <math id="S5.SS3.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S5.SS3.p1.1.m1.1a"><msup id="S5.SS3.p1.1.m1.1.1" xref="S5.SS3.p1.1.m1.1.1.cmml"><mi id="S5.SS3.p1.1.m1.1.1.2" xref="S5.SS3.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S5.SS3.p1.1.m1.1.1.3" xref="S5.SS3.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S5.SS3.p1.1.m1.1b"><apply id="S5.SS3.p1.1.m1.1.1.cmml" xref="S5.SS3.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS3.p1.1.m1.1.1.1.cmml" xref="S5.SS3.p1.1.m1.1.1">superscript</csymbol><ci id="S5.SS3.p1.1.m1.1.1.2.cmml" xref="S5.SS3.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S5.SS3.p1.1.m1.1.1.3.cmml" xref="S5.SS3.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math>, allowing the model to solve hard state-tracking problems:</p>
</div>
<div id="S5.Thmtheorem3" class="ltx_theorem ltx_theorem_corollary">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="S5.Thmtheorem3.1.1.1" class="ltx_text ltx_font_bold">Corollary 5.3</span></span><span id="S5.Thmtheorem3.2.2" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="S5.Thmtheorem3.p1" class="ltx_para">
<p id="S5.Thmtheorem3.p1.2" class="ltx_p"><span id="S5.Thmtheorem3.p1.2.2" class="ltx_text ltx_font_italic">There exist a one-layer log-precision RNN-SSM and WFA-SSM that express the word problem for <math id="S5.Thmtheorem3.p1.1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S5.Thmtheorem3.p1.1.1.m1.1a"><msub id="S5.Thmtheorem3.p1.1.1.m1.1.1" xref="S5.Thmtheorem3.p1.1.1.m1.1.1.cmml"><mi id="S5.Thmtheorem3.p1.1.1.m1.1.1.2" xref="S5.Thmtheorem3.p1.1.1.m1.1.1.2.cmml">S</mi><mn id="S5.Thmtheorem3.p1.1.1.m1.1.1.3" xref="S5.Thmtheorem3.p1.1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem3.p1.1.1.m1.1b"><apply id="S5.Thmtheorem3.p1.1.1.m1.1.1.cmml" xref="S5.Thmtheorem3.p1.1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.Thmtheorem3.p1.1.1.m1.1.1.1.cmml" xref="S5.Thmtheorem3.p1.1.1.m1.1.1">subscript</csymbol><ci id="S5.Thmtheorem3.p1.1.1.m1.1.1.2.cmml" xref="S5.Thmtheorem3.p1.1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S5.Thmtheorem3.p1.1.1.m1.1.1.3.cmml" xref="S5.Thmtheorem3.p1.1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem3.p1.1.1.m1.1c">S_{5}</annotation></semantics></math> (with a beginning-of-string symbol), and these these SSMs cannot be simulated in <math id="S5.Thmtheorem3.p1.2.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S5.Thmtheorem3.p1.2.2.m2.1a"><msup id="S5.Thmtheorem3.p1.2.2.m2.1.1" xref="S5.Thmtheorem3.p1.2.2.m2.1.1.cmml"><mi id="S5.Thmtheorem3.p1.2.2.m2.1.1.2" xref="S5.Thmtheorem3.p1.2.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S5.Thmtheorem3.p1.2.2.m2.1.1.3" xref="S5.Thmtheorem3.p1.2.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S5.Thmtheorem3.p1.2.2.m2.1b"><apply id="S5.Thmtheorem3.p1.2.2.m2.1.1.cmml" xref="S5.Thmtheorem3.p1.2.2.m2.1.1"><csymbol cd="ambiguous" id="S5.Thmtheorem3.p1.2.2.m2.1.1.1.cmml" xref="S5.Thmtheorem3.p1.2.2.m2.1.1">superscript</csymbol><ci id="S5.Thmtheorem3.p1.2.2.m2.1.1.2.cmml" xref="S5.Thmtheorem3.p1.2.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S5.Thmtheorem3.p1.2.2.m2.1.1.3.cmml" xref="S5.Thmtheorem3.p1.2.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.Thmtheorem3.p1.2.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="S5.SS3.p2" class="ltx_para">
<p id="S5.SS3.p2.1" class="ltx_p">But would these variants of SSMs be feasible to use in practice? Besides expressive power, there are two competing practical concerns that might make these extensions problematic: parallelism and the impact on learning dynamics.</p>
</div>
<section id="S5.SS3.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Parallelism.</h4>

<div id="S5.SS3.SSS0.Px1.p1" class="ltx_para">
<p id="S5.SS3.SSS0.Px1.p1.2" class="ltx_p">To be used effectively in an LLM, a model architecture must be parallelizable on practical hardware. Architectures in <math id="S5.SS3.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S5.SS3.SSS0.Px1.p1.1.m1.1a"><msup id="S5.SS3.SSS0.Px1.p1.1.m1.1.1" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1.cmml"><mi id="S5.SS3.SSS0.Px1.p1.1.m1.1.1.2" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1.2.cmml">𝖳𝖢</mi><mn id="S5.SS3.SSS0.Px1.p1.1.m1.1.1.3" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS0.Px1.p1.1.m1.1b"><apply id="S5.SS3.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S5.SS3.SSS0.Px1.p1.1.m1.1.1.1.cmml" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1">superscript</csymbol><ci id="S5.SS3.SSS0.Px1.p1.1.m1.1.1.2.cmml" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1.2">𝖳𝖢</ci><cn type="integer" id="S5.SS3.SSS0.Px1.p1.1.m1.1.1.3.cmml" xref="S5.SS3.SSS0.Px1.p1.1.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS0.Px1.p1.1.m1.1c">\mathsf{TC}^{0}</annotation></semantics></math> are parallelizable by design <cite class="ltx_cite ltx_citemacro_citep">(Merrill &amp; Sabharwal, <a href="#bib.bib25" title="" class="ltx_ref">2023a</a>)</cite>, but architectures in <math id="S5.SS3.SSS0.Px1.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S5.SS3.SSS0.Px1.p1.2.m2.1a"><msup id="S5.SS3.SSS0.Px1.p1.2.m2.1.1" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1.cmml"><mi id="S5.SS3.SSS0.Px1.p1.2.m2.1.1.2" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S5.SS3.SSS0.Px1.p1.2.m2.1.1.3" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S5.SS3.SSS0.Px1.p1.2.m2.1b"><apply id="S5.SS3.SSS0.Px1.p1.2.m2.1.1.cmml" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S5.SS3.SSS0.Px1.p1.2.m2.1.1.1.cmml" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1">superscript</csymbol><ci id="S5.SS3.SSS0.Px1.p1.2.m2.1.1.2.cmml" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S5.SS3.SSS0.Px1.p1.2.m2.1.1.3.cmml" xref="S5.SS3.SSS0.Px1.p1.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.SSS0.Px1.p1.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math> may still be parallelizable to log depth even if they cannot be parallelized to constant depth. For the WFA-SSM, the bottleneck would be computing iterated matrix product with a log-depth computation graph. Similarly, for the RNN-SSM, the bottleneck would be computing the state updates with a log-depth binary tree rather than left to right. If this could be accomplished on practical hardware, these architectures could be parallelizable enough to scale on modern hardware.</p>
</div>
</section>
<section id="S5.SS3.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Learning Dynamics.</h4>

<div id="S5.SS3.SSS0.Px2.p1" class="ltx_para">
<p id="S5.SS3.SSS0.Px2.p1.1" class="ltx_p">Another potential concern for these SSM variants compared to the original SSM is whether the learning dynamics are as good. In particular, for the WFA-SSM, an iterated product of matrices may lead to vanishing gradient issues. However, this is already potentially an issue for the S6 architecture, where the selective gating involves computing an iterated product of scalars.</p>
</div>
</section>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Can SSMs Learn Permutations in Practice?</h2>

<figure id="S6.F3" class="ltx_figure"><img src="/html/2404.08819/assets/x1.png" id="S6.F3.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="162" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure"><span id="S6.F3.4.2.1" class="ltx_text" style="font-size:90%;">Figure 3</span>: </span><span id="S6.F3.2.1" class="ltx_text" style="font-size:90%;">Minimum number of layers required to attain <math id="S6.F3.2.1.m1.1" class="ltx_Math" alttext="&gt;90\%" display="inline"><semantics id="S6.F3.2.1.m1.1b"><mrow id="S6.F3.2.1.m1.1.1" xref="S6.F3.2.1.m1.1.1.cmml"><mi id="S6.F3.2.1.m1.1.1.2" xref="S6.F3.2.1.m1.1.1.2.cmml"></mi><mo id="S6.F3.2.1.m1.1.1.1" xref="S6.F3.2.1.m1.1.1.1.cmml">&gt;</mo><mrow id="S6.F3.2.1.m1.1.1.3" xref="S6.F3.2.1.m1.1.1.3.cmml"><mn id="S6.F3.2.1.m1.1.1.3.2" xref="S6.F3.2.1.m1.1.1.3.2.cmml">90</mn><mo id="S6.F3.2.1.m1.1.1.3.1" xref="S6.F3.2.1.m1.1.1.3.1.cmml">%</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S6.F3.2.1.m1.1c"><apply id="S6.F3.2.1.m1.1.1.cmml" xref="S6.F3.2.1.m1.1.1"><gt id="S6.F3.2.1.m1.1.1.1.cmml" xref="S6.F3.2.1.m1.1.1.1"></gt><csymbol cd="latexml" id="S6.F3.2.1.m1.1.1.2.cmml" xref="S6.F3.2.1.m1.1.1.2">absent</csymbol><apply id="S6.F3.2.1.m1.1.1.3.cmml" xref="S6.F3.2.1.m1.1.1.3"><csymbol cd="latexml" id="S6.F3.2.1.m1.1.1.3.1.cmml" xref="S6.F3.2.1.m1.1.1.3.1">percent</csymbol><cn type="integer" id="S6.F3.2.1.m1.1.1.3.2.cmml" xref="S6.F3.2.1.m1.1.1.3.2">90</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.F3.2.1.m1.1d">&gt;90\%</annotation></semantics></math> validation accuracy on group multiplication problems by sequence length and group. RNN and IDS4 models of constant depth can solve arbitrarily long sequences, while transformer, S4, and Mamba models require depths monotonically increasing in sequence length.</span></figcaption>
</figure>
<div id="S6.p1" class="ltx_para">
<p id="S6.p1.3" class="ltx_p">Having established theoretical limitations of SSMs for state tracking, we now empirically test
how well SSMs can learn such tasks,
focusing on the word problem for <math id="S6.p1.1.m1.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.p1.1.m1.1a"><msub id="S6.p1.1.m1.1.1" xref="S6.p1.1.m1.1.1.cmml"><mi id="S6.p1.1.m1.1.1.2" xref="S6.p1.1.m1.1.1.2.cmml">A</mi><mn id="S6.p1.1.m1.1.1.3" xref="S6.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.p1.1.m1.1b"><apply id="S6.p1.1.m1.1.1.cmml" xref="S6.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S6.p1.1.m1.1.1.1.cmml" xref="S6.p1.1.m1.1.1">subscript</csymbol><ci id="S6.p1.1.m1.1.1.2.cmml" xref="S6.p1.1.m1.1.1.2">𝐴</ci><cn type="integer" id="S6.p1.1.m1.1.1.3.cmml" xref="S6.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.p1.1.m1.1c">A_{5}</annotation></semantics></math>. Since this problem is <math id="S6.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S6.p1.2.m2.1a"><msup id="S6.p1.2.m2.1.1" xref="S6.p1.2.m2.1.1.cmml"><mi id="S6.p1.2.m2.1.1.2" xref="S6.p1.2.m2.1.1.2.cmml">𝖭𝖢</mi><mn id="S6.p1.2.m2.1.1.3" xref="S6.p1.2.m2.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S6.p1.2.m2.1b"><apply id="S6.p1.2.m2.1.1.cmml" xref="S6.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S6.p1.2.m2.1.1.1.cmml" xref="S6.p1.2.m2.1.1">superscript</csymbol><ci id="S6.p1.2.m2.1.1.2.cmml" xref="S6.p1.2.m2.1.1.2">𝖭𝖢</ci><cn type="integer" id="S6.p1.2.m2.1.1.3.cmml" xref="S6.p1.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.p1.2.m2.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete and both transformers and SSMs can only express functions in <math id="S6.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S6.p1.3.m3.1a"><msup id="S6.p1.3.m3.1.1" xref="S6.p1.3.m3.1.1.cmml"><mi id="S6.p1.3.m3.1.1.2" xref="S6.p1.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S6.p1.3.m3.1.1.3" xref="S6.p1.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S6.p1.3.m3.1b"><apply id="S6.p1.3.m3.1.1.cmml" xref="S6.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S6.p1.3.m3.1.1.1.cmml" xref="S6.p1.3.m3.1.1">superscript</csymbol><ci id="S6.p1.3.m3.1.1.2.cmml" xref="S6.p1.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S6.p1.3.m3.1.1.3.cmml" xref="S6.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.p1.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math>, to solve instances of this problem these models should require a dynamic depth that grows with the input length.</p>
</div>
<section id="S6.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Setup.</h4>

<div id="S6.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px1.p1.9" class="ltx_p">We model word problems (see <a href="#S3.SS1" title="3.1 State Tracking as a Monoid Word Problem ‣ 3 State Tracking ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">3.1</span></a>) as a token-tagging task. Models are given as input a sequence of elements <math id="S6.SS0.SSS0.Px1.p1.1.m1.1" class="ltx_Math" alttext="g_{0}g_{1}g_{2}\cdots g_{n}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.1.m1.1a"><mrow id="S6.SS0.SSS0.Px1.p1.1.m1.1.1" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.cmml"><msub id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.cmml"><mi id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.2" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.2.cmml">g</mi><mn id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.3" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.3.cmml">0</mn></msub><mo lspace="0em" rspace="0em" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml">​</mo><msub id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.cmml"><mi id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.2" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.cmml">g</mi><mn id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.3" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.cmml">1</mn></msub><mo lspace="0em" rspace="0em" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1a" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml">​</mo><msub id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.cmml"><mi id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.2" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.2.cmml">g</mi><mn id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.3" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.3.cmml">2</mn></msub><mo lspace="0em" rspace="0em" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1b" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml">​</mo><mi mathvariant="normal" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.5" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.5.cmml">⋯</mi><mo lspace="0em" rspace="0em" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1c" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml">​</mo><msub id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.cmml"><mi id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.2" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.2.cmml">g</mi><mi id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.3" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.3.cmml">n</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.1.m1.1b"><apply id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1"><times id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.1"></times><apply id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.2.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.2">𝑔</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.3.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.2.3">0</cn></apply><apply id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.2.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.2">𝑔</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.3.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.3.3">1</cn></apply><apply id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.2.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.2">𝑔</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.3.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.4.3">2</cn></apply><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.5.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.5">⋯</ci><apply id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.1.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.2.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.2">𝑔</ci><ci id="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.3.cmml" xref="S6.SS0.SSS0.Px1.p1.1.m1.1.1.6.3">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.1.m1.1c">g_{0}g_{1}g_{2}\cdots g_{n}</annotation></semantics></math> drawn from one of <math id="S6.SS0.SSS0.Px1.p1.2.m2.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.2.m2.1a"><msub id="S6.SS0.SSS0.Px1.p1.2.m2.1.1" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1.cmml"><mi id="S6.SS0.SSS0.Px1.p1.2.m2.1.1.2" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1.2.cmml">A</mi><mn id="S6.SS0.SSS0.Px1.p1.2.m2.1.1.3" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.2.m2.1b"><apply id="S6.SS0.SSS0.Px1.p1.2.m2.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.2.m2.1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.2.m2.1.1.2.cmml" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1.2">𝐴</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.2.m2.1.1.3.cmml" xref="S6.SS0.SSS0.Px1.p1.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.2.m2.1c">A_{5}</annotation></semantics></math>, <math id="S6.SS0.SSS0.Px1.p1.3.m3.1" class="ltx_Math" alttext="A_{4}\times\mathbb{Z}_{5}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.3.m3.1a"><mrow id="S6.SS0.SSS0.Px1.p1.3.m3.1.1" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.cmml"><msub id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.cmml"><mi id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.2" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.2.cmml">A</mi><mn id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.3" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.3.cmml">4</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.1" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.1.cmml">×</mo><msub id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.cmml"><mi id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.2" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.2.cmml">ℤ</mi><mn id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.3" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.3.cmml">5</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.3.m3.1b"><apply id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1"><times id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.1"></times><apply id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.1.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.2.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.2">𝐴</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.3.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.2.3">4</cn></apply><apply id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.1.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.2.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.2">ℤ</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.3.cmml" xref="S6.SS0.SSS0.Px1.p1.3.m3.1.1.3.3">5</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.3.m3.1c">A_{4}\times\mathbb{Z}_{5}</annotation></semantics></math>, or <math id="S6.SS0.SSS0.Px1.p1.4.m4.1" class="ltx_Math" alttext="\mathbb{Z}_{60}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.4.m4.1a"><msub id="S6.SS0.SSS0.Px1.p1.4.m4.1.1" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1.cmml"><mi id="S6.SS0.SSS0.Px1.p1.4.m4.1.1.2" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1.2.cmml">ℤ</mi><mn id="S6.SS0.SSS0.Px1.p1.4.m4.1.1.3" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1.3.cmml">60</mn></msub><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.4.m4.1b"><apply id="S6.SS0.SSS0.Px1.p1.4.m4.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.4.m4.1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.4.m4.1.1.2.cmml" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1.2">ℤ</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.4.m4.1.1.3.cmml" xref="S6.SS0.SSS0.Px1.p1.4.m4.1.1.3">60</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.4.m4.1c">\mathbb{Z}_{60}</annotation></semantics></math>, and for each step <math id="S6.SS0.SSS0.Px1.p1.5.m5.1" class="ltx_Math" alttext="t_{i}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.5.m5.1a"><msub id="S6.SS0.SSS0.Px1.p1.5.m5.1.1" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1.cmml"><mi id="S6.SS0.SSS0.Px1.p1.5.m5.1.1.2" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1.2.cmml">t</mi><mi id="S6.SS0.SSS0.Px1.p1.5.m5.1.1.3" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.5.m5.1b"><apply id="S6.SS0.SSS0.Px1.p1.5.m5.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.5.m5.1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.5.m5.1.1.2.cmml" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1.2">𝑡</ci><ci id="S6.SS0.SSS0.Px1.p1.5.m5.1.1.3.cmml" xref="S6.SS0.SSS0.Px1.p1.5.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.5.m5.1c">t_{i}</annotation></semantics></math> must predict the result of multiplying the first <math id="S6.SS0.SSS0.Px1.p1.6.m6.1" class="ltx_Math" alttext="i" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.6.m6.1a"><mi id="S6.SS0.SSS0.Px1.p1.6.m6.1.1" xref="S6.SS0.SSS0.Px1.p1.6.m6.1.1.cmml">i</mi><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.6.m6.1b"><ci id="S6.SS0.SSS0.Px1.p1.6.m6.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.6.m6.1.1">𝑖</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.6.m6.1c">i</annotation></semantics></math> elements of the sequence together. Modeling the problem as a tagging task rather than as a sequence classification task provides the models with more supervision during training, making it as easy as possible to learn the correct function. We tokenize inputs such that each element gets a unique token. We train models on sequences of length <math id="S6.SS0.SSS0.Px1.p1.7.m7.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.7.m7.1a"><mi id="S6.SS0.SSS0.Px1.p1.7.m7.1.1" xref="S6.SS0.SSS0.Px1.p1.7.m7.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.7.m7.1b"><ci id="S6.SS0.SSS0.Px1.p1.7.m7.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.7.m7.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.7.m7.1c">n</annotation></semantics></math> for successively larger values of <math id="S6.SS0.SSS0.Px1.p1.8.m8.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.8.m8.1a"><mi id="S6.SS0.SSS0.Px1.p1.8.m8.1.1" xref="S6.SS0.SSS0.Px1.p1.8.m8.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.8.m8.1b"><ci id="S6.SS0.SSS0.Px1.p1.8.m8.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.8.m8.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.8.m8.1c">n</annotation></semantics></math> and report full-sequence accuracy on a validation set.<span id="footnote8" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">8</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">8</sup><span class="ltx_tag ltx_tag_note">8</span>We always include the full set of <math id="footnote8.m1.1" class="ltx_Math" alttext="3600" display="inline"><semantics id="footnote8.m1.1b"><mn id="footnote8.m1.1.1" xref="footnote8.m1.1.1.cmml">3600</mn><annotation-xml encoding="MathML-Content" id="footnote8.m1.1c"><cn type="integer" id="footnote8.m1.1.1.cmml" xref="footnote8.m1.1.1">3600</cn></annotation-xml><annotation encoding="application/x-tex" id="footnote8.m1.1d">3600</annotation></semantics></math> pairwise sequences of length <math id="footnote8.m2.1" class="ltx_Math" alttext="2" display="inline"><semantics id="footnote8.m2.1b"><mn id="footnote8.m2.1.1" xref="footnote8.m2.1.1.cmml">2</mn><annotation-xml encoding="MathML-Content" id="footnote8.m2.1c"><cn type="integer" id="footnote8.m2.1.1.cmml" xref="footnote8.m2.1.1">2</cn></annotation-xml><annotation encoding="application/x-tex" id="footnote8.m2.1d">2</annotation></semantics></math> in the training data, along with the training split of length-<math id="footnote8.m3.1" class="ltx_Math" alttext="n" display="inline"><semantics id="footnote8.m3.1b"><mi id="footnote8.m3.1.1" xref="footnote8.m3.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="footnote8.m3.1c"><ci id="footnote8.m3.1.1.cmml" xref="footnote8.m3.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="footnote8.m3.1d">n</annotation></semantics></math> sequences.</span></span></span> To validate the predictions of SSMs and transformers being depth-bounded for expressing the word problem for <math id="S6.SS0.SSS0.Px1.p1.9.m9.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.SS0.SSS0.Px1.p1.9.m9.1a"><msub id="S6.SS0.SSS0.Px1.p1.9.m9.1.1" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1.cmml"><mi id="S6.SS0.SSS0.Px1.p1.9.m9.1.1.2" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1.2.cmml">A</mi><mn id="S6.SS0.SSS0.Px1.p1.9.m9.1.1.3" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.SS0.SSS0.Px1.p1.9.m9.1b"><apply id="S6.SS0.SSS0.Px1.p1.9.m9.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1"><csymbol cd="ambiguous" id="S6.SS0.SSS0.Px1.p1.9.m9.1.1.1.cmml" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1">subscript</csymbol><ci id="S6.SS0.SSS0.Px1.p1.9.m9.1.1.2.cmml" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1.2">𝐴</ci><cn type="integer" id="S6.SS0.SSS0.Px1.p1.9.m9.1.1.3.cmml" xref="S6.SS0.SSS0.Px1.p1.9.m9.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS0.SSS0.Px1.p1.9.m9.1c">A_{5}</annotation></semantics></math>, we sweep over the number of layers each S4, Mamba and transformer model has, and compare these results to a single-layer simple RNN and our proposed single-layer Input-Dependent S4 (IDS4) architecture.</p>
</div>
</section>
<section id="S6.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Results.</h4>

<div id="S6.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="S6.SS0.SSS0.Px2.p1.1" class="ltx_p"><a href="#S6.F3" title="In 6 Can SSMs Learn Permutations in Practice? ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">3</span></a> shows that single-layer RNN and IDS4 models learn the word problem for arbitrarily long sequences from all three groups. In contrast, transformer, S4, and Mamba models require depth monotonically increasing in sequence length to attain good accuracy on a validation set on non-commutative groups. We draw three main conclusions from this:</p>
</div>
<div id="S6.SS0.SSS0.Px2.p2" class="ltx_para">
<ol id="S6.I1" class="ltx_enumerate">
<li id="S6.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S6.I1.i1.p1" class="ltx_para">
<p id="S6.I1.i1.p1.5" class="ltx_p">Mamba and S4 show the same qualitative limitations as transformers on the inherently sequential task <math id="S6.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.I1.i1.p1.1.m1.1a"><msub id="S6.I1.i1.p1.1.m1.1.1" xref="S6.I1.i1.p1.1.m1.1.1.cmml"><mi id="S6.I1.i1.p1.1.m1.1.1.2" xref="S6.I1.i1.p1.1.m1.1.1.2.cmml">A</mi><mn id="S6.I1.i1.p1.1.m1.1.1.3" xref="S6.I1.i1.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.I1.i1.p1.1.m1.1b"><apply id="S6.I1.i1.p1.1.m1.1.1.cmml" xref="S6.I1.i1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="S6.I1.i1.p1.1.m1.1.1.1.cmml" xref="S6.I1.i1.p1.1.m1.1.1">subscript</csymbol><ci id="S6.I1.i1.p1.1.m1.1.1.2.cmml" xref="S6.I1.i1.p1.1.m1.1.1.2">𝐴</ci><cn type="integer" id="S6.I1.i1.p1.1.m1.1.1.3.cmml" xref="S6.I1.i1.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i1.p1.1.m1.1c">A_{5}</annotation></semantics></math>: longer <math id="S6.I1.i1.p1.2.m2.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.I1.i1.p1.2.m2.1a"><msub id="S6.I1.i1.p1.2.m2.1.1" xref="S6.I1.i1.p1.2.m2.1.1.cmml"><mi id="S6.I1.i1.p1.2.m2.1.1.2" xref="S6.I1.i1.p1.2.m2.1.1.2.cmml">A</mi><mn id="S6.I1.i1.p1.2.m2.1.1.3" xref="S6.I1.i1.p1.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.I1.i1.p1.2.m2.1b"><apply id="S6.I1.i1.p1.2.m2.1.1.cmml" xref="S6.I1.i1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S6.I1.i1.p1.2.m2.1.1.1.cmml" xref="S6.I1.i1.p1.2.m2.1.1">subscript</csymbol><ci id="S6.I1.i1.p1.2.m2.1.1.2.cmml" xref="S6.I1.i1.p1.2.m2.1.1.2">𝐴</ci><cn type="integer" id="S6.I1.i1.p1.2.m2.1.1.3.cmml" xref="S6.I1.i1.p1.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i1.p1.2.m2.1c">A_{5}</annotation></semantics></math> sequences require deeper models. This is consistent with both transformers, S4, and Mamba being in <math id="S6.I1.i1.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S6.I1.i1.p1.3.m3.1a"><msup id="S6.I1.i1.p1.3.m3.1.1" xref="S6.I1.i1.p1.3.m3.1.1.cmml"><mi id="S6.I1.i1.p1.3.m3.1.1.2" xref="S6.I1.i1.p1.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="S6.I1.i1.p1.3.m3.1.1.3" xref="S6.I1.i1.p1.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S6.I1.i1.p1.3.m3.1b"><apply id="S6.I1.i1.p1.3.m3.1.1.cmml" xref="S6.I1.i1.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S6.I1.i1.p1.3.m3.1.1.1.cmml" xref="S6.I1.i1.p1.3.m3.1.1">superscript</csymbol><ci id="S6.I1.i1.p1.3.m3.1.1.2.cmml" xref="S6.I1.i1.p1.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="S6.I1.i1.p1.3.m3.1.1.3.cmml" xref="S6.I1.i1.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i1.p1.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math>. On the other hand, RNNs (which resemble automata; <cite class="ltx_cite ltx_citemacro_citep">Merrill, <a href="#bib.bib24" title="" class="ltx_ref">2019</a></cite>) and IDS4 can solve <math id="S6.I1.i1.p1.4.m4.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S6.I1.i1.p1.4.m4.1a"><msup id="S6.I1.i1.p1.4.m4.1.1" xref="S6.I1.i1.p1.4.m4.1.1.cmml"><mi id="S6.I1.i1.p1.4.m4.1.1.2" xref="S6.I1.i1.p1.4.m4.1.1.2.cmml">𝖭𝖢</mi><mn id="S6.I1.i1.p1.4.m4.1.1.3" xref="S6.I1.i1.p1.4.m4.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S6.I1.i1.p1.4.m4.1b"><apply id="S6.I1.i1.p1.4.m4.1.1.cmml" xref="S6.I1.i1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S6.I1.i1.p1.4.m4.1.1.1.cmml" xref="S6.I1.i1.p1.4.m4.1.1">superscript</csymbol><ci id="S6.I1.i1.p1.4.m4.1.1.2.cmml" xref="S6.I1.i1.p1.4.m4.1.1.2">𝖭𝖢</ci><cn type="integer" id="S6.I1.i1.p1.4.m4.1.1.3.cmml" xref="S6.I1.i1.p1.4.m4.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i1.p1.4.m4.1c">\mathsf{NC}^{1}</annotation></semantics></math>-complete problems <cite class="ltx_cite ltx_citemacro_citep">(McKenzie et al., <a href="#bib.bib22" title="" class="ltx_ref">1991</a>)</cite> including the <math id="S6.I1.i1.p1.5.m5.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.I1.i1.p1.5.m5.1a"><msub id="S6.I1.i1.p1.5.m5.1.1" xref="S6.I1.i1.p1.5.m5.1.1.cmml"><mi id="S6.I1.i1.p1.5.m5.1.1.2" xref="S6.I1.i1.p1.5.m5.1.1.2.cmml">A</mi><mn id="S6.I1.i1.p1.5.m5.1.1.3" xref="S6.I1.i1.p1.5.m5.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.I1.i1.p1.5.m5.1b"><apply id="S6.I1.i1.p1.5.m5.1.1.cmml" xref="S6.I1.i1.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S6.I1.i1.p1.5.m5.1.1.1.cmml" xref="S6.I1.i1.p1.5.m5.1.1">subscript</csymbol><ci id="S6.I1.i1.p1.5.m5.1.1.2.cmml" xref="S6.I1.i1.p1.5.m5.1.1.2">𝐴</ci><cn type="integer" id="S6.I1.i1.p1.5.m5.1.1.3.cmml" xref="S6.I1.i1.p1.5.m5.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i1.p1.5.m5.1c">A_{5}</annotation></semantics></math> word problem.</p>
</div>
</li>
<li id="S6.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S6.I1.i2.p1" class="ltx_para">
<p id="S6.I1.i2.p1.6" class="ltx_p">S4, Mamba, and transformers require greater depth even <math id="S6.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="A_{4}\times\mathbb{Z}_{5}" display="inline"><semantics id="S6.I1.i2.p1.1.m1.1a"><mrow id="S6.I1.i2.p1.1.m1.1.1" xref="S6.I1.i2.p1.1.m1.1.1.cmml"><msub id="S6.I1.i2.p1.1.m1.1.1.2" xref="S6.I1.i2.p1.1.m1.1.1.2.cmml"><mi id="S6.I1.i2.p1.1.m1.1.1.2.2" xref="S6.I1.i2.p1.1.m1.1.1.2.2.cmml">A</mi><mn id="S6.I1.i2.p1.1.m1.1.1.2.3" xref="S6.I1.i2.p1.1.m1.1.1.2.3.cmml">4</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S6.I1.i2.p1.1.m1.1.1.1" xref="S6.I1.i2.p1.1.m1.1.1.1.cmml">×</mo><msub id="S6.I1.i2.p1.1.m1.1.1.3" xref="S6.I1.i2.p1.1.m1.1.1.3.cmml"><mi id="S6.I1.i2.p1.1.m1.1.1.3.2" xref="S6.I1.i2.p1.1.m1.1.1.3.2.cmml">ℤ</mi><mn id="S6.I1.i2.p1.1.m1.1.1.3.3" xref="S6.I1.i2.p1.1.m1.1.1.3.3.cmml">5</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S6.I1.i2.p1.1.m1.1b"><apply id="S6.I1.i2.p1.1.m1.1.1.cmml" xref="S6.I1.i2.p1.1.m1.1.1"><times id="S6.I1.i2.p1.1.m1.1.1.1.cmml" xref="S6.I1.i2.p1.1.m1.1.1.1"></times><apply id="S6.I1.i2.p1.1.m1.1.1.2.cmml" xref="S6.I1.i2.p1.1.m1.1.1.2"><csymbol cd="ambiguous" id="S6.I1.i2.p1.1.m1.1.1.2.1.cmml" xref="S6.I1.i2.p1.1.m1.1.1.2">subscript</csymbol><ci id="S6.I1.i2.p1.1.m1.1.1.2.2.cmml" xref="S6.I1.i2.p1.1.m1.1.1.2.2">𝐴</ci><cn type="integer" id="S6.I1.i2.p1.1.m1.1.1.2.3.cmml" xref="S6.I1.i2.p1.1.m1.1.1.2.3">4</cn></apply><apply id="S6.I1.i2.p1.1.m1.1.1.3.cmml" xref="S6.I1.i2.p1.1.m1.1.1.3"><csymbol cd="ambiguous" id="S6.I1.i2.p1.1.m1.1.1.3.1.cmml" xref="S6.I1.i2.p1.1.m1.1.1.3">subscript</csymbol><ci id="S6.I1.i2.p1.1.m1.1.1.3.2.cmml" xref="S6.I1.i2.p1.1.m1.1.1.3.2">ℤ</ci><cn type="integer" id="S6.I1.i2.p1.1.m1.1.1.3.3.cmml" xref="S6.I1.i2.p1.1.m1.1.1.3.3">5</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i2.p1.1.m1.1c">A_{4}\times\mathbb{Z}_{5}</annotation></semantics></math>, which can be theoretically expressed by <math id="S6.I1.i2.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S6.I1.i2.p1.2.m2.1a"><msup id="S6.I1.i2.p1.2.m2.1.1" xref="S6.I1.i2.p1.2.m2.1.1.cmml"><mi id="S6.I1.i2.p1.2.m2.1.1.2" xref="S6.I1.i2.p1.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S6.I1.i2.p1.2.m2.1.1.3" xref="S6.I1.i2.p1.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S6.I1.i2.p1.2.m2.1b"><apply id="S6.I1.i2.p1.2.m2.1.1.cmml" xref="S6.I1.i2.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S6.I1.i2.p1.2.m2.1.1.1.cmml" xref="S6.I1.i2.p1.2.m2.1.1">superscript</csymbol><ci id="S6.I1.i2.p1.2.m2.1.1.2.cmml" xref="S6.I1.i2.p1.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S6.I1.i2.p1.2.m2.1.1.3.cmml" xref="S6.I1.i2.p1.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i2.p1.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuits. Although transformer and Mamba models of a given depth perform as good or better on <math id="S6.I1.i2.p1.3.m3.1" class="ltx_Math" alttext="A_{4}\times\mathbb{Z}_{5}" display="inline"><semantics id="S6.I1.i2.p1.3.m3.1a"><mrow id="S6.I1.i2.p1.3.m3.1.1" xref="S6.I1.i2.p1.3.m3.1.1.cmml"><msub id="S6.I1.i2.p1.3.m3.1.1.2" xref="S6.I1.i2.p1.3.m3.1.1.2.cmml"><mi id="S6.I1.i2.p1.3.m3.1.1.2.2" xref="S6.I1.i2.p1.3.m3.1.1.2.2.cmml">A</mi><mn id="S6.I1.i2.p1.3.m3.1.1.2.3" xref="S6.I1.i2.p1.3.m3.1.1.2.3.cmml">4</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S6.I1.i2.p1.3.m3.1.1.1" xref="S6.I1.i2.p1.3.m3.1.1.1.cmml">×</mo><msub id="S6.I1.i2.p1.3.m3.1.1.3" xref="S6.I1.i2.p1.3.m3.1.1.3.cmml"><mi id="S6.I1.i2.p1.3.m3.1.1.3.2" xref="S6.I1.i2.p1.3.m3.1.1.3.2.cmml">ℤ</mi><mn id="S6.I1.i2.p1.3.m3.1.1.3.3" xref="S6.I1.i2.p1.3.m3.1.1.3.3.cmml">5</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S6.I1.i2.p1.3.m3.1b"><apply id="S6.I1.i2.p1.3.m3.1.1.cmml" xref="S6.I1.i2.p1.3.m3.1.1"><times id="S6.I1.i2.p1.3.m3.1.1.1.cmml" xref="S6.I1.i2.p1.3.m3.1.1.1"></times><apply id="S6.I1.i2.p1.3.m3.1.1.2.cmml" xref="S6.I1.i2.p1.3.m3.1.1.2"><csymbol cd="ambiguous" id="S6.I1.i2.p1.3.m3.1.1.2.1.cmml" xref="S6.I1.i2.p1.3.m3.1.1.2">subscript</csymbol><ci id="S6.I1.i2.p1.3.m3.1.1.2.2.cmml" xref="S6.I1.i2.p1.3.m3.1.1.2.2">𝐴</ci><cn type="integer" id="S6.I1.i2.p1.3.m3.1.1.2.3.cmml" xref="S6.I1.i2.p1.3.m3.1.1.2.3">4</cn></apply><apply id="S6.I1.i2.p1.3.m3.1.1.3.cmml" xref="S6.I1.i2.p1.3.m3.1.1.3"><csymbol cd="ambiguous" id="S6.I1.i2.p1.3.m3.1.1.3.1.cmml" xref="S6.I1.i2.p1.3.m3.1.1.3">subscript</csymbol><ci id="S6.I1.i2.p1.3.m3.1.1.3.2.cmml" xref="S6.I1.i2.p1.3.m3.1.1.3.2">ℤ</ci><cn type="integer" id="S6.I1.i2.p1.3.m3.1.1.3.3.cmml" xref="S6.I1.i2.p1.3.m3.1.1.3.3">5</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i2.p1.3.m3.1c">A_{4}\times\mathbb{Z}_{5}</annotation></semantics></math> as they on <math id="S6.I1.i2.p1.4.m4.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.I1.i2.p1.4.m4.1a"><msub id="S6.I1.i2.p1.4.m4.1.1" xref="S6.I1.i2.p1.4.m4.1.1.cmml"><mi id="S6.I1.i2.p1.4.m4.1.1.2" xref="S6.I1.i2.p1.4.m4.1.1.2.cmml">A</mi><mn id="S6.I1.i2.p1.4.m4.1.1.3" xref="S6.I1.i2.p1.4.m4.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.I1.i2.p1.4.m4.1b"><apply id="S6.I1.i2.p1.4.m4.1.1.cmml" xref="S6.I1.i2.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S6.I1.i2.p1.4.m4.1.1.1.cmml" xref="S6.I1.i2.p1.4.m4.1.1">subscript</csymbol><ci id="S6.I1.i2.p1.4.m4.1.1.2.cmml" xref="S6.I1.i2.p1.4.m4.1.1.2">𝐴</ci><cn type="integer" id="S6.I1.i2.p1.4.m4.1.1.3.cmml" xref="S6.I1.i2.p1.4.m4.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i2.p1.4.m4.1c">A_{5}</annotation></semantics></math>, they still require increasingly many layers to handle proportionally longer sequences. There are two possible interpretations of this. First, it could be that while these word problems are expressible in <math id="S6.I1.i2.p1.5.m5.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S6.I1.i2.p1.5.m5.1a"><msup id="S6.I1.i2.p1.5.m5.1.1" xref="S6.I1.i2.p1.5.m5.1.1.cmml"><mi id="S6.I1.i2.p1.5.m5.1.1.2" xref="S6.I1.i2.p1.5.m5.1.1.2.cmml">𝖳𝖢</mi><mn id="S6.I1.i2.p1.5.m5.1.1.3" xref="S6.I1.i2.p1.5.m5.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S6.I1.i2.p1.5.m5.1b"><apply id="S6.I1.i2.p1.5.m5.1.1.cmml" xref="S6.I1.i2.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S6.I1.i2.p1.5.m5.1.1.1.cmml" xref="S6.I1.i2.p1.5.m5.1.1">superscript</csymbol><ci id="S6.I1.i2.p1.5.m5.1.1.2.cmml" xref="S6.I1.i2.p1.5.m5.1.1.2">𝖳𝖢</ci><cn type="integer" id="S6.I1.i2.p1.5.m5.1.1.3.cmml" xref="S6.I1.i2.p1.5.m5.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i2.p1.5.m5.1c">\mathsf{TC}^{0}</annotation></semantics></math>, they cannot be expressed by S4, Mamba, or transformers (which can each likely recognize only a proper subset of <math id="S6.I1.i2.p1.6.m6.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S6.I1.i2.p1.6.m6.1a"><msup id="S6.I1.i2.p1.6.m6.1.1" xref="S6.I1.i2.p1.6.m6.1.1.cmml"><mi id="S6.I1.i2.p1.6.m6.1.1.2" xref="S6.I1.i2.p1.6.m6.1.1.2.cmml">𝖳𝖢</mi><mn id="S6.I1.i2.p1.6.m6.1.1.3" xref="S6.I1.i2.p1.6.m6.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S6.I1.i2.p1.6.m6.1b"><apply id="S6.I1.i2.p1.6.m6.1.1.cmml" xref="S6.I1.i2.p1.6.m6.1.1"><csymbol cd="ambiguous" id="S6.I1.i2.p1.6.m6.1.1.1.cmml" xref="S6.I1.i2.p1.6.m6.1.1">superscript</csymbol><ci id="S6.I1.i2.p1.6.m6.1.1.2.cmml" xref="S6.I1.i2.p1.6.m6.1.1.2">𝖳𝖢</ci><cn type="integer" id="S6.I1.i2.p1.6.m6.1.1.3.cmml" xref="S6.I1.i2.p1.6.m6.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i2.p1.6.m6.1c">\mathsf{TC}^{0}</annotation></semantics></math>). On the other hand, it is possible that these word problems <em id="S6.I1.i2.p1.6.1" class="ltx_emph ltx_font_italic">are</em> expressible by transformers, S4, and Mamba but that effectively learning a constant-depth solution is difficult.</p>
</div>
</li>
<li id="S6.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">3.</span> 
<div id="S6.I1.i3.p1" class="ltx_para">
<p id="S6.I1.i3.p1.3" class="ltx_p">Despite this limitation, S4 and Mamba appear <em id="S6.I1.i3.p1.3.1" class="ltx_emph ltx_font_italic">empirically better</em> at approximate state tracking on the non-commutative tasks than the transformer. For length-<math id="S6.I1.i3.p1.1.m1.1" class="ltx_Math" alttext="n" display="inline"><semantics id="S6.I1.i3.p1.1.m1.1a"><mi id="S6.I1.i3.p1.1.m1.1.1" xref="S6.I1.i3.p1.1.m1.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="S6.I1.i3.p1.1.m1.1b"><ci id="S6.I1.i3.p1.1.m1.1.1.cmml" xref="S6.I1.i3.p1.1.m1.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i3.p1.1.m1.1c">n</annotation></semantics></math> sequences from <math id="S6.I1.i3.p1.2.m2.1" class="ltx_Math" alttext="A_{4}\times\mathbb{Z}_{5}" display="inline"><semantics id="S6.I1.i3.p1.2.m2.1a"><mrow id="S6.I1.i3.p1.2.m2.1.1" xref="S6.I1.i3.p1.2.m2.1.1.cmml"><msub id="S6.I1.i3.p1.2.m2.1.1.2" xref="S6.I1.i3.p1.2.m2.1.1.2.cmml"><mi id="S6.I1.i3.p1.2.m2.1.1.2.2" xref="S6.I1.i3.p1.2.m2.1.1.2.2.cmml">A</mi><mn id="S6.I1.i3.p1.2.m2.1.1.2.3" xref="S6.I1.i3.p1.2.m2.1.1.2.3.cmml">4</mn></msub><mo lspace="0.222em" rspace="0.222em" id="S6.I1.i3.p1.2.m2.1.1.1" xref="S6.I1.i3.p1.2.m2.1.1.1.cmml">×</mo><msub id="S6.I1.i3.p1.2.m2.1.1.3" xref="S6.I1.i3.p1.2.m2.1.1.3.cmml"><mi id="S6.I1.i3.p1.2.m2.1.1.3.2" xref="S6.I1.i3.p1.2.m2.1.1.3.2.cmml">ℤ</mi><mn id="S6.I1.i3.p1.2.m2.1.1.3.3" xref="S6.I1.i3.p1.2.m2.1.1.3.3.cmml">5</mn></msub></mrow><annotation-xml encoding="MathML-Content" id="S6.I1.i3.p1.2.m2.1b"><apply id="S6.I1.i3.p1.2.m2.1.1.cmml" xref="S6.I1.i3.p1.2.m2.1.1"><times id="S6.I1.i3.p1.2.m2.1.1.1.cmml" xref="S6.I1.i3.p1.2.m2.1.1.1"></times><apply id="S6.I1.i3.p1.2.m2.1.1.2.cmml" xref="S6.I1.i3.p1.2.m2.1.1.2"><csymbol cd="ambiguous" id="S6.I1.i3.p1.2.m2.1.1.2.1.cmml" xref="S6.I1.i3.p1.2.m2.1.1.2">subscript</csymbol><ci id="S6.I1.i3.p1.2.m2.1.1.2.2.cmml" xref="S6.I1.i3.p1.2.m2.1.1.2.2">𝐴</ci><cn type="integer" id="S6.I1.i3.p1.2.m2.1.1.2.3.cmml" xref="S6.I1.i3.p1.2.m2.1.1.2.3">4</cn></apply><apply id="S6.I1.i3.p1.2.m2.1.1.3.cmml" xref="S6.I1.i3.p1.2.m2.1.1.3"><csymbol cd="ambiguous" id="S6.I1.i3.p1.2.m2.1.1.3.1.cmml" xref="S6.I1.i3.p1.2.m2.1.1.3">subscript</csymbol><ci id="S6.I1.i3.p1.2.m2.1.1.3.2.cmml" xref="S6.I1.i3.p1.2.m2.1.1.3.2">ℤ</ci><cn type="integer" id="S6.I1.i3.p1.2.m2.1.1.3.3.cmml" xref="S6.I1.i3.p1.2.m2.1.1.3.3">5</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i3.p1.2.m2.1c">A_{4}\times\mathbb{Z}_{5}</annotation></semantics></math> or <math id="S6.I1.i3.p1.3.m3.1" class="ltx_Math" alttext="A_{5}" display="inline"><semantics id="S6.I1.i3.p1.3.m3.1a"><msub id="S6.I1.i3.p1.3.m3.1.1" xref="S6.I1.i3.p1.3.m3.1.1.cmml"><mi id="S6.I1.i3.p1.3.m3.1.1.2" xref="S6.I1.i3.p1.3.m3.1.1.2.cmml">A</mi><mn id="S6.I1.i3.p1.3.m3.1.1.3" xref="S6.I1.i3.p1.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S6.I1.i3.p1.3.m3.1b"><apply id="S6.I1.i3.p1.3.m3.1.1.cmml" xref="S6.I1.i3.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S6.I1.i3.p1.3.m3.1.1.1.cmml" xref="S6.I1.i3.p1.3.m3.1.1">subscript</csymbol><ci id="S6.I1.i3.p1.3.m3.1.1.2.cmml" xref="S6.I1.i3.p1.3.m3.1.1.2">𝐴</ci><cn type="integer" id="S6.I1.i3.p1.3.m3.1.1.3.cmml" xref="S6.I1.i3.p1.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.I1.i3.p1.3.m3.1c">A_{5}</annotation></semantics></math>, the transformer requires at least as many (and frequently fewer) layers as S4 or Mamba to solve the task.</p>
</div>
</li>
</ol>
</div>
</section>
</section>
<section id="S7" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">7 </span>Conclusion</h2>

<div id="S7.p1" class="ltx_para">
<p id="S7.p1.6" class="ltx_p">We have shown that SSMs, like transformers, can only express computation in the complexity class <math id="S7.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="S7.p1.1.m1.1a"><mi id="S7.p1.1.m1.1.1" xref="S7.p1.1.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="S7.p1.1.m1.1b"><ci id="S7.p1.1.m1.1.1.cmml" xref="S7.p1.1.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.1.m1.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="S7.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="S7.p1.2.m2.1a"><msup id="S7.p1.2.m2.1.1" xref="S7.p1.2.m2.1.1.cmml"><mi id="S7.p1.2.m2.1.1.2" xref="S7.p1.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="S7.p1.2.m2.1.1.3" xref="S7.p1.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="S7.p1.2.m2.1b"><apply id="S7.p1.2.m2.1.1.cmml" xref="S7.p1.2.m2.1.1"><csymbol cd="ambiguous" id="S7.p1.2.m2.1.1.1.cmml" xref="S7.p1.2.m2.1.1">superscript</csymbol><ci id="S7.p1.2.m2.1.1.2.cmml" xref="S7.p1.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="S7.p1.2.m2.1.1.3.cmml" xref="S7.p1.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math>. This means they cannot solve inherently sequential problems like graph connectivity, boolean formula evaluation, and—of particular interest for state tracking—the permutation composition problem <math id="S7.p1.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S7.p1.3.m3.1a"><msub id="S7.p1.3.m3.1.1" xref="S7.p1.3.m3.1.1.cmml"><mi id="S7.p1.3.m3.1.1.2" xref="S7.p1.3.m3.1.1.2.cmml">S</mi><mn id="S7.p1.3.m3.1.1.3" xref="S7.p1.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S7.p1.3.m3.1b"><apply id="S7.p1.3.m3.1.1.cmml" xref="S7.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S7.p1.3.m3.1.1.1.cmml" xref="S7.p1.3.m3.1.1">subscript</csymbol><ci id="S7.p1.3.m3.1.1.2.cmml" xref="S7.p1.3.m3.1.1.2">𝑆</ci><cn type="integer" id="S7.p1.3.m3.1.1.3.cmml" xref="S7.p1.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.3.m3.1c">S_{5}</annotation></semantics></math>. <math id="S7.p1.4.m4.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S7.p1.4.m4.1a"><msub id="S7.p1.4.m4.1.1" xref="S7.p1.4.m4.1.1.cmml"><mi id="S7.p1.4.m4.1.1.2" xref="S7.p1.4.m4.1.1.2.cmml">S</mi><mn id="S7.p1.4.m4.1.1.3" xref="S7.p1.4.m4.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S7.p1.4.m4.1b"><apply id="S7.p1.4.m4.1.1.cmml" xref="S7.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S7.p1.4.m4.1.1.1.cmml" xref="S7.p1.4.m4.1.1">subscript</csymbol><ci id="S7.p1.4.m4.1.1.2.cmml" xref="S7.p1.4.m4.1.1.2">𝑆</ci><cn type="integer" id="S7.p1.4.m4.1.1.3.cmml" xref="S7.p1.4.m4.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.4.m4.1c">S_{5}</annotation></semantics></math> can be naturally expressed by true recurrent models like RNNs and captures the essence of hard state tracking due to its <math id="S7.p1.5.m5.1" class="ltx_Math" alttext="\mathsf{NC}^{1}" display="inline"><semantics id="S7.p1.5.m5.1a"><msup id="S7.p1.5.m5.1.1" xref="S7.p1.5.m5.1.1.cmml"><mi id="S7.p1.5.m5.1.1.2" xref="S7.p1.5.m5.1.1.2.cmml">𝖭𝖢</mi><mn id="S7.p1.5.m5.1.1.3" xref="S7.p1.5.m5.1.1.3.cmml">1</mn></msup><annotation-xml encoding="MathML-Content" id="S7.p1.5.m5.1b"><apply id="S7.p1.5.m5.1.1.cmml" xref="S7.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S7.p1.5.m5.1.1.1.cmml" xref="S7.p1.5.m5.1.1">superscript</csymbol><ci id="S7.p1.5.m5.1.1.2.cmml" xref="S7.p1.5.m5.1.1.2">𝖭𝖢</ci><cn type="integer" id="S7.p1.5.m5.1.1.3.cmml" xref="S7.p1.5.m5.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.5.m5.1c">\mathsf{NC}^{1}</annotation></semantics></math>-completeness. In practice, one-layer RNNs can easily learn a task capturing <math id="S7.p1.6.m6.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S7.p1.6.m6.1a"><msub id="S7.p1.6.m6.1.1" xref="S7.p1.6.m6.1.1.cmml"><mi id="S7.p1.6.m6.1.1.2" xref="S7.p1.6.m6.1.1.2.cmml">S</mi><mn id="S7.p1.6.m6.1.1.3" xref="S7.p1.6.m6.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S7.p1.6.m6.1b"><apply id="S7.p1.6.m6.1.1.cmml" xref="S7.p1.6.m6.1.1"><csymbol cd="ambiguous" id="S7.p1.6.m6.1.1.1.cmml" xref="S7.p1.6.m6.1.1">subscript</csymbol><ci id="S7.p1.6.m6.1.1.2.cmml" xref="S7.p1.6.m6.1.1.2">𝑆</ci><cn type="integer" id="S7.p1.6.m6.1.1.3.cmml" xref="S7.p1.6.m6.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.6.m6.1c">S_{5}</annotation></semantics></math> SSMs require depth growing with the sequence length. These results reveal that SSMs cannot truly track state: rather, they can only solve simple state-tracking problems for which shallow shortcuts exist <cite class="ltx_cite ltx_citemacro_citep">(Liu et al., <a href="#bib.bib21" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
<div id="S7.p2" class="ltx_para">
<p id="S7.p2.1" class="ltx_p">We also showed that simple extensions of SSMs can express <math id="S7.p2.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="S7.p2.1.m1.1a"><msub id="S7.p2.1.m1.1.1" xref="S7.p2.1.m1.1.1.cmml"><mi id="S7.p2.1.m1.1.1.2" xref="S7.p2.1.m1.1.1.2.cmml">S</mi><mn id="S7.p2.1.m1.1.1.3" xref="S7.p2.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="S7.p2.1.m1.1b"><apply id="S7.p2.1.m1.1.1.cmml" xref="S7.p2.1.m1.1.1"><csymbol cd="ambiguous" id="S7.p2.1.m1.1.1.1.cmml" xref="S7.p2.1.m1.1.1">subscript</csymbol><ci id="S7.p2.1.m1.1.1.2.cmml" xref="S7.p2.1.m1.1.1.2">𝑆</ci><cn type="integer" id="S7.p2.1.m1.1.1.3.cmml" xref="S7.p2.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S7.p2.1.m1.1c">S_{5}</annotation></semantics></math>, although this comes with potential drawbacks as far as parallelism and learning dynamics.
In future work, it would be interesting to more thoroughly explore the practical viability of our SSM extensions.
Ultimately, this line of work has the potential to unlock new neural architectures that balance the parallelism of transformers and SSMs with full expressive power for state tracking, enabling LLMs that can benefit from scale while enjoying a greater capacity to reason about games, code, and language.
</p>
</div>
</section>
<section id="Sx1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">Broader Impact</h2>

<div id="Sx1.p1" class="ltx_para">
<p id="Sx1.p1.1" class="ltx_p">This paper aims to advance the foundational understanding of state-space architectures for deep learning.
Such work can affect the development and deployment of deep learning models in a variety of ways, which in turn can have societal impacts. However, we find it difficult to meaningfully speculate about or anticipate these downstream impacts here.</p>
</div>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Angluin et al. (2023)</span>
<span class="ltx_bibblock">
Angluin, D., Chiang, D., and Yang, A.

</span>
<span class="ltx_bibblock">Masked hard-attention transformers and Boolean RASP recognize exactly the star-free languages, 2023.

</span>
<span class="ltx_bibblock">arXiv:2310.13897.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Barrington (1989)</span>
<span class="ltx_bibblock">
Barrington, D. A.

</span>
<span class="ltx_bibblock">Bounded-width polynomial-size branching programs recognize exactly those languages in nc1.

</span>
<span class="ltx_bibblock"><em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">Journal of Computer and System Sciences</em>, 38(1):150–164, 1989.

</span>
<span class="ltx_bibblock">ISSN 0022-0000.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">https://doi.org/10.1016/0022-0000(89)90037-8</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://www.sciencedirect.com/science/article/pii/0022000089900378" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://www.sciencedirect.com/science/article/pii/0022000089900378</a>.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chiang et al. (2023)</span>
<span class="ltx_bibblock">
Chiang, D., Cholak, P., and Pillay, A.

</span>
<span class="ltx_bibblock">Tighter bounds on the expressivity of transformer encoders.

</span>
<span class="ltx_bibblock">In <em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">ICML</em>, 2023.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chiu et al. (2001)</span>
<span class="ltx_bibblock">
Chiu, A., Davida, G. I., and Litow, B. E.

</span>
<span class="ltx_bibblock">Division in logspace-uniform NC1.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">RAIRO Theor. Informatics Appl.</em>, 35:259–275, 2001.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cho et al. (2014)</span>
<span class="ltx_bibblock">
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y.

</span>
<span class="ltx_bibblock">Learning phrase representations using RNN encoder–decoder for statistical machine translation.

</span>
<span class="ltx_bibblock">In Moschitti, A., Pang, B., and Daelemans, W. (eds.), <em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)</em>, pp.  1724–1734, Doha, Qatar, October 2014. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.3115/v1/D14-1179</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/D14-1179" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/D14-1179</a>.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Feng et al. (2023)</span>
<span class="ltx_bibblock">
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., and Wang, L.

</span>
<span class="ltx_bibblock">Towards revealing the mystery behind chain of thought: A theoretical perspective.

</span>
<span class="ltx_bibblock">In <em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">Thirty-seventh Conference on Neural Information Processing Systems</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=qHrADgAdYu" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=qHrADgAdYu</a>.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Fu et al. (2023)</span>
<span class="ltx_bibblock">
Fu, D. Y., Dao, T., Saab, K. K., Thomas, A. W., Rudra, A., and Re, C.

</span>
<span class="ltx_bibblock">Hungry hungry hippos: Towards language modeling with state space models.

</span>
<span class="ltx_bibblock">In <em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=COZDy0WYGg" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=COZDy0WYGg</a>.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gu &amp; Dao (2023)</span>
<span class="ltx_bibblock">
Gu, A. and Dao, T.

</span>
<span class="ltx_bibblock">Mamba: Linear-time sequence modeling with selective state spaces, 2023.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gu et al. (2021)</span>
<span class="ltx_bibblock">
Gu, A., Johnson, I., Goel, K., Saab, K. K., Dao, T., Rudra, A., and Re, C.

</span>
<span class="ltx_bibblock">Combining recurrent, convolutional, and continuous-time models with linear state space layers.

</span>
<span class="ltx_bibblock">In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), <em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 2021.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=yWd42CWN3c" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=yWd42CWN3c</a>.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gu et al. (2022a)</span>
<span class="ltx_bibblock">
Gu, A., Goel, K., and Re, C.

</span>
<span class="ltx_bibblock">Efficiently modeling long sequences with structured state spaces.

</span>
<span class="ltx_bibblock">In <em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">International Conference on Learning Representations</em>, 2022a.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=uYLFoz1vlAC" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=uYLFoz1vlAC</a>.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gu et al. (2022b)</span>
<span class="ltx_bibblock">
Gu, A., Goel, K., Saab, K., and Ré, C.

</span>
<span class="ltx_bibblock">Structured state spaces: Combining continuous-time, recurrent, and convolutional models, January 2022b.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://hazyresearch.stanford.edu/blog/2022-01-14-s4-3" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://hazyresearch.stanford.edu/blog/2022-01-14-s4-3</a>.

</span>
<span class="ltx_bibblock">Blog post accessed January 31, 2024.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hao et al. (2022)</span>
<span class="ltx_bibblock">
Hao, S., Angluin, D., and Frank, R.

</span>
<span class="ltx_bibblock">Formal language recognition by hard attention transformers: Perspectives from circuit complexity.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">TACL</em>, 10:800–810, 2022.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Heim (1983)</span>
<span class="ltx_bibblock">
Heim, I.

</span>
<span class="ltx_bibblock">File change semantics and the familiarity theory of definiteness.

</span>
<span class="ltx_bibblock"><em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">Semantics Critical Concepts in Linguistics</em>, pp.  108–135, 1983.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hesse (2001)</span>
<span class="ltx_bibblock">
Hesse, W.

</span>
<span class="ltx_bibblock">Division is in uniform <math id="bib.bib14.1.m1.1" class="ltx_Math" alttext="TC^{0}" display="inline"><semantics id="bib.bib14.1.m1.1a"><mrow id="bib.bib14.1.m1.1.1" xref="bib.bib14.1.m1.1.1.cmml"><mi id="bib.bib14.1.m1.1.1.2" xref="bib.bib14.1.m1.1.1.2.cmml">T</mi><mo lspace="0em" rspace="0em" id="bib.bib14.1.m1.1.1.1" xref="bib.bib14.1.m1.1.1.1.cmml">​</mo><msup id="bib.bib14.1.m1.1.1.3" xref="bib.bib14.1.m1.1.1.3.cmml"><mi id="bib.bib14.1.m1.1.1.3.2" xref="bib.bib14.1.m1.1.1.3.2.cmml">C</mi><mn id="bib.bib14.1.m1.1.1.3.3" xref="bib.bib14.1.m1.1.1.3.3.cmml">0</mn></msup></mrow><annotation-xml encoding="MathML-Content" id="bib.bib14.1.m1.1b"><apply id="bib.bib14.1.m1.1.1.cmml" xref="bib.bib14.1.m1.1.1"><times id="bib.bib14.1.m1.1.1.1.cmml" xref="bib.bib14.1.m1.1.1.1"></times><ci id="bib.bib14.1.m1.1.1.2.cmml" xref="bib.bib14.1.m1.1.1.2">𝑇</ci><apply id="bib.bib14.1.m1.1.1.3.cmml" xref="bib.bib14.1.m1.1.1.3"><csymbol cd="ambiguous" id="bib.bib14.1.m1.1.1.3.1.cmml" xref="bib.bib14.1.m1.1.1.3">superscript</csymbol><ci id="bib.bib14.1.m1.1.1.3.2.cmml" xref="bib.bib14.1.m1.1.1.3.2">𝐶</ci><cn type="integer" id="bib.bib14.1.m1.1.1.3.3.cmml" xref="bib.bib14.1.m1.1.1.3.3">0</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="bib.bib14.1.m1.1c">TC^{0}</annotation></semantics></math>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib14.2.1" class="ltx_emph ltx_font_italic">International Colloquium on Automata, Languages, and Programming</em>, pp.  104–114, 2001.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hesse et al. (2002)</span>
<span class="ltx_bibblock">
Hesse, W., Allender, E., and Barrington, D. A. M.

</span>
<span class="ltx_bibblock">Uniform constant-depth threshold circuits for division and iterated multiplication.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">J. Comput. Syst. Sci.</em>, 65:695–716, 2002.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hochreiter &amp; Schmidhuber (1997)</span>
<span class="ltx_bibblock">
Hochreiter, S. and Schmidhuber, J.

</span>
<span class="ltx_bibblock">Long short-term memory.

</span>
<span class="ltx_bibblock"><em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">Neural Comput.</em>, 9(8):1735–1780, nov 1997.

</span>
<span class="ltx_bibblock">ISSN 0899-7667.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1162/neco.1997.9.8.1735</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://doi.org/10.1162/neco.1997.9.8.1735" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://doi.org/10.1162/neco.1997.9.8.1735</a>.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hopcroft et al. (2001)</span>
<span class="ltx_bibblock">
Hopcroft, J. E., Motwani, R., and Ullman, J. D.

</span>
<span class="ltx_bibblock">Introduction to automata theory, languages, and computation.

</span>
<span class="ltx_bibblock"><em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">Acm Sigact News</em>, 32(1):60–65, 2001.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Immerman &amp; Landau (1989)</span>
<span class="ltx_bibblock">
Immerman, N. and Landau, S.

</span>
<span class="ltx_bibblock">The complexity of iterated multiplication.

</span>
<span class="ltx_bibblock">In <em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">[1989] Proceedings. Structure in Complexity Theory Fourth Annual Conference</em>, pp.  104–111, 1989.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1109/SCT.1989.41816</span>.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kim &amp; Schuster (2023)</span>
<span class="ltx_bibblock">
Kim, N. and Schuster, S.

</span>
<span class="ltx_bibblock">Entity tracking in language models.

</span>
<span class="ltx_bibblock">In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), <em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pp.  3835–3855, Toronto, Canada, July 2023. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/2023.acl-long.213</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2023.acl-long.213" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2023.acl-long.213</a>.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Krohn &amp; Rhodes (1965)</span>
<span class="ltx_bibblock">
Krohn, K. and Rhodes, J.

</span>
<span class="ltx_bibblock">Algebraic theory of machines. i. prime decomposition theorem for finite semigroups and machines.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">Transactions of the American Mathematical Society</em>, 116:450–464, 1965.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. (2023)</span>
<span class="ltx_bibblock">
Liu, B., Ash, J. T., Goel, S., Krishnamurthy, A., and Zhang, C.

</span>
<span class="ltx_bibblock">Transformers learn shortcuts to automata.

</span>
<span class="ltx_bibblock">In <em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">The Eleventh International Conference on Learning Representations</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=De4FYqjFueZ" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=De4FYqjFueZ</a>.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">McKenzie et al. (1991)</span>
<span class="ltx_bibblock">
McKenzie, P., Péladeau, P., and Therien, D.

</span>
<span class="ltx_bibblock">NC1: The automata-theoretic viewpoint.

</span>
<span class="ltx_bibblock"><em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">Computational complexity</em>, 1:330–359, 1991.

</span>
<span class="ltx_bibblock">ISSN 1016-3328,1420-8954.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1007/bf01212963</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://link.springer.com/article/10.1007/BF01212963#citeas" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://link.springer.com/article/10.1007/BF01212963#citeas</a>.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mereghetti &amp; Palano (2000)</span>
<span class="ltx_bibblock">
Mereghetti, C. and Palano, B.

</span>
<span class="ltx_bibblock">Threshold circuits for iterated matrix product and powering.

</span>
<span class="ltx_bibblock"><em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">RAIRO-Theor. Inf. Appl.</em>, 34(1):39–46, 2000.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1051/ita:2000105</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://doi.org/10.1051/ita:2000105" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://doi.org/10.1051/ita:2000105</a>.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Merrill (2019)</span>
<span class="ltx_bibblock">
Merrill, W.

</span>
<span class="ltx_bibblock">Sequential neural networks as automata.

</span>
<span class="ltx_bibblock">In Eisner, J., Gallé, M., Heinz, J., Quattoni, A., and Rabusseau, G. (eds.), <em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">Proceedings of the Workshop on Deep Learning and Formal Languages: Building Bridges</em>, pp.  1–13, Florence, August 2019. Association for Computational Linguistics.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.18653/v1/W19-3901</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/W19-3901" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/W19-3901</a>.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Merrill &amp; Sabharwal (2023a)</span>
<span class="ltx_bibblock">
Merrill, W. and Sabharwal, A.

</span>
<span class="ltx_bibblock">The parallelism tradeoff: Limitations of log-precision transformers.

</span>
<span class="ltx_bibblock"><em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">Transactions of the Association for Computational Linguistics</em>, 11, 2023a.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://aclanthology.org/2023.tacl-1.31" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://aclanthology.org/2023.tacl-1.31</a>.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Merrill &amp; Sabharwal (2023b)</span>
<span class="ltx_bibblock">
Merrill, W. and Sabharwal, A.

</span>
<span class="ltx_bibblock">A logic for expressing log-precision transformers.

</span>
<span class="ltx_bibblock">In <em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">NeurIPS</em>, 2023b.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Minsky (1954)</span>
<span class="ltx_bibblock">
Minsky, M.

</span>
<span class="ltx_bibblock">Neural nets and the brain-model problem.

</span>
<span class="ltx_bibblock"><em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">Unpublished doctoral dissertation, Princeton University, NJ</em>, 1954.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mohri (2009)</span>
<span class="ltx_bibblock">
Mohri, M.

</span>
<span class="ltx_bibblock"><em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">Weighted Automata Algorithms</em>, pp.  213–254.

</span>
<span class="ltx_bibblock">Springer Berlin Heidelberg, Berlin, Heidelberg, 2009.

</span>
<span class="ltx_bibblock">ISBN 978-3-642-01492-5.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1007/978-3-642-01492-5˙6</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://doi.org/10.1007/978-3-642-01492-5_6" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://doi.org/10.1007/978-3-642-01492-5_6</a>.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Reif &amp; Tate (1992)</span>
<span class="ltx_bibblock">
Reif, J. H. and Tate, S. R.

</span>
<span class="ltx_bibblock">On threshold circuits and polynomial computation.

</span>
<span class="ltx_bibblock"><em id="bib.bib29.1.1" class="ltx_emph ltx_font_italic">SIAM Journal on Computing</em>, 21(5):896–908, 1992.

</span>
<span class="ltx_bibblock">doi: <span class="ltx_ref ltx_nolink ltx_Url ltx_ref_self">10.1137/0221053</span>.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://doi.org/10.1137/0221053" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://doi.org/10.1137/0221053</a>.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Strobl et al. (2023)</span>
<span class="ltx_bibblock">
Strobl, L., Merrill, W., Weiss, G., Chiang, D., and Angluin, D.

</span>
<span class="ltx_bibblock">Transformers as recognizers of formal languages: A survey on expressivity, 2023.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2024)</span>
<span class="ltx_bibblock">
Wang, J., Gangavarapu, T., Yan, J. N., and Rush, A. M.

</span>
<span class="ltx_bibblock">Mambabyte: Token-free selective state space model, 2024.

</span>
</li>
</ul>
</section>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Floating-Point Arithmetic</h2>

<div id="A1.Thmtheorem1" class="ltx_theorem ltx_theorem_definition">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="A1.Thmtheorem1.2.1.1" class="ltx_text ltx_font_bold">Definition A.1</span></span><span id="A1.Thmtheorem1.3.2" class="ltx_text ltx_font_bold"> </span>(Iterated <math id="A1.Thmtheorem1.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem1.1.m1.1b"><mi id="A1.Thmtheorem1.1.m1.1.1" xref="A1.Thmtheorem1.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem1.1.m1.1c"><ci id="A1.Thmtheorem1.1.m1.1.1.cmml" xref="A1.Thmtheorem1.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem1.1.m1.1d">\mathbb{D}</annotation></semantics></math>-product)<span id="A1.Thmtheorem1.4.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="A1.Thmtheorem1.p1" class="ltx_para">
<p id="A1.Thmtheorem1.p1.2" class="ltx_p">For <math id="A1.Thmtheorem1.p1.1.m1.3" class="ltx_Math" alttext="\phi_{1},\ldots,\phi_{z}\in\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem1.p1.1.m1.3a"><mrow id="A1.Thmtheorem1.p1.1.m1.3.3" xref="A1.Thmtheorem1.p1.1.m1.3.3.cmml"><mrow id="A1.Thmtheorem1.p1.1.m1.3.3.2.2" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.3.cmml"><msub id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.cmml"><mi id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.2" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.2.cmml">ϕ</mi><mn id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.3" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.3.cmml">1</mn></msub><mo id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.3" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="A1.Thmtheorem1.p1.1.m1.1.1" xref="A1.Thmtheorem1.p1.1.m1.1.1.cmml">…</mi><mo id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.4" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.3.cmml">,</mo><msub id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.cmml"><mi id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.2" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.2.cmml">ϕ</mi><mi id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.3" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.3.cmml">z</mi></msub></mrow><mo id="A1.Thmtheorem1.p1.1.m1.3.3.3" xref="A1.Thmtheorem1.p1.1.m1.3.3.3.cmml">∈</mo><mi id="A1.Thmtheorem1.p1.1.m1.3.3.4" xref="A1.Thmtheorem1.p1.1.m1.3.3.4.cmml">𝔻</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem1.p1.1.m1.3b"><apply id="A1.Thmtheorem1.p1.1.m1.3.3.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3"><in id="A1.Thmtheorem1.p1.1.m1.3.3.3.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.3"></in><list id="A1.Thmtheorem1.p1.1.m1.3.3.2.3.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2"><apply id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.cmml" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.1.cmml" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1">subscript</csymbol><ci id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.2.cmml" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.2">italic-ϕ</ci><cn type="integer" id="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.3.cmml" xref="A1.Thmtheorem1.p1.1.m1.2.2.1.1.1.3">1</cn></apply><ci id="A1.Thmtheorem1.p1.1.m1.1.1.cmml" xref="A1.Thmtheorem1.p1.1.m1.1.1">…</ci><apply id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.1.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2">subscript</csymbol><ci id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.2.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.2">italic-ϕ</ci><ci id="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.3.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.2.2.2.3">𝑧</ci></apply></list><ci id="A1.Thmtheorem1.p1.1.m1.3.3.4.cmml" xref="A1.Thmtheorem1.p1.1.m1.3.3.4">𝔻</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem1.p1.1.m1.3c">\phi_{1},\ldots,\phi_{z}\in\mathbb{D}</annotation></semantics></math>, their <em id="A1.Thmtheorem1.p1.2.1" class="ltx_emph ltx_font_italic">iterated <math id="A1.Thmtheorem1.p1.2.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem1.p1.2.1.m1.1a"><mi id="A1.Thmtheorem1.p1.2.1.m1.1.1" xref="A1.Thmtheorem1.p1.2.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem1.p1.2.1.m1.1b"><ci id="A1.Thmtheorem1.p1.2.1.m1.1.1.cmml" xref="A1.Thmtheorem1.p1.2.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem1.p1.2.1.m1.1c">\mathbb{D}</annotation></semantics></math>-product</em> is:</p>
<table id="A1.E8" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="A1.E8.m1.1" class="ltx_Math" alttext="\bigotimes_{i=1}^{z}\phi_{i}\,\triangleq\,\mathsf{cast}_{\mathbb{D}}\left(\prod_{i=1}^{z}\mathsf{cast}_{\mathbb{R}}(\phi_{i})\right)." display="block"><semantics id="A1.E8.m1.1a"><mrow id="A1.E8.m1.1.1.1" xref="A1.E8.m1.1.1.1.1.cmml"><mrow id="A1.E8.m1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.cmml"><mrow id="A1.E8.m1.1.1.1.1.3" xref="A1.E8.m1.1.1.1.1.3.cmml"><munderover id="A1.E8.m1.1.1.1.1.3.1" xref="A1.E8.m1.1.1.1.1.3.1.cmml"><mo movablelimits="false" id="A1.E8.m1.1.1.1.1.3.1.2.2" xref="A1.E8.m1.1.1.1.1.3.1.2.2.cmml">⨂</mo><mrow id="A1.E8.m1.1.1.1.1.3.1.2.3" xref="A1.E8.m1.1.1.1.1.3.1.2.3.cmml"><mi id="A1.E8.m1.1.1.1.1.3.1.2.3.2" xref="A1.E8.m1.1.1.1.1.3.1.2.3.2.cmml">i</mi><mo id="A1.E8.m1.1.1.1.1.3.1.2.3.1" xref="A1.E8.m1.1.1.1.1.3.1.2.3.1.cmml">=</mo><mn id="A1.E8.m1.1.1.1.1.3.1.2.3.3" xref="A1.E8.m1.1.1.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="A1.E8.m1.1.1.1.1.3.1.3" xref="A1.E8.m1.1.1.1.1.3.1.3.cmml">z</mi></munderover><msub id="A1.E8.m1.1.1.1.1.3.2" xref="A1.E8.m1.1.1.1.1.3.2.cmml"><mi id="A1.E8.m1.1.1.1.1.3.2.2" xref="A1.E8.m1.1.1.1.1.3.2.2.cmml">ϕ</mi><mi id="A1.E8.m1.1.1.1.1.3.2.3" xref="A1.E8.m1.1.1.1.1.3.2.3.cmml">i</mi></msub></mrow><mo rspace="0.448em" id="A1.E8.m1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.2.cmml">≜</mo><mrow id="A1.E8.m1.1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.1.cmml"><msub id="A1.E8.m1.1.1.1.1.1.3" xref="A1.E8.m1.1.1.1.1.1.3.cmml"><mi id="A1.E8.m1.1.1.1.1.1.3.2" xref="A1.E8.m1.1.1.1.1.1.3.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.E8.m1.1.1.1.1.1.3.3" xref="A1.E8.m1.1.1.1.1.1.3.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="A1.E8.m1.1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.1.2.cmml">​</mo><mrow id="A1.E8.m1.1.1.1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.1.1.1.1.cmml"><mo id="A1.E8.m1.1.1.1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.cmml">(</mo><mrow id="A1.E8.m1.1.1.1.1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.1.1.1.1.cmml"><munderover id="A1.E8.m1.1.1.1.1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.cmml"><mo lspace="0em" movablelimits="false" id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.2.cmml">∏</mo><mrow id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.cmml"><mi id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.2.cmml">i</mi><mo id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.1" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.1.cmml">=</mo><mn id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.3.cmml">1</mn></mrow><mi id="A1.E8.m1.1.1.1.1.1.1.1.1.2.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.3.cmml">z</mi></munderover><mrow id="A1.E8.m1.1.1.1.1.1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.cmml"><msub id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.cmml"><mi id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.3.cmml">ℝ</mi></msub><mo lspace="0em" rspace="0em" id="A1.E8.m1.1.1.1.1.1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.2.cmml">​</mo><mrow id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.cmml">(</mo><msub id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.cmml"><mi id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.2.cmml">ϕ</mi><mi id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo id="A1.E8.m1.1.1.1.1.1.1.1.3" xref="A1.E8.m1.1.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="A1.E8.m1.1.1.1.2" xref="A1.E8.m1.1.1.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="A1.E8.m1.1b"><apply id="A1.E8.m1.1.1.1.1.cmml" xref="A1.E8.m1.1.1.1"><ci id="A1.E8.m1.1.1.1.1.2.cmml" xref="A1.E8.m1.1.1.1.1.2">≜</ci><apply id="A1.E8.m1.1.1.1.1.3.cmml" xref="A1.E8.m1.1.1.1.1.3"><apply id="A1.E8.m1.1.1.1.1.3.1.cmml" xref="A1.E8.m1.1.1.1.1.3.1"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.3.1.1.cmml" xref="A1.E8.m1.1.1.1.1.3.1">superscript</csymbol><apply id="A1.E8.m1.1.1.1.1.3.1.2.cmml" xref="A1.E8.m1.1.1.1.1.3.1"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.3.1.2.1.cmml" xref="A1.E8.m1.1.1.1.1.3.1">subscript</csymbol><csymbol cd="latexml" id="A1.E8.m1.1.1.1.1.3.1.2.2.cmml" xref="A1.E8.m1.1.1.1.1.3.1.2.2">tensor-product</csymbol><apply id="A1.E8.m1.1.1.1.1.3.1.2.3.cmml" xref="A1.E8.m1.1.1.1.1.3.1.2.3"><eq id="A1.E8.m1.1.1.1.1.3.1.2.3.1.cmml" xref="A1.E8.m1.1.1.1.1.3.1.2.3.1"></eq><ci id="A1.E8.m1.1.1.1.1.3.1.2.3.2.cmml" xref="A1.E8.m1.1.1.1.1.3.1.2.3.2">𝑖</ci><cn type="integer" id="A1.E8.m1.1.1.1.1.3.1.2.3.3.cmml" xref="A1.E8.m1.1.1.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="A1.E8.m1.1.1.1.1.3.1.3.cmml" xref="A1.E8.m1.1.1.1.1.3.1.3">𝑧</ci></apply><apply id="A1.E8.m1.1.1.1.1.3.2.cmml" xref="A1.E8.m1.1.1.1.1.3.2"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.3.2.1.cmml" xref="A1.E8.m1.1.1.1.1.3.2">subscript</csymbol><ci id="A1.E8.m1.1.1.1.1.3.2.2.cmml" xref="A1.E8.m1.1.1.1.1.3.2.2">italic-ϕ</ci><ci id="A1.E8.m1.1.1.1.1.3.2.3.cmml" xref="A1.E8.m1.1.1.1.1.3.2.3">𝑖</ci></apply></apply><apply id="A1.E8.m1.1.1.1.1.1.cmml" xref="A1.E8.m1.1.1.1.1.1"><times id="A1.E8.m1.1.1.1.1.1.2.cmml" xref="A1.E8.m1.1.1.1.1.1.2"></times><apply id="A1.E8.m1.1.1.1.1.1.3.cmml" xref="A1.E8.m1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.1.3.1.cmml" xref="A1.E8.m1.1.1.1.1.1.3">subscript</csymbol><ci id="A1.E8.m1.1.1.1.1.1.3.2.cmml" xref="A1.E8.m1.1.1.1.1.1.3.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.E8.m1.1.1.1.1.1.3.3.cmml" xref="A1.E8.m1.1.1.1.1.1.3.3">𝔻</ci></apply><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1"><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.1.1.1.1.2.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2">superscript</csymbol><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.2">product</csymbol><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3"><eq id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.1"></eq><ci id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.2">𝑖</ci><cn type="integer" id="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.3.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.2.3.3">1</cn></apply></apply><ci id="A1.E8.m1.1.1.1.1.1.1.1.1.2.3.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.2.3">𝑧</ci></apply><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1"><times id="A1.E8.m1.1.1.1.1.1.1.1.1.1.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.2"></times><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3">subscript</csymbol><ci id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.3.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.3.3">ℝ</ci></apply><apply id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.2">italic-ϕ</ci><ci id="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="A1.E8.m1.1.1.1.1.1.1.1.1.1.1.1.1.3">𝑖</ci></apply></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.E8.m1.1c">\bigotimes_{i=1}^{z}\phi_{i}\,\triangleq\,\mathsf{cast}_{\mathbb{D}}\left(\prod_{i=1}^{z}\mathsf{cast}_{\mathbb{R}}(\phi_{i})\right).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(8)</span></td>
</tr></tbody>
</table>
</div>
</div>
<div id="A1.p1" class="ltx_para">
<p id="A1.p1.5" class="ltx_p">Here <math id="A1.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}" display="inline"><semantics id="A1.p1.1.m1.1a"><msub id="A1.p1.1.m1.1.1" xref="A1.p1.1.m1.1.1.cmml"><mi id="A1.p1.1.m1.1.1.2" xref="A1.p1.1.m1.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.p1.1.m1.1.1.3" xref="A1.p1.1.m1.1.1.3.cmml">ℝ</mi></msub><annotation-xml encoding="MathML-Content" id="A1.p1.1.m1.1b"><apply id="A1.p1.1.m1.1.1.cmml" xref="A1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="A1.p1.1.m1.1.1.1.cmml" xref="A1.p1.1.m1.1.1">subscript</csymbol><ci id="A1.p1.1.m1.1.1.2.cmml" xref="A1.p1.1.m1.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.p1.1.m1.1.1.3.cmml" xref="A1.p1.1.m1.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p1.1.m1.1c">\mathsf{cast}_{\mathbb{R}}</annotation></semantics></math> converts a number in <math id="A1.p1.2.m2.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.p1.2.m2.1a"><mi id="A1.p1.2.m2.1.1" xref="A1.p1.2.m2.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.p1.2.m2.1b"><ci id="A1.p1.2.m2.1.1.cmml" xref="A1.p1.2.m2.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p1.2.m2.1c">\mathbb{D}</annotation></semantics></math> to the corresponding real number. <math id="A1.p1.3.m3.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.p1.3.m3.1a"><mi id="A1.p1.3.m3.1.1" xref="A1.p1.3.m3.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.p1.3.m3.1b"><ci id="A1.p1.3.m3.1.1.cmml" xref="A1.p1.3.m3.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p1.3.m3.1c">\mathbb{D}</annotation></semantics></math> is implicit in the notations <math id="A1.p1.4.m4.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}" display="inline"><semantics id="A1.p1.4.m4.1a"><msub id="A1.p1.4.m4.1.1" xref="A1.p1.4.m4.1.1.cmml"><mi id="A1.p1.4.m4.1.1.2" xref="A1.p1.4.m4.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.p1.4.m4.1.1.3" xref="A1.p1.4.m4.1.1.3.cmml">ℝ</mi></msub><annotation-xml encoding="MathML-Content" id="A1.p1.4.m4.1b"><apply id="A1.p1.4.m4.1.1.cmml" xref="A1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="A1.p1.4.m4.1.1.1.cmml" xref="A1.p1.4.m4.1.1">subscript</csymbol><ci id="A1.p1.4.m4.1.1.2.cmml" xref="A1.p1.4.m4.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.p1.4.m4.1.1.3.cmml" xref="A1.p1.4.m4.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p1.4.m4.1c">\mathsf{cast}_{\mathbb{R}}</annotation></semantics></math> and <math id="A1.p1.5.m5.1" class="ltx_Math" alttext="\bigotimes" display="inline"><semantics id="A1.p1.5.m5.1a"><mo id="A1.p1.5.m5.1.1" xref="A1.p1.5.m5.1.1.cmml">⨂</mo><annotation-xml encoding="MathML-Content" id="A1.p1.5.m5.1b"><csymbol cd="latexml" id="A1.p1.5.m5.1.1.cmml" xref="A1.p1.5.m5.1.1">tensor-product</csymbol></annotation-xml><annotation encoding="application/x-tex" id="A1.p1.5.m5.1c">\bigotimes</annotation></semantics></math>.</p>
</div>
<div id="A1.Thmtheorem2" class="ltx_theorem ltx_theorem_lemma">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="A1.Thmtheorem2.2.1.1" class="ltx_text ltx_font_bold">Lemma A.2</span></span><span id="A1.Thmtheorem2.3.2" class="ltx_text ltx_font_bold"> </span>(Iterated <math id="A1.Thmtheorem2.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem2.1.m1.1b"><mi id="A1.Thmtheorem2.1.m1.1.1" xref="A1.Thmtheorem2.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.1.m1.1c"><ci id="A1.Thmtheorem2.1.m1.1.1.cmml" xref="A1.Thmtheorem2.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.1.m1.1d">\mathbb{D}</annotation></semantics></math>-product)<span id="A1.Thmtheorem2.4.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="A1.Thmtheorem2.p1" class="ltx_para">
<p id="A1.Thmtheorem2.p1.12" class="ltx_p"><span id="A1.Thmtheorem2.p1.12.12" class="ltx_text ltx_font_italic">Let <math id="A1.Thmtheorem2.p1.1.1.m1.3" class="ltx_Math" alttext="\phi_{1},\ldots,\phi_{z}\in\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem2.p1.1.1.m1.3a"><mrow id="A1.Thmtheorem2.p1.1.1.m1.3.3" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.cmml"><mrow id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.3.cmml"><msub id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.cmml"><mi id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.2" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.2.cmml">ϕ</mi><mn id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.3" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.3.cmml">1</mn></msub><mo id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.3" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.3.cmml">,</mo><mi mathvariant="normal" id="A1.Thmtheorem2.p1.1.1.m1.1.1" xref="A1.Thmtheorem2.p1.1.1.m1.1.1.cmml">…</mi><mo id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.4" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.3.cmml">,</mo><msub id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.cmml"><mi id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.2" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.2.cmml">ϕ</mi><mi id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.3" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.3.cmml">z</mi></msub></mrow><mo id="A1.Thmtheorem2.p1.1.1.m1.3.3.3" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.3.cmml">∈</mo><mi id="A1.Thmtheorem2.p1.1.1.m1.3.3.4" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.4.cmml">𝔻</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.1.1.m1.3b"><apply id="A1.Thmtheorem2.p1.1.1.m1.3.3.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3"><in id="A1.Thmtheorem2.p1.1.1.m1.3.3.3.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.3"></in><list id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.3.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2"><apply id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.1.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1">subscript</csymbol><ci id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.2.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.2">italic-ϕ</ci><cn type="integer" id="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.3.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.2.2.1.1.1.3">1</cn></apply><ci id="A1.Thmtheorem2.p1.1.1.m1.1.1.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.1.1">…</ci><apply id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.1.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2">subscript</csymbol><ci id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.2.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.2">italic-ϕ</ci><ci id="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.3.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.2.2.2.3">𝑧</ci></apply></list><ci id="A1.Thmtheorem2.p1.1.1.m1.3.3.4.cmml" xref="A1.Thmtheorem2.p1.1.1.m1.3.3.4">𝔻</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.1.1.m1.3c">\phi_{1},\ldots,\phi_{z}\in\mathbb{D}</annotation></semantics></math> be such that <math id="A1.Thmtheorem2.p1.2.2.m2.1" class="ltx_Math" alttext="z\leq n" display="inline"><semantics id="A1.Thmtheorem2.p1.2.2.m2.1a"><mrow id="A1.Thmtheorem2.p1.2.2.m2.1.1" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.cmml"><mi id="A1.Thmtheorem2.p1.2.2.m2.1.1.2" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.2.cmml">z</mi><mo id="A1.Thmtheorem2.p1.2.2.m2.1.1.1" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.1.cmml">≤</mo><mi id="A1.Thmtheorem2.p1.2.2.m2.1.1.3" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.3.cmml">n</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.2.2.m2.1b"><apply id="A1.Thmtheorem2.p1.2.2.m2.1.1.cmml" xref="A1.Thmtheorem2.p1.2.2.m2.1.1"><leq id="A1.Thmtheorem2.p1.2.2.m2.1.1.1.cmml" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.1"></leq><ci id="A1.Thmtheorem2.p1.2.2.m2.1.1.2.cmml" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.2">𝑧</ci><ci id="A1.Thmtheorem2.p1.2.2.m2.1.1.3.cmml" xref="A1.Thmtheorem2.p1.2.2.m2.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.2.2.m2.1c">z\leq n</annotation></semantics></math> and each <math id="A1.Thmtheorem2.p1.3.3.m3.1" class="ltx_Math" alttext="\phi_{i}" display="inline"><semantics id="A1.Thmtheorem2.p1.3.3.m3.1a"><msub id="A1.Thmtheorem2.p1.3.3.m3.1.1" xref="A1.Thmtheorem2.p1.3.3.m3.1.1.cmml"><mi id="A1.Thmtheorem2.p1.3.3.m3.1.1.2" xref="A1.Thmtheorem2.p1.3.3.m3.1.1.2.cmml">ϕ</mi><mi id="A1.Thmtheorem2.p1.3.3.m3.1.1.3" xref="A1.Thmtheorem2.p1.3.3.m3.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.3.3.m3.1b"><apply id="A1.Thmtheorem2.p1.3.3.m3.1.1.cmml" xref="A1.Thmtheorem2.p1.3.3.m3.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.3.3.m3.1.1.1.cmml" xref="A1.Thmtheorem2.p1.3.3.m3.1.1">subscript</csymbol><ci id="A1.Thmtheorem2.p1.3.3.m3.1.1.2.cmml" xref="A1.Thmtheorem2.p1.3.3.m3.1.1.2">italic-ϕ</ci><ci id="A1.Thmtheorem2.p1.3.3.m3.1.1.3.cmml" xref="A1.Thmtheorem2.p1.3.3.m3.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.3.3.m3.1c">\phi_{i}</annotation></semantics></math> can be represented as an <math id="A1.Thmtheorem2.p1.4.4.m4.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.Thmtheorem2.p1.4.4.m4.1a"><mi id="A1.Thmtheorem2.p1.4.4.m4.1.1" xref="A1.Thmtheorem2.p1.4.4.m4.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.4.4.m4.1b"><ci id="A1.Thmtheorem2.p1.4.4.m4.1.1.cmml" xref="A1.Thmtheorem2.p1.4.4.m4.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.4.4.m4.1c">n</annotation></semantics></math>-bit integer. If operators <math id="A1.Thmtheorem2.p1.5.5.m5.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{D}}" display="inline"><semantics id="A1.Thmtheorem2.p1.5.5.m5.1a"><msub id="A1.Thmtheorem2.p1.5.5.m5.1.1" xref="A1.Thmtheorem2.p1.5.5.m5.1.1.cmml"><mi id="A1.Thmtheorem2.p1.5.5.m5.1.1.2" xref="A1.Thmtheorem2.p1.5.5.m5.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.Thmtheorem2.p1.5.5.m5.1.1.3" xref="A1.Thmtheorem2.p1.5.5.m5.1.1.3.cmml">𝔻</mi></msub><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.5.5.m5.1b"><apply id="A1.Thmtheorem2.p1.5.5.m5.1.1.cmml" xref="A1.Thmtheorem2.p1.5.5.m5.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.5.5.m5.1.1.1.cmml" xref="A1.Thmtheorem2.p1.5.5.m5.1.1">subscript</csymbol><ci id="A1.Thmtheorem2.p1.5.5.m5.1.1.2.cmml" xref="A1.Thmtheorem2.p1.5.5.m5.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.Thmtheorem2.p1.5.5.m5.1.1.3.cmml" xref="A1.Thmtheorem2.p1.5.5.m5.1.1.3">𝔻</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.5.5.m5.1c">\mathsf{cast}_{\mathbb{D}}</annotation></semantics></math> and <math id="A1.Thmtheorem2.p1.6.6.m6.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}" display="inline"><semantics id="A1.Thmtheorem2.p1.6.6.m6.1a"><msub id="A1.Thmtheorem2.p1.6.6.m6.1.1" xref="A1.Thmtheorem2.p1.6.6.m6.1.1.cmml"><mi id="A1.Thmtheorem2.p1.6.6.m6.1.1.2" xref="A1.Thmtheorem2.p1.6.6.m6.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.Thmtheorem2.p1.6.6.m6.1.1.3" xref="A1.Thmtheorem2.p1.6.6.m6.1.1.3.cmml">ℝ</mi></msub><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.6.6.m6.1b"><apply id="A1.Thmtheorem2.p1.6.6.m6.1.1.cmml" xref="A1.Thmtheorem2.p1.6.6.m6.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.6.6.m6.1.1.1.cmml" xref="A1.Thmtheorem2.p1.6.6.m6.1.1">subscript</csymbol><ci id="A1.Thmtheorem2.p1.6.6.m6.1.1.2.cmml" xref="A1.Thmtheorem2.p1.6.6.m6.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.Thmtheorem2.p1.6.6.m6.1.1.3.cmml" xref="A1.Thmtheorem2.p1.6.6.m6.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.6.6.m6.1c">\mathsf{cast}_{\mathbb{R}}</annotation></semantics></math> are in <math id="A1.Thmtheorem2.p1.7.7.m7.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.Thmtheorem2.p1.7.7.m7.1a"><mi id="A1.Thmtheorem2.p1.7.7.m7.1.1" xref="A1.Thmtheorem2.p1.7.7.m7.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.7.7.m7.1b"><ci id="A1.Thmtheorem2.p1.7.7.m7.1.1.cmml" xref="A1.Thmtheorem2.p1.7.7.m7.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.7.7.m7.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.Thmtheorem2.p1.8.8.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.Thmtheorem2.p1.8.8.m8.1a"><msup id="A1.Thmtheorem2.p1.8.8.m8.1.1" xref="A1.Thmtheorem2.p1.8.8.m8.1.1.cmml"><mi id="A1.Thmtheorem2.p1.8.8.m8.1.1.2" xref="A1.Thmtheorem2.p1.8.8.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.Thmtheorem2.p1.8.8.m8.1.1.3" xref="A1.Thmtheorem2.p1.8.8.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.8.8.m8.1b"><apply id="A1.Thmtheorem2.p1.8.8.m8.1.1.cmml" xref="A1.Thmtheorem2.p1.8.8.m8.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.8.8.m8.1.1.1.cmml" xref="A1.Thmtheorem2.p1.8.8.m8.1.1">superscript</csymbol><ci id="A1.Thmtheorem2.p1.8.8.m8.1.1.2.cmml" xref="A1.Thmtheorem2.p1.8.8.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.Thmtheorem2.p1.8.8.m8.1.1.3.cmml" xref="A1.Thmtheorem2.p1.8.8.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.8.8.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math>, then the iterated <math id="A1.Thmtheorem2.p1.9.9.m9.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem2.p1.9.9.m9.1a"><mi id="A1.Thmtheorem2.p1.9.9.m9.1.1" xref="A1.Thmtheorem2.p1.9.9.m9.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.9.9.m9.1b"><ci id="A1.Thmtheorem2.p1.9.9.m9.1.1.cmml" xref="A1.Thmtheorem2.p1.9.9.m9.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.9.9.m9.1c">\mathbb{D}</annotation></semantics></math>-product <math id="A1.Thmtheorem2.p1.10.10.m10.1" class="ltx_Math" alttext="\bigotimes_{i=1}^{z}\phi_{i}" display="inline"><semantics id="A1.Thmtheorem2.p1.10.10.m10.1a"><mrow id="A1.Thmtheorem2.p1.10.10.m10.1.1" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.cmml"><msubsup id="A1.Thmtheorem2.p1.10.10.m10.1.1.1" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.cmml"><mo id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.2" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.2.cmml">⨂</mo><mrow id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.cmml"><mi id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.2" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.2.cmml">i</mi><mo id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.1" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.1.cmml">=</mo><mn id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.3" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.3" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.3.cmml">z</mi></msubsup><msub id="A1.Thmtheorem2.p1.10.10.m10.1.1.2" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2.cmml"><mi id="A1.Thmtheorem2.p1.10.10.m10.1.1.2.2" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2.2.cmml">ϕ</mi><mi id="A1.Thmtheorem2.p1.10.10.m10.1.1.2.3" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.10.10.m10.1b"><apply id="A1.Thmtheorem2.p1.10.10.m10.1.1.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1"><apply id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.1.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1">superscript</csymbol><apply id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.1.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1">subscript</csymbol><csymbol cd="latexml" id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.2.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.2">tensor-product</csymbol><apply id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3"><eq id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.1.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.1"></eq><ci id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.2.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.2">𝑖</ci><cn type="integer" id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.3.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.2.3.3">1</cn></apply></apply><ci id="A1.Thmtheorem2.p1.10.10.m10.1.1.1.3.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.1.3">𝑧</ci></apply><apply id="A1.Thmtheorem2.p1.10.10.m10.1.1.2.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.10.10.m10.1.1.2.1.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2">subscript</csymbol><ci id="A1.Thmtheorem2.p1.10.10.m10.1.1.2.2.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2.2">italic-ϕ</ci><ci id="A1.Thmtheorem2.p1.10.10.m10.1.1.2.3.cmml" xref="A1.Thmtheorem2.p1.10.10.m10.1.1.2.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.10.10.m10.1c">\bigotimes_{i=1}^{z}\phi_{i}</annotation></semantics></math> can be computed in <math id="A1.Thmtheorem2.p1.11.11.m11.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.Thmtheorem2.p1.11.11.m11.1a"><mi id="A1.Thmtheorem2.p1.11.11.m11.1.1" xref="A1.Thmtheorem2.p1.11.11.m11.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.11.11.m11.1b"><ci id="A1.Thmtheorem2.p1.11.11.m11.1.1.cmml" xref="A1.Thmtheorem2.p1.11.11.m11.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.11.11.m11.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.Thmtheorem2.p1.12.12.m12.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.Thmtheorem2.p1.12.12.m12.1a"><msup id="A1.Thmtheorem2.p1.12.12.m12.1.1" xref="A1.Thmtheorem2.p1.12.12.m12.1.1.cmml"><mi id="A1.Thmtheorem2.p1.12.12.m12.1.1.2" xref="A1.Thmtheorem2.p1.12.12.m12.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.Thmtheorem2.p1.12.12.m12.1.1.3" xref="A1.Thmtheorem2.p1.12.12.m12.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem2.p1.12.12.m12.1b"><apply id="A1.Thmtheorem2.p1.12.12.m12.1.1.cmml" xref="A1.Thmtheorem2.p1.12.12.m12.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem2.p1.12.12.m12.1.1.1.cmml" xref="A1.Thmtheorem2.p1.12.12.m12.1.1">superscript</csymbol><ci id="A1.Thmtheorem2.p1.12.12.m12.1.1.2.cmml" xref="A1.Thmtheorem2.p1.12.12.m12.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.Thmtheorem2.p1.12.12.m12.1.1.3.cmml" xref="A1.Thmtheorem2.p1.12.12.m12.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem2.p1.12.12.m12.1c">\mathsf{TC}^{0}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="A1.1" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="A1.1.p1" class="ltx_para">
<p id="A1.1.p1.17" class="ltx_p">By preconditions of the lemma, we can compute <math id="A1.1.p1.1.m1.1" class="ltx_Math" alttext="y_{i}=\mathsf{cast}_{\mathbb{R}}(\phi_{i})" display="inline"><semantics id="A1.1.p1.1.m1.1a"><mrow id="A1.1.p1.1.m1.1.1" xref="A1.1.p1.1.m1.1.1.cmml"><msub id="A1.1.p1.1.m1.1.1.3" xref="A1.1.p1.1.m1.1.1.3.cmml"><mi id="A1.1.p1.1.m1.1.1.3.2" xref="A1.1.p1.1.m1.1.1.3.2.cmml">y</mi><mi id="A1.1.p1.1.m1.1.1.3.3" xref="A1.1.p1.1.m1.1.1.3.3.cmml">i</mi></msub><mo id="A1.1.p1.1.m1.1.1.2" xref="A1.1.p1.1.m1.1.1.2.cmml">=</mo><mrow id="A1.1.p1.1.m1.1.1.1" xref="A1.1.p1.1.m1.1.1.1.cmml"><msub id="A1.1.p1.1.m1.1.1.1.3" xref="A1.1.p1.1.m1.1.1.1.3.cmml"><mi id="A1.1.p1.1.m1.1.1.1.3.2" xref="A1.1.p1.1.m1.1.1.1.3.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.1.p1.1.m1.1.1.1.3.3" xref="A1.1.p1.1.m1.1.1.1.3.3.cmml">ℝ</mi></msub><mo lspace="0em" rspace="0em" id="A1.1.p1.1.m1.1.1.1.2" xref="A1.1.p1.1.m1.1.1.1.2.cmml">​</mo><mrow id="A1.1.p1.1.m1.1.1.1.1.1" xref="A1.1.p1.1.m1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="A1.1.p1.1.m1.1.1.1.1.1.2" xref="A1.1.p1.1.m1.1.1.1.1.1.1.cmml">(</mo><msub id="A1.1.p1.1.m1.1.1.1.1.1.1" xref="A1.1.p1.1.m1.1.1.1.1.1.1.cmml"><mi id="A1.1.p1.1.m1.1.1.1.1.1.1.2" xref="A1.1.p1.1.m1.1.1.1.1.1.1.2.cmml">ϕ</mi><mi id="A1.1.p1.1.m1.1.1.1.1.1.1.3" xref="A1.1.p1.1.m1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="A1.1.p1.1.m1.1.1.1.1.1.3" xref="A1.1.p1.1.m1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.1.p1.1.m1.1b"><apply id="A1.1.p1.1.m1.1.1.cmml" xref="A1.1.p1.1.m1.1.1"><eq id="A1.1.p1.1.m1.1.1.2.cmml" xref="A1.1.p1.1.m1.1.1.2"></eq><apply id="A1.1.p1.1.m1.1.1.3.cmml" xref="A1.1.p1.1.m1.1.1.3"><csymbol cd="ambiguous" id="A1.1.p1.1.m1.1.1.3.1.cmml" xref="A1.1.p1.1.m1.1.1.3">subscript</csymbol><ci id="A1.1.p1.1.m1.1.1.3.2.cmml" xref="A1.1.p1.1.m1.1.1.3.2">𝑦</ci><ci id="A1.1.p1.1.m1.1.1.3.3.cmml" xref="A1.1.p1.1.m1.1.1.3.3">𝑖</ci></apply><apply id="A1.1.p1.1.m1.1.1.1.cmml" xref="A1.1.p1.1.m1.1.1.1"><times id="A1.1.p1.1.m1.1.1.1.2.cmml" xref="A1.1.p1.1.m1.1.1.1.2"></times><apply id="A1.1.p1.1.m1.1.1.1.3.cmml" xref="A1.1.p1.1.m1.1.1.1.3"><csymbol cd="ambiguous" id="A1.1.p1.1.m1.1.1.1.3.1.cmml" xref="A1.1.p1.1.m1.1.1.1.3">subscript</csymbol><ci id="A1.1.p1.1.m1.1.1.1.3.2.cmml" xref="A1.1.p1.1.m1.1.1.1.3.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.1.p1.1.m1.1.1.1.3.3.cmml" xref="A1.1.p1.1.m1.1.1.1.3.3">ℝ</ci></apply><apply id="A1.1.p1.1.m1.1.1.1.1.1.1.cmml" xref="A1.1.p1.1.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="A1.1.p1.1.m1.1.1.1.1.1.1.1.cmml" xref="A1.1.p1.1.m1.1.1.1.1.1">subscript</csymbol><ci id="A1.1.p1.1.m1.1.1.1.1.1.1.2.cmml" xref="A1.1.p1.1.m1.1.1.1.1.1.1.2">italic-ϕ</ci><ci id="A1.1.p1.1.m1.1.1.1.1.1.1.3.cmml" xref="A1.1.p1.1.m1.1.1.1.1.1.1.3">𝑖</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.1.m1.1c">y_{i}=\mathsf{cast}_{\mathbb{R}}(\phi_{i})</annotation></semantics></math> for each <math id="A1.1.p1.2.m2.1" class="ltx_Math" alttext="i" display="inline"><semantics id="A1.1.p1.2.m2.1a"><mi id="A1.1.p1.2.m2.1.1" xref="A1.1.p1.2.m2.1.1.cmml">i</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.2.m2.1b"><ci id="A1.1.p1.2.m2.1.1.cmml" xref="A1.1.p1.2.m2.1.1">𝑖</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.2.m2.1c">i</annotation></semantics></math> in <math id="A1.1.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.1.p1.3.m3.1a"><mi id="A1.1.p1.3.m3.1.1" xref="A1.1.p1.3.m3.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.3.m3.1b"><ci id="A1.1.p1.3.m3.1.1.cmml" xref="A1.1.p1.3.m3.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.3.m3.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.1.p1.4.m4.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.1.p1.4.m4.1a"><msup id="A1.1.p1.4.m4.1.1" xref="A1.1.p1.4.m4.1.1.cmml"><mi id="A1.1.p1.4.m4.1.1.2" xref="A1.1.p1.4.m4.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.1.p1.4.m4.1.1.3" xref="A1.1.p1.4.m4.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.1.p1.4.m4.1b"><apply id="A1.1.p1.4.m4.1.1.cmml" xref="A1.1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="A1.1.p1.4.m4.1.1.1.cmml" xref="A1.1.p1.4.m4.1.1">superscript</csymbol><ci id="A1.1.p1.4.m4.1.1.2.cmml" xref="A1.1.p1.4.m4.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.1.p1.4.m4.1.1.3.cmml" xref="A1.1.p1.4.m4.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.4.m4.1c">\mathsf{TC}^{0}</annotation></semantics></math>. Since each <math id="A1.1.p1.5.m5.1" class="ltx_Math" alttext="\phi_{i}" display="inline"><semantics id="A1.1.p1.5.m5.1a"><msub id="A1.1.p1.5.m5.1.1" xref="A1.1.p1.5.m5.1.1.cmml"><mi id="A1.1.p1.5.m5.1.1.2" xref="A1.1.p1.5.m5.1.1.2.cmml">ϕ</mi><mi id="A1.1.p1.5.m5.1.1.3" xref="A1.1.p1.5.m5.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.1.p1.5.m5.1b"><apply id="A1.1.p1.5.m5.1.1.cmml" xref="A1.1.p1.5.m5.1.1"><csymbol cd="ambiguous" id="A1.1.p1.5.m5.1.1.1.cmml" xref="A1.1.p1.5.m5.1.1">subscript</csymbol><ci id="A1.1.p1.5.m5.1.1.2.cmml" xref="A1.1.p1.5.m5.1.1.2">italic-ϕ</ci><ci id="A1.1.p1.5.m5.1.1.3.cmml" xref="A1.1.p1.5.m5.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.5.m5.1c">\phi_{i}</annotation></semantics></math> is equivalent to an <math id="A1.1.p1.6.m6.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.1.p1.6.m6.1a"><mi id="A1.1.p1.6.m6.1.1" xref="A1.1.p1.6.m6.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.6.m6.1b"><ci id="A1.1.p1.6.m6.1.1.cmml" xref="A1.1.p1.6.m6.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.6.m6.1c">n</annotation></semantics></math>-bit integer, <math id="A1.1.p1.7.m7.1" class="ltx_Math" alttext="y_{i}" display="inline"><semantics id="A1.1.p1.7.m7.1a"><msub id="A1.1.p1.7.m7.1.1" xref="A1.1.p1.7.m7.1.1.cmml"><mi id="A1.1.p1.7.m7.1.1.2" xref="A1.1.p1.7.m7.1.1.2.cmml">y</mi><mi id="A1.1.p1.7.m7.1.1.3" xref="A1.1.p1.7.m7.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.1.p1.7.m7.1b"><apply id="A1.1.p1.7.m7.1.1.cmml" xref="A1.1.p1.7.m7.1.1"><csymbol cd="ambiguous" id="A1.1.p1.7.m7.1.1.1.cmml" xref="A1.1.p1.7.m7.1.1">subscript</csymbol><ci id="A1.1.p1.7.m7.1.1.2.cmml" xref="A1.1.p1.7.m7.1.1.2">𝑦</ci><ci id="A1.1.p1.7.m7.1.1.3.cmml" xref="A1.1.p1.7.m7.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.7.m7.1c">y_{i}</annotation></semantics></math> can be viewed as an <math id="A1.1.p1.8.m8.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.1.p1.8.m8.1a"><mi id="A1.1.p1.8.m8.1.1" xref="A1.1.p1.8.m8.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.8.m8.1b"><ci id="A1.1.p1.8.m8.1.1.cmml" xref="A1.1.p1.8.m8.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.8.m8.1c">n</annotation></semantics></math>-bit integer. The iterated integer product <math id="A1.1.p1.9.m9.1" class="ltx_Math" alttext="y=\prod_{i=1}^{z}y_{i}" display="inline"><semantics id="A1.1.p1.9.m9.1a"><mrow id="A1.1.p1.9.m9.1.1" xref="A1.1.p1.9.m9.1.1.cmml"><mi id="A1.1.p1.9.m9.1.1.2" xref="A1.1.p1.9.m9.1.1.2.cmml">y</mi><mo rspace="0.111em" id="A1.1.p1.9.m9.1.1.1" xref="A1.1.p1.9.m9.1.1.1.cmml">=</mo><mrow id="A1.1.p1.9.m9.1.1.3" xref="A1.1.p1.9.m9.1.1.3.cmml"><msubsup id="A1.1.p1.9.m9.1.1.3.1" xref="A1.1.p1.9.m9.1.1.3.1.cmml"><mo id="A1.1.p1.9.m9.1.1.3.1.2.2" xref="A1.1.p1.9.m9.1.1.3.1.2.2.cmml">∏</mo><mrow id="A1.1.p1.9.m9.1.1.3.1.2.3" xref="A1.1.p1.9.m9.1.1.3.1.2.3.cmml"><mi id="A1.1.p1.9.m9.1.1.3.1.2.3.2" xref="A1.1.p1.9.m9.1.1.3.1.2.3.2.cmml">i</mi><mo id="A1.1.p1.9.m9.1.1.3.1.2.3.1" xref="A1.1.p1.9.m9.1.1.3.1.2.3.1.cmml">=</mo><mn id="A1.1.p1.9.m9.1.1.3.1.2.3.3" xref="A1.1.p1.9.m9.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="A1.1.p1.9.m9.1.1.3.1.3" xref="A1.1.p1.9.m9.1.1.3.1.3.cmml">z</mi></msubsup><msub id="A1.1.p1.9.m9.1.1.3.2" xref="A1.1.p1.9.m9.1.1.3.2.cmml"><mi id="A1.1.p1.9.m9.1.1.3.2.2" xref="A1.1.p1.9.m9.1.1.3.2.2.cmml">y</mi><mi id="A1.1.p1.9.m9.1.1.3.2.3" xref="A1.1.p1.9.m9.1.1.3.2.3.cmml">i</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.1.p1.9.m9.1b"><apply id="A1.1.p1.9.m9.1.1.cmml" xref="A1.1.p1.9.m9.1.1"><eq id="A1.1.p1.9.m9.1.1.1.cmml" xref="A1.1.p1.9.m9.1.1.1"></eq><ci id="A1.1.p1.9.m9.1.1.2.cmml" xref="A1.1.p1.9.m9.1.1.2">𝑦</ci><apply id="A1.1.p1.9.m9.1.1.3.cmml" xref="A1.1.p1.9.m9.1.1.3"><apply id="A1.1.p1.9.m9.1.1.3.1.cmml" xref="A1.1.p1.9.m9.1.1.3.1"><csymbol cd="ambiguous" id="A1.1.p1.9.m9.1.1.3.1.1.cmml" xref="A1.1.p1.9.m9.1.1.3.1">superscript</csymbol><apply id="A1.1.p1.9.m9.1.1.3.1.2.cmml" xref="A1.1.p1.9.m9.1.1.3.1"><csymbol cd="ambiguous" id="A1.1.p1.9.m9.1.1.3.1.2.1.cmml" xref="A1.1.p1.9.m9.1.1.3.1">subscript</csymbol><csymbol cd="latexml" id="A1.1.p1.9.m9.1.1.3.1.2.2.cmml" xref="A1.1.p1.9.m9.1.1.3.1.2.2">product</csymbol><apply id="A1.1.p1.9.m9.1.1.3.1.2.3.cmml" xref="A1.1.p1.9.m9.1.1.3.1.2.3"><eq id="A1.1.p1.9.m9.1.1.3.1.2.3.1.cmml" xref="A1.1.p1.9.m9.1.1.3.1.2.3.1"></eq><ci id="A1.1.p1.9.m9.1.1.3.1.2.3.2.cmml" xref="A1.1.p1.9.m9.1.1.3.1.2.3.2">𝑖</ci><cn type="integer" id="A1.1.p1.9.m9.1.1.3.1.2.3.3.cmml" xref="A1.1.p1.9.m9.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="A1.1.p1.9.m9.1.1.3.1.3.cmml" xref="A1.1.p1.9.m9.1.1.3.1.3">𝑧</ci></apply><apply id="A1.1.p1.9.m9.1.1.3.2.cmml" xref="A1.1.p1.9.m9.1.1.3.2"><csymbol cd="ambiguous" id="A1.1.p1.9.m9.1.1.3.2.1.cmml" xref="A1.1.p1.9.m9.1.1.3.2">subscript</csymbol><ci id="A1.1.p1.9.m9.1.1.3.2.2.cmml" xref="A1.1.p1.9.m9.1.1.3.2.2">𝑦</ci><ci id="A1.1.p1.9.m9.1.1.3.2.3.cmml" xref="A1.1.p1.9.m9.1.1.3.2.3">𝑖</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.9.m9.1c">y=\prod_{i=1}^{z}y_{i}</annotation></semantics></math> can be computed with an <math id="A1.1.p1.10.m10.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.1.p1.10.m10.1a"><mi id="A1.1.p1.10.m10.1.1" xref="A1.1.p1.10.m10.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.10.m10.1b"><ci id="A1.1.p1.10.m10.1.1.cmml" xref="A1.1.p1.10.m10.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.10.m10.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.1.p1.11.m11.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.1.p1.11.m11.1a"><msup id="A1.1.p1.11.m11.1.1" xref="A1.1.p1.11.m11.1.1.cmml"><mi id="A1.1.p1.11.m11.1.1.2" xref="A1.1.p1.11.m11.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.1.p1.11.m11.1.1.3" xref="A1.1.p1.11.m11.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.1.p1.11.m11.1b"><apply id="A1.1.p1.11.m11.1.1.cmml" xref="A1.1.p1.11.m11.1.1"><csymbol cd="ambiguous" id="A1.1.p1.11.m11.1.1.1.cmml" xref="A1.1.p1.11.m11.1.1">superscript</csymbol><ci id="A1.1.p1.11.m11.1.1.2.cmml" xref="A1.1.p1.11.m11.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.1.p1.11.m11.1.1.3.cmml" xref="A1.1.p1.11.m11.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.11.m11.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit <cite class="ltx_cite ltx_citemacro_citep">(Hesse, <a href="#bib.bib14" title="" class="ltx_ref">2001</a>)</cite>. Finally, by a precondition of the lemma, we can cast the result back to <math id="A1.1.p1.12.m12.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.1.p1.12.m12.1a"><mi id="A1.1.p1.12.m12.1.1" xref="A1.1.p1.12.m12.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.12.m12.1b"><ci id="A1.1.p1.12.m12.1.1.cmml" xref="A1.1.p1.12.m12.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.12.m12.1c">\mathbb{D}</annotation></semantics></math>, i.e., compute <math id="A1.1.p1.13.m13.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{D}}(y)" display="inline"><semantics id="A1.1.p1.13.m13.1a"><mrow id="A1.1.p1.13.m13.1.2" xref="A1.1.p1.13.m13.1.2.cmml"><msub id="A1.1.p1.13.m13.1.2.2" xref="A1.1.p1.13.m13.1.2.2.cmml"><mi id="A1.1.p1.13.m13.1.2.2.2" xref="A1.1.p1.13.m13.1.2.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.1.p1.13.m13.1.2.2.3" xref="A1.1.p1.13.m13.1.2.2.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="A1.1.p1.13.m13.1.2.1" xref="A1.1.p1.13.m13.1.2.1.cmml">​</mo><mrow id="A1.1.p1.13.m13.1.2.3.2" xref="A1.1.p1.13.m13.1.2.cmml"><mo stretchy="false" id="A1.1.p1.13.m13.1.2.3.2.1" xref="A1.1.p1.13.m13.1.2.cmml">(</mo><mi id="A1.1.p1.13.m13.1.1" xref="A1.1.p1.13.m13.1.1.cmml">y</mi><mo stretchy="false" id="A1.1.p1.13.m13.1.2.3.2.2" xref="A1.1.p1.13.m13.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.1.p1.13.m13.1b"><apply id="A1.1.p1.13.m13.1.2.cmml" xref="A1.1.p1.13.m13.1.2"><times id="A1.1.p1.13.m13.1.2.1.cmml" xref="A1.1.p1.13.m13.1.2.1"></times><apply id="A1.1.p1.13.m13.1.2.2.cmml" xref="A1.1.p1.13.m13.1.2.2"><csymbol cd="ambiguous" id="A1.1.p1.13.m13.1.2.2.1.cmml" xref="A1.1.p1.13.m13.1.2.2">subscript</csymbol><ci id="A1.1.p1.13.m13.1.2.2.2.cmml" xref="A1.1.p1.13.m13.1.2.2.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.1.p1.13.m13.1.2.2.3.cmml" xref="A1.1.p1.13.m13.1.2.2.3">𝔻</ci></apply><ci id="A1.1.p1.13.m13.1.1.cmml" xref="A1.1.p1.13.m13.1.1">𝑦</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.13.m13.1c">\mathsf{cast}_{\mathbb{D}}(y)</annotation></semantics></math> which equals the iterated <math id="A1.1.p1.14.m14.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.1.p1.14.m14.1a"><mi id="A1.1.p1.14.m14.1.1" xref="A1.1.p1.14.m14.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.14.m14.1b"><ci id="A1.1.p1.14.m14.1.1.cmml" xref="A1.1.p1.14.m14.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.14.m14.1c">\mathbb{D}</annotation></semantics></math>-product <math id="A1.1.p1.15.m15.1" class="ltx_Math" alttext="\bigotimes_{i=1}^{z}\phi_{i}" display="inline"><semantics id="A1.1.p1.15.m15.1a"><mrow id="A1.1.p1.15.m15.1.1" xref="A1.1.p1.15.m15.1.1.cmml"><msubsup id="A1.1.p1.15.m15.1.1.1" xref="A1.1.p1.15.m15.1.1.1.cmml"><mo id="A1.1.p1.15.m15.1.1.1.2.2" xref="A1.1.p1.15.m15.1.1.1.2.2.cmml">⨂</mo><mrow id="A1.1.p1.15.m15.1.1.1.2.3" xref="A1.1.p1.15.m15.1.1.1.2.3.cmml"><mi id="A1.1.p1.15.m15.1.1.1.2.3.2" xref="A1.1.p1.15.m15.1.1.1.2.3.2.cmml">i</mi><mo id="A1.1.p1.15.m15.1.1.1.2.3.1" xref="A1.1.p1.15.m15.1.1.1.2.3.1.cmml">=</mo><mn id="A1.1.p1.15.m15.1.1.1.2.3.3" xref="A1.1.p1.15.m15.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="A1.1.p1.15.m15.1.1.1.3" xref="A1.1.p1.15.m15.1.1.1.3.cmml">z</mi></msubsup><msub id="A1.1.p1.15.m15.1.1.2" xref="A1.1.p1.15.m15.1.1.2.cmml"><mi id="A1.1.p1.15.m15.1.1.2.2" xref="A1.1.p1.15.m15.1.1.2.2.cmml">ϕ</mi><mi id="A1.1.p1.15.m15.1.1.2.3" xref="A1.1.p1.15.m15.1.1.2.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="A1.1.p1.15.m15.1b"><apply id="A1.1.p1.15.m15.1.1.cmml" xref="A1.1.p1.15.m15.1.1"><apply id="A1.1.p1.15.m15.1.1.1.cmml" xref="A1.1.p1.15.m15.1.1.1"><csymbol cd="ambiguous" id="A1.1.p1.15.m15.1.1.1.1.cmml" xref="A1.1.p1.15.m15.1.1.1">superscript</csymbol><apply id="A1.1.p1.15.m15.1.1.1.2.cmml" xref="A1.1.p1.15.m15.1.1.1"><csymbol cd="ambiguous" id="A1.1.p1.15.m15.1.1.1.2.1.cmml" xref="A1.1.p1.15.m15.1.1.1">subscript</csymbol><csymbol cd="latexml" id="A1.1.p1.15.m15.1.1.1.2.2.cmml" xref="A1.1.p1.15.m15.1.1.1.2.2">tensor-product</csymbol><apply id="A1.1.p1.15.m15.1.1.1.2.3.cmml" xref="A1.1.p1.15.m15.1.1.1.2.3"><eq id="A1.1.p1.15.m15.1.1.1.2.3.1.cmml" xref="A1.1.p1.15.m15.1.1.1.2.3.1"></eq><ci id="A1.1.p1.15.m15.1.1.1.2.3.2.cmml" xref="A1.1.p1.15.m15.1.1.1.2.3.2">𝑖</ci><cn type="integer" id="A1.1.p1.15.m15.1.1.1.2.3.3.cmml" xref="A1.1.p1.15.m15.1.1.1.2.3.3">1</cn></apply></apply><ci id="A1.1.p1.15.m15.1.1.1.3.cmml" xref="A1.1.p1.15.m15.1.1.1.3">𝑧</ci></apply><apply id="A1.1.p1.15.m15.1.1.2.cmml" xref="A1.1.p1.15.m15.1.1.2"><csymbol cd="ambiguous" id="A1.1.p1.15.m15.1.1.2.1.cmml" xref="A1.1.p1.15.m15.1.1.2">subscript</csymbol><ci id="A1.1.p1.15.m15.1.1.2.2.cmml" xref="A1.1.p1.15.m15.1.1.2.2">italic-ϕ</ci><ci id="A1.1.p1.15.m15.1.1.2.3.cmml" xref="A1.1.p1.15.m15.1.1.2.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.15.m15.1c">\bigotimes_{i=1}^{z}\phi_{i}</annotation></semantics></math>, with an <math id="A1.1.p1.16.m16.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.1.p1.16.m16.1a"><mi id="A1.1.p1.16.m16.1.1" xref="A1.1.p1.16.m16.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.1.p1.16.m16.1b"><ci id="A1.1.p1.16.m16.1.1.cmml" xref="A1.1.p1.16.m16.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.16.m16.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.1.p1.17.m17.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.1.p1.17.m17.1a"><msup id="A1.1.p1.17.m17.1.1" xref="A1.1.p1.17.m17.1.1.cmml"><mi id="A1.1.p1.17.m17.1.1.2" xref="A1.1.p1.17.m17.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.1.p1.17.m17.1.1.3" xref="A1.1.p1.17.m17.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.1.p1.17.m17.1b"><apply id="A1.1.p1.17.m17.1.1.cmml" xref="A1.1.p1.17.m17.1.1"><csymbol cd="ambiguous" id="A1.1.p1.17.m17.1.1.1.cmml" xref="A1.1.p1.17.m17.1.1">superscript</csymbol><ci id="A1.1.p1.17.m17.1.1.2.cmml" xref="A1.1.p1.17.m17.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.1.p1.17.m17.1.1.3.cmml" xref="A1.1.p1.17.m17.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.1.p1.17.m17.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit.
∎</p>
</div>
</div>
<div id="A1.p2" class="ltx_para">
<p id="A1.p2.1" class="ltx_p">See <a href="#S2.Thmtheorem3" title="Lemma 2.3 (Iterated float product). ‣ 2.2 Numeric Datatype ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.3</span></a></p>
</div>
<div id="A1.6" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="A1.2.p1" class="ltx_para">
<p id="A1.2.p1.2" class="ltx_p">The idea is to convert (by scaling up) the sequence of <math id="A1.2.p1.1.m1.1" class="ltx_Math" alttext="\phi_{i}" display="inline"><semantics id="A1.2.p1.1.m1.1a"><msub id="A1.2.p1.1.m1.1.1" xref="A1.2.p1.1.m1.1.1.cmml"><mi id="A1.2.p1.1.m1.1.1.2" xref="A1.2.p1.1.m1.1.1.2.cmml">ϕ</mi><mi id="A1.2.p1.1.m1.1.1.3" xref="A1.2.p1.1.m1.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.2.p1.1.m1.1b"><apply id="A1.2.p1.1.m1.1.1.cmml" xref="A1.2.p1.1.m1.1.1"><csymbol cd="ambiguous" id="A1.2.p1.1.m1.1.1.1.cmml" xref="A1.2.p1.1.m1.1.1">subscript</csymbol><ci id="A1.2.p1.1.m1.1.1.2.cmml" xref="A1.2.p1.1.m1.1.1.2">italic-ϕ</ci><ci id="A1.2.p1.1.m1.1.1.3.cmml" xref="A1.2.p1.1.m1.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.2.p1.1.m1.1c">\phi_{i}</annotation></semantics></math> to another sequence of floats that are all representable as integers, apply <a href="#A1.Thmtheorem2" title="Lemma A.2 (Iterated 𝔻-product). ‣ Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">A.2</span></a>, reverse the scaling, and cast the result back to a <math id="A1.2.p1.2.m2.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.2.p1.2.m2.1a"><mrow id="A1.2.p1.2.m2.1.1" xref="A1.2.p1.2.m2.1.1.cmml"><mi id="A1.2.p1.2.m2.1.1.2" xref="A1.2.p1.2.m2.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.2.p1.2.m2.1.1.1" xref="A1.2.p1.2.m2.1.1.1.cmml">​</mo><mrow id="A1.2.p1.2.m2.1.1.3" xref="A1.2.p1.2.m2.1.1.3.cmml"><mi id="A1.2.p1.2.m2.1.1.3.1" xref="A1.2.p1.2.m2.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.2.p1.2.m2.1.1.3a" xref="A1.2.p1.2.m2.1.1.3.cmml">⁡</mo><mi id="A1.2.p1.2.m2.1.1.3.2" xref="A1.2.p1.2.m2.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.2.p1.2.m2.1b"><apply id="A1.2.p1.2.m2.1.1.cmml" xref="A1.2.p1.2.m2.1.1"><times id="A1.2.p1.2.m2.1.1.1.cmml" xref="A1.2.p1.2.m2.1.1.1"></times><ci id="A1.2.p1.2.m2.1.1.2.cmml" xref="A1.2.p1.2.m2.1.1.2">𝑐</ci><apply id="A1.2.p1.2.m2.1.1.3.cmml" xref="A1.2.p1.2.m2.1.1.3"><log id="A1.2.p1.2.m2.1.1.3.1.cmml" xref="A1.2.p1.2.m2.1.1.3.1"></log><ci id="A1.2.p1.2.m2.1.1.3.2.cmml" xref="A1.2.p1.2.m2.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.2.p1.2.m2.1c">c\log n</annotation></semantics></math> precision float.</p>
</div>
<div id="A1.3.p2" class="ltx_para">
<p id="A1.3.p2.13" class="ltx_p">Let <math id="A1.3.p2.1.m1.1" class="ltx_Math" alttext="e" display="inline"><semantics id="A1.3.p2.1.m1.1a"><mi id="A1.3.p2.1.m1.1.1" xref="A1.3.p2.1.m1.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="A1.3.p2.1.m1.1b"><ci id="A1.3.p2.1.m1.1.1.cmml" xref="A1.3.p2.1.m1.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.1.m1.1c">e</annotation></semantics></math> be the smallest exponent across all <math id="A1.3.p2.2.m2.1" class="ltx_Math" alttext="\phi_{i}" display="inline"><semantics id="A1.3.p2.2.m2.1a"><msub id="A1.3.p2.2.m2.1.1" xref="A1.3.p2.2.m2.1.1.cmml"><mi id="A1.3.p2.2.m2.1.1.2" xref="A1.3.p2.2.m2.1.1.2.cmml">ϕ</mi><mi id="A1.3.p2.2.m2.1.1.3" xref="A1.3.p2.2.m2.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.3.p2.2.m2.1b"><apply id="A1.3.p2.2.m2.1.1.cmml" xref="A1.3.p2.2.m2.1.1"><csymbol cd="ambiguous" id="A1.3.p2.2.m2.1.1.1.cmml" xref="A1.3.p2.2.m2.1.1">subscript</csymbol><ci id="A1.3.p2.2.m2.1.1.2.cmml" xref="A1.3.p2.2.m2.1.1.2">italic-ϕ</ci><ci id="A1.3.p2.2.m2.1.1.3.cmml" xref="A1.3.p2.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.2.m2.1c">\phi_{i}</annotation></semantics></math> and <math id="A1.3.p2.3.m3.3" class="ltx_Math" alttext="q=\max\{0,-e\}" display="inline"><semantics id="A1.3.p2.3.m3.3a"><mrow id="A1.3.p2.3.m3.3.3" xref="A1.3.p2.3.m3.3.3.cmml"><mi id="A1.3.p2.3.m3.3.3.3" xref="A1.3.p2.3.m3.3.3.3.cmml">q</mi><mo id="A1.3.p2.3.m3.3.3.2" xref="A1.3.p2.3.m3.3.3.2.cmml">=</mo><mrow id="A1.3.p2.3.m3.3.3.1.1" xref="A1.3.p2.3.m3.3.3.1.2.cmml"><mi id="A1.3.p2.3.m3.1.1" xref="A1.3.p2.3.m3.1.1.cmml">max</mi><mo id="A1.3.p2.3.m3.3.3.1.1a" xref="A1.3.p2.3.m3.3.3.1.2.cmml">⁡</mo><mrow id="A1.3.p2.3.m3.3.3.1.1.1" xref="A1.3.p2.3.m3.3.3.1.2.cmml"><mo stretchy="false" id="A1.3.p2.3.m3.3.3.1.1.1.2" xref="A1.3.p2.3.m3.3.3.1.2.cmml">{</mo><mn id="A1.3.p2.3.m3.2.2" xref="A1.3.p2.3.m3.2.2.cmml">0</mn><mo id="A1.3.p2.3.m3.3.3.1.1.1.3" xref="A1.3.p2.3.m3.3.3.1.2.cmml">,</mo><mrow id="A1.3.p2.3.m3.3.3.1.1.1.1" xref="A1.3.p2.3.m3.3.3.1.1.1.1.cmml"><mo id="A1.3.p2.3.m3.3.3.1.1.1.1a" xref="A1.3.p2.3.m3.3.3.1.1.1.1.cmml">−</mo><mi id="A1.3.p2.3.m3.3.3.1.1.1.1.2" xref="A1.3.p2.3.m3.3.3.1.1.1.1.2.cmml">e</mi></mrow><mo stretchy="false" id="A1.3.p2.3.m3.3.3.1.1.1.4" xref="A1.3.p2.3.m3.3.3.1.2.cmml">}</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.3.p2.3.m3.3b"><apply id="A1.3.p2.3.m3.3.3.cmml" xref="A1.3.p2.3.m3.3.3"><eq id="A1.3.p2.3.m3.3.3.2.cmml" xref="A1.3.p2.3.m3.3.3.2"></eq><ci id="A1.3.p2.3.m3.3.3.3.cmml" xref="A1.3.p2.3.m3.3.3.3">𝑞</ci><apply id="A1.3.p2.3.m3.3.3.1.2.cmml" xref="A1.3.p2.3.m3.3.3.1.1"><max id="A1.3.p2.3.m3.1.1.cmml" xref="A1.3.p2.3.m3.1.1"></max><cn type="integer" id="A1.3.p2.3.m3.2.2.cmml" xref="A1.3.p2.3.m3.2.2">0</cn><apply id="A1.3.p2.3.m3.3.3.1.1.1.1.cmml" xref="A1.3.p2.3.m3.3.3.1.1.1.1"><minus id="A1.3.p2.3.m3.3.3.1.1.1.1.1.cmml" xref="A1.3.p2.3.m3.3.3.1.1.1.1"></minus><ci id="A1.3.p2.3.m3.3.3.1.1.1.1.2.cmml" xref="A1.3.p2.3.m3.3.3.1.1.1.1.2">𝑒</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.3.m3.3c">q=\max\{0,-e\}</annotation></semantics></math>. Construct re-scaled floats <math id="A1.3.p2.4.m4.1" class="ltx_Math" alttext="\psi_{i}=\phi_{i}2^{q}" display="inline"><semantics id="A1.3.p2.4.m4.1a"><mrow id="A1.3.p2.4.m4.1.1" xref="A1.3.p2.4.m4.1.1.cmml"><msub id="A1.3.p2.4.m4.1.1.2" xref="A1.3.p2.4.m4.1.1.2.cmml"><mi id="A1.3.p2.4.m4.1.1.2.2" xref="A1.3.p2.4.m4.1.1.2.2.cmml">ψ</mi><mi id="A1.3.p2.4.m4.1.1.2.3" xref="A1.3.p2.4.m4.1.1.2.3.cmml">i</mi></msub><mo id="A1.3.p2.4.m4.1.1.1" xref="A1.3.p2.4.m4.1.1.1.cmml">=</mo><mrow id="A1.3.p2.4.m4.1.1.3" xref="A1.3.p2.4.m4.1.1.3.cmml"><msub id="A1.3.p2.4.m4.1.1.3.2" xref="A1.3.p2.4.m4.1.1.3.2.cmml"><mi id="A1.3.p2.4.m4.1.1.3.2.2" xref="A1.3.p2.4.m4.1.1.3.2.2.cmml">ϕ</mi><mi id="A1.3.p2.4.m4.1.1.3.2.3" xref="A1.3.p2.4.m4.1.1.3.2.3.cmml">i</mi></msub><mo lspace="0em" rspace="0em" id="A1.3.p2.4.m4.1.1.3.1" xref="A1.3.p2.4.m4.1.1.3.1.cmml">​</mo><msup id="A1.3.p2.4.m4.1.1.3.3" xref="A1.3.p2.4.m4.1.1.3.3.cmml"><mn id="A1.3.p2.4.m4.1.1.3.3.2" xref="A1.3.p2.4.m4.1.1.3.3.2.cmml">2</mn><mi id="A1.3.p2.4.m4.1.1.3.3.3" xref="A1.3.p2.4.m4.1.1.3.3.3.cmml">q</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.3.p2.4.m4.1b"><apply id="A1.3.p2.4.m4.1.1.cmml" xref="A1.3.p2.4.m4.1.1"><eq id="A1.3.p2.4.m4.1.1.1.cmml" xref="A1.3.p2.4.m4.1.1.1"></eq><apply id="A1.3.p2.4.m4.1.1.2.cmml" xref="A1.3.p2.4.m4.1.1.2"><csymbol cd="ambiguous" id="A1.3.p2.4.m4.1.1.2.1.cmml" xref="A1.3.p2.4.m4.1.1.2">subscript</csymbol><ci id="A1.3.p2.4.m4.1.1.2.2.cmml" xref="A1.3.p2.4.m4.1.1.2.2">𝜓</ci><ci id="A1.3.p2.4.m4.1.1.2.3.cmml" xref="A1.3.p2.4.m4.1.1.2.3">𝑖</ci></apply><apply id="A1.3.p2.4.m4.1.1.3.cmml" xref="A1.3.p2.4.m4.1.1.3"><times id="A1.3.p2.4.m4.1.1.3.1.cmml" xref="A1.3.p2.4.m4.1.1.3.1"></times><apply id="A1.3.p2.4.m4.1.1.3.2.cmml" xref="A1.3.p2.4.m4.1.1.3.2"><csymbol cd="ambiguous" id="A1.3.p2.4.m4.1.1.3.2.1.cmml" xref="A1.3.p2.4.m4.1.1.3.2">subscript</csymbol><ci id="A1.3.p2.4.m4.1.1.3.2.2.cmml" xref="A1.3.p2.4.m4.1.1.3.2.2">italic-ϕ</ci><ci id="A1.3.p2.4.m4.1.1.3.2.3.cmml" xref="A1.3.p2.4.m4.1.1.3.2.3">𝑖</ci></apply><apply id="A1.3.p2.4.m4.1.1.3.3.cmml" xref="A1.3.p2.4.m4.1.1.3.3"><csymbol cd="ambiguous" id="A1.3.p2.4.m4.1.1.3.3.1.cmml" xref="A1.3.p2.4.m4.1.1.3.3">superscript</csymbol><cn type="integer" id="A1.3.p2.4.m4.1.1.3.3.2.cmml" xref="A1.3.p2.4.m4.1.1.3.3.2">2</cn><ci id="A1.3.p2.4.m4.1.1.3.3.3.cmml" xref="A1.3.p2.4.m4.1.1.3.3.3">𝑞</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.4.m4.1c">\psi_{i}=\phi_{i}2^{q}</annotation></semantics></math> by adding <math id="A1.3.p2.5.m5.1" class="ltx_Math" alttext="q" display="inline"><semantics id="A1.3.p2.5.m5.1a"><mi id="A1.3.p2.5.m5.1.1" xref="A1.3.p2.5.m5.1.1.cmml">q</mi><annotation-xml encoding="MathML-Content" id="A1.3.p2.5.m5.1b"><ci id="A1.3.p2.5.m5.1.1.cmml" xref="A1.3.p2.5.m5.1.1">𝑞</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.5.m5.1c">q</annotation></semantics></math> to the exponent of <math id="A1.3.p2.6.m6.1" class="ltx_Math" alttext="\phi_{i}" display="inline"><semantics id="A1.3.p2.6.m6.1a"><msub id="A1.3.p2.6.m6.1.1" xref="A1.3.p2.6.m6.1.1.cmml"><mi id="A1.3.p2.6.m6.1.1.2" xref="A1.3.p2.6.m6.1.1.2.cmml">ϕ</mi><mi id="A1.3.p2.6.m6.1.1.3" xref="A1.3.p2.6.m6.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.3.p2.6.m6.1b"><apply id="A1.3.p2.6.m6.1.1.cmml" xref="A1.3.p2.6.m6.1.1"><csymbol cd="ambiguous" id="A1.3.p2.6.m6.1.1.1.cmml" xref="A1.3.p2.6.m6.1.1">subscript</csymbol><ci id="A1.3.p2.6.m6.1.1.2.cmml" xref="A1.3.p2.6.m6.1.1.2">italic-ϕ</ci><ci id="A1.3.p2.6.m6.1.1.3.cmml" xref="A1.3.p2.6.m6.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.6.m6.1c">\phi_{i}</annotation></semantics></math>, using up to <math id="A1.3.p2.7.m7.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.3.p2.7.m7.1a"><mrow id="A1.3.p2.7.m7.1.1" xref="A1.3.p2.7.m7.1.1.cmml"><mi id="A1.3.p2.7.m7.1.1.2" xref="A1.3.p2.7.m7.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.3.p2.7.m7.1.1.1" xref="A1.3.p2.7.m7.1.1.1.cmml">​</mo><mrow id="A1.3.p2.7.m7.1.1.3" xref="A1.3.p2.7.m7.1.1.3.cmml"><mi id="A1.3.p2.7.m7.1.1.3.1" xref="A1.3.p2.7.m7.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.3.p2.7.m7.1.1.3a" xref="A1.3.p2.7.m7.1.1.3.cmml">⁡</mo><mi id="A1.3.p2.7.m7.1.1.3.2" xref="A1.3.p2.7.m7.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.3.p2.7.m7.1b"><apply id="A1.3.p2.7.m7.1.1.cmml" xref="A1.3.p2.7.m7.1.1"><times id="A1.3.p2.7.m7.1.1.1.cmml" xref="A1.3.p2.7.m7.1.1.1"></times><ci id="A1.3.p2.7.m7.1.1.2.cmml" xref="A1.3.p2.7.m7.1.1.2">𝑐</ci><apply id="A1.3.p2.7.m7.1.1.3.cmml" xref="A1.3.p2.7.m7.1.1.3"><log id="A1.3.p2.7.m7.1.1.3.1.cmml" xref="A1.3.p2.7.m7.1.1.3.1"></log><ci id="A1.3.p2.7.m7.1.1.3.2.cmml" xref="A1.3.p2.7.m7.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.7.m7.1c">c\log n</annotation></semantics></math> additional bits in the exponent if necessary to keep the computation exact. Note that <math id="A1.3.p2.8.m8.1" class="ltx_Math" alttext="e" display="inline"><semantics id="A1.3.p2.8.m8.1a"><mi id="A1.3.p2.8.m8.1.1" xref="A1.3.p2.8.m8.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="A1.3.p2.8.m8.1b"><ci id="A1.3.p2.8.m8.1.1.cmml" xref="A1.3.p2.8.m8.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.8.m8.1c">e</annotation></semantics></math>, <math id="A1.3.p2.9.m9.1" class="ltx_Math" alttext="q" display="inline"><semantics id="A1.3.p2.9.m9.1a"><mi id="A1.3.p2.9.m9.1.1" xref="A1.3.p2.9.m9.1.1.cmml">q</mi><annotation-xml encoding="MathML-Content" id="A1.3.p2.9.m9.1b"><ci id="A1.3.p2.9.m9.1.1.cmml" xref="A1.3.p2.9.m9.1.1">𝑞</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.9.m9.1c">q</annotation></semantics></math>, and all <math id="A1.3.p2.10.m10.1" class="ltx_Math" alttext="\psi_{i}" display="inline"><semantics id="A1.3.p2.10.m10.1a"><msub id="A1.3.p2.10.m10.1.1" xref="A1.3.p2.10.m10.1.1.cmml"><mi id="A1.3.p2.10.m10.1.1.2" xref="A1.3.p2.10.m10.1.1.2.cmml">ψ</mi><mi id="A1.3.p2.10.m10.1.1.3" xref="A1.3.p2.10.m10.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.3.p2.10.m10.1b"><apply id="A1.3.p2.10.m10.1.1.cmml" xref="A1.3.p2.10.m10.1.1"><csymbol cd="ambiguous" id="A1.3.p2.10.m10.1.1.1.cmml" xref="A1.3.p2.10.m10.1.1">subscript</csymbol><ci id="A1.3.p2.10.m10.1.1.2.cmml" xref="A1.3.p2.10.m10.1.1.2">𝜓</ci><ci id="A1.3.p2.10.m10.1.1.3.cmml" xref="A1.3.p2.10.m10.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.10.m10.1c">\psi_{i}</annotation></semantics></math> can easily be computed exactly by an <math id="A1.3.p2.11.m11.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.3.p2.11.m11.1a"><mi id="A1.3.p2.11.m11.1.1" xref="A1.3.p2.11.m11.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.3.p2.11.m11.1b"><ci id="A1.3.p2.11.m11.1.1.cmml" xref="A1.3.p2.11.m11.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.11.m11.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.3.p2.12.m12.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.3.p2.12.m12.1a"><msup id="A1.3.p2.12.m12.1.1" xref="A1.3.p2.12.m12.1.1.cmml"><mi id="A1.3.p2.12.m12.1.1.2" xref="A1.3.p2.12.m12.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.3.p2.12.m12.1.1.3" xref="A1.3.p2.12.m12.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.3.p2.12.m12.1b"><apply id="A1.3.p2.12.m12.1.1.cmml" xref="A1.3.p2.12.m12.1.1"><csymbol cd="ambiguous" id="A1.3.p2.12.m12.1.1.1.cmml" xref="A1.3.p2.12.m12.1.1">superscript</csymbol><ci id="A1.3.p2.12.m12.1.1.2.cmml" xref="A1.3.p2.12.m12.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.3.p2.12.m12.1.1.3.cmml" xref="A1.3.p2.12.m12.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.12.m12.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit as they involve fixed-arity arithmetic operations. Further, by construction, every <math id="A1.3.p2.13.m13.1" class="ltx_Math" alttext="\psi_{i}" display="inline"><semantics id="A1.3.p2.13.m13.1a"><msub id="A1.3.p2.13.m13.1.1" xref="A1.3.p2.13.m13.1.1.cmml"><mi id="A1.3.p2.13.m13.1.1.2" xref="A1.3.p2.13.m13.1.1.2.cmml">ψ</mi><mi id="A1.3.p2.13.m13.1.1.3" xref="A1.3.p2.13.m13.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.3.p2.13.m13.1b"><apply id="A1.3.p2.13.m13.1.1.cmml" xref="A1.3.p2.13.m13.1.1"><csymbol cd="ambiguous" id="A1.3.p2.13.m13.1.1.1.cmml" xref="A1.3.p2.13.m13.1.1">subscript</csymbol><ci id="A1.3.p2.13.m13.1.1.2.cmml" xref="A1.3.p2.13.m13.1.1.2">𝜓</ci><ci id="A1.3.p2.13.m13.1.1.3.cmml" xref="A1.3.p2.13.m13.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.3.p2.13.m13.1c">\psi_{i}</annotation></semantics></math> has a non-negative exponent and thus represents an integer.</p>
</div>
<div id="A1.4.p3" class="ltx_para">
<p id="A1.4.p3.17" class="ltx_p">The maximum number representable by each <math id="A1.4.p3.1.m1.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.4.p3.1.m1.1a"><mrow id="A1.4.p3.1.m1.1.1" xref="A1.4.p3.1.m1.1.1.cmml"><mi id="A1.4.p3.1.m1.1.1.2" xref="A1.4.p3.1.m1.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.4.p3.1.m1.1.1.1" xref="A1.4.p3.1.m1.1.1.1.cmml">​</mo><mrow id="A1.4.p3.1.m1.1.1.3" xref="A1.4.p3.1.m1.1.1.3.cmml"><mi id="A1.4.p3.1.m1.1.1.3.1" xref="A1.4.p3.1.m1.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.4.p3.1.m1.1.1.3a" xref="A1.4.p3.1.m1.1.1.3.cmml">⁡</mo><mi id="A1.4.p3.1.m1.1.1.3.2" xref="A1.4.p3.1.m1.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.4.p3.1.m1.1b"><apply id="A1.4.p3.1.m1.1.1.cmml" xref="A1.4.p3.1.m1.1.1"><times id="A1.4.p3.1.m1.1.1.1.cmml" xref="A1.4.p3.1.m1.1.1.1"></times><ci id="A1.4.p3.1.m1.1.1.2.cmml" xref="A1.4.p3.1.m1.1.1.2">𝑐</ci><apply id="A1.4.p3.1.m1.1.1.3.cmml" xref="A1.4.p3.1.m1.1.1.3"><log id="A1.4.p3.1.m1.1.1.3.1.cmml" xref="A1.4.p3.1.m1.1.1.3.1"></log><ci id="A1.4.p3.1.m1.1.1.3.2.cmml" xref="A1.4.p3.1.m1.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.1.m1.1c">c\log n</annotation></semantics></math> precision float <math id="A1.4.p3.2.m2.1" class="ltx_Math" alttext="\phi_{i}" display="inline"><semantics id="A1.4.p3.2.m2.1a"><msub id="A1.4.p3.2.m2.1.1" xref="A1.4.p3.2.m2.1.1.cmml"><mi id="A1.4.p3.2.m2.1.1.2" xref="A1.4.p3.2.m2.1.1.2.cmml">ϕ</mi><mi id="A1.4.p3.2.m2.1.1.3" xref="A1.4.p3.2.m2.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.4.p3.2.m2.1b"><apply id="A1.4.p3.2.m2.1.1.cmml" xref="A1.4.p3.2.m2.1.1"><csymbol cd="ambiguous" id="A1.4.p3.2.m2.1.1.1.cmml" xref="A1.4.p3.2.m2.1.1">subscript</csymbol><ci id="A1.4.p3.2.m2.1.1.2.cmml" xref="A1.4.p3.2.m2.1.1.2">italic-ϕ</ci><ci id="A1.4.p3.2.m2.1.1.3.cmml" xref="A1.4.p3.2.m2.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.2.m2.1c">\phi_{i}</annotation></semantics></math> is upper bounded by <math id="A1.4.p3.3.m3.1" class="ltx_Math" alttext="2^{n^{c}}" display="inline"><semantics id="A1.4.p3.3.m3.1a"><msup id="A1.4.p3.3.m3.1.1" xref="A1.4.p3.3.m3.1.1.cmml"><mn id="A1.4.p3.3.m3.1.1.2" xref="A1.4.p3.3.m3.1.1.2.cmml">2</mn><msup id="A1.4.p3.3.m3.1.1.3" xref="A1.4.p3.3.m3.1.1.3.cmml"><mi id="A1.4.p3.3.m3.1.1.3.2" xref="A1.4.p3.3.m3.1.1.3.2.cmml">n</mi><mi id="A1.4.p3.3.m3.1.1.3.3" xref="A1.4.p3.3.m3.1.1.3.3.cmml">c</mi></msup></msup><annotation-xml encoding="MathML-Content" id="A1.4.p3.3.m3.1b"><apply id="A1.4.p3.3.m3.1.1.cmml" xref="A1.4.p3.3.m3.1.1"><csymbol cd="ambiguous" id="A1.4.p3.3.m3.1.1.1.cmml" xref="A1.4.p3.3.m3.1.1">superscript</csymbol><cn type="integer" id="A1.4.p3.3.m3.1.1.2.cmml" xref="A1.4.p3.3.m3.1.1.2">2</cn><apply id="A1.4.p3.3.m3.1.1.3.cmml" xref="A1.4.p3.3.m3.1.1.3"><csymbol cd="ambiguous" id="A1.4.p3.3.m3.1.1.3.1.cmml" xref="A1.4.p3.3.m3.1.1.3">superscript</csymbol><ci id="A1.4.p3.3.m3.1.1.3.2.cmml" xref="A1.4.p3.3.m3.1.1.3.2">𝑛</ci><ci id="A1.4.p3.3.m3.1.1.3.3.cmml" xref="A1.4.p3.3.m3.1.1.3.3">𝑐</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.3.m3.1c">2^{n^{c}}</annotation></semantics></math>. Thus, the maximum number representable by each entry <math id="A1.4.p3.4.m4.1" class="ltx_Math" alttext="\psi_{i}" display="inline"><semantics id="A1.4.p3.4.m4.1a"><msub id="A1.4.p3.4.m4.1.1" xref="A1.4.p3.4.m4.1.1.cmml"><mi id="A1.4.p3.4.m4.1.1.2" xref="A1.4.p3.4.m4.1.1.2.cmml">ψ</mi><mi id="A1.4.p3.4.m4.1.1.3" xref="A1.4.p3.4.m4.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.4.p3.4.m4.1b"><apply id="A1.4.p3.4.m4.1.1.cmml" xref="A1.4.p3.4.m4.1.1"><csymbol cd="ambiguous" id="A1.4.p3.4.m4.1.1.1.cmml" xref="A1.4.p3.4.m4.1.1">subscript</csymbol><ci id="A1.4.p3.4.m4.1.1.2.cmml" xref="A1.4.p3.4.m4.1.1.2">𝜓</ci><ci id="A1.4.p3.4.m4.1.1.3.cmml" xref="A1.4.p3.4.m4.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.4.m4.1c">\psi_{i}</annotation></semantics></math> is <math id="A1.4.p3.5.m5.1" class="ltx_Math" alttext="2^{n^{c}}\times 2^{q}=2^{n^{c}+q}" display="inline"><semantics id="A1.4.p3.5.m5.1a"><mrow id="A1.4.p3.5.m5.1.1" xref="A1.4.p3.5.m5.1.1.cmml"><mrow id="A1.4.p3.5.m5.1.1.2" xref="A1.4.p3.5.m5.1.1.2.cmml"><msup id="A1.4.p3.5.m5.1.1.2.2" xref="A1.4.p3.5.m5.1.1.2.2.cmml"><mn id="A1.4.p3.5.m5.1.1.2.2.2" xref="A1.4.p3.5.m5.1.1.2.2.2.cmml">2</mn><msup id="A1.4.p3.5.m5.1.1.2.2.3" xref="A1.4.p3.5.m5.1.1.2.2.3.cmml"><mi id="A1.4.p3.5.m5.1.1.2.2.3.2" xref="A1.4.p3.5.m5.1.1.2.2.3.2.cmml">n</mi><mi id="A1.4.p3.5.m5.1.1.2.2.3.3" xref="A1.4.p3.5.m5.1.1.2.2.3.3.cmml">c</mi></msup></msup><mo lspace="0.222em" rspace="0.222em" id="A1.4.p3.5.m5.1.1.2.1" xref="A1.4.p3.5.m5.1.1.2.1.cmml">×</mo><msup id="A1.4.p3.5.m5.1.1.2.3" xref="A1.4.p3.5.m5.1.1.2.3.cmml"><mn id="A1.4.p3.5.m5.1.1.2.3.2" xref="A1.4.p3.5.m5.1.1.2.3.2.cmml">2</mn><mi id="A1.4.p3.5.m5.1.1.2.3.3" xref="A1.4.p3.5.m5.1.1.2.3.3.cmml">q</mi></msup></mrow><mo id="A1.4.p3.5.m5.1.1.1" xref="A1.4.p3.5.m5.1.1.1.cmml">=</mo><msup id="A1.4.p3.5.m5.1.1.3" xref="A1.4.p3.5.m5.1.1.3.cmml"><mn id="A1.4.p3.5.m5.1.1.3.2" xref="A1.4.p3.5.m5.1.1.3.2.cmml">2</mn><mrow id="A1.4.p3.5.m5.1.1.3.3" xref="A1.4.p3.5.m5.1.1.3.3.cmml"><msup id="A1.4.p3.5.m5.1.1.3.3.2" xref="A1.4.p3.5.m5.1.1.3.3.2.cmml"><mi id="A1.4.p3.5.m5.1.1.3.3.2.2" xref="A1.4.p3.5.m5.1.1.3.3.2.2.cmml">n</mi><mi id="A1.4.p3.5.m5.1.1.3.3.2.3" xref="A1.4.p3.5.m5.1.1.3.3.2.3.cmml">c</mi></msup><mo id="A1.4.p3.5.m5.1.1.3.3.1" xref="A1.4.p3.5.m5.1.1.3.3.1.cmml">+</mo><mi id="A1.4.p3.5.m5.1.1.3.3.3" xref="A1.4.p3.5.m5.1.1.3.3.3.cmml">q</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="A1.4.p3.5.m5.1b"><apply id="A1.4.p3.5.m5.1.1.cmml" xref="A1.4.p3.5.m5.1.1"><eq id="A1.4.p3.5.m5.1.1.1.cmml" xref="A1.4.p3.5.m5.1.1.1"></eq><apply id="A1.4.p3.5.m5.1.1.2.cmml" xref="A1.4.p3.5.m5.1.1.2"><times id="A1.4.p3.5.m5.1.1.2.1.cmml" xref="A1.4.p3.5.m5.1.1.2.1"></times><apply id="A1.4.p3.5.m5.1.1.2.2.cmml" xref="A1.4.p3.5.m5.1.1.2.2"><csymbol cd="ambiguous" id="A1.4.p3.5.m5.1.1.2.2.1.cmml" xref="A1.4.p3.5.m5.1.1.2.2">superscript</csymbol><cn type="integer" id="A1.4.p3.5.m5.1.1.2.2.2.cmml" xref="A1.4.p3.5.m5.1.1.2.2.2">2</cn><apply id="A1.4.p3.5.m5.1.1.2.2.3.cmml" xref="A1.4.p3.5.m5.1.1.2.2.3"><csymbol cd="ambiguous" id="A1.4.p3.5.m5.1.1.2.2.3.1.cmml" xref="A1.4.p3.5.m5.1.1.2.2.3">superscript</csymbol><ci id="A1.4.p3.5.m5.1.1.2.2.3.2.cmml" xref="A1.4.p3.5.m5.1.1.2.2.3.2">𝑛</ci><ci id="A1.4.p3.5.m5.1.1.2.2.3.3.cmml" xref="A1.4.p3.5.m5.1.1.2.2.3.3">𝑐</ci></apply></apply><apply id="A1.4.p3.5.m5.1.1.2.3.cmml" xref="A1.4.p3.5.m5.1.1.2.3"><csymbol cd="ambiguous" id="A1.4.p3.5.m5.1.1.2.3.1.cmml" xref="A1.4.p3.5.m5.1.1.2.3">superscript</csymbol><cn type="integer" id="A1.4.p3.5.m5.1.1.2.3.2.cmml" xref="A1.4.p3.5.m5.1.1.2.3.2">2</cn><ci id="A1.4.p3.5.m5.1.1.2.3.3.cmml" xref="A1.4.p3.5.m5.1.1.2.3.3">𝑞</ci></apply></apply><apply id="A1.4.p3.5.m5.1.1.3.cmml" xref="A1.4.p3.5.m5.1.1.3"><csymbol cd="ambiguous" id="A1.4.p3.5.m5.1.1.3.1.cmml" xref="A1.4.p3.5.m5.1.1.3">superscript</csymbol><cn type="integer" id="A1.4.p3.5.m5.1.1.3.2.cmml" xref="A1.4.p3.5.m5.1.1.3.2">2</cn><apply id="A1.4.p3.5.m5.1.1.3.3.cmml" xref="A1.4.p3.5.m5.1.1.3.3"><plus id="A1.4.p3.5.m5.1.1.3.3.1.cmml" xref="A1.4.p3.5.m5.1.1.3.3.1"></plus><apply id="A1.4.p3.5.m5.1.1.3.3.2.cmml" xref="A1.4.p3.5.m5.1.1.3.3.2"><csymbol cd="ambiguous" id="A1.4.p3.5.m5.1.1.3.3.2.1.cmml" xref="A1.4.p3.5.m5.1.1.3.3.2">superscript</csymbol><ci id="A1.4.p3.5.m5.1.1.3.3.2.2.cmml" xref="A1.4.p3.5.m5.1.1.3.3.2.2">𝑛</ci><ci id="A1.4.p3.5.m5.1.1.3.3.2.3.cmml" xref="A1.4.p3.5.m5.1.1.3.3.2.3">𝑐</ci></apply><ci id="A1.4.p3.5.m5.1.1.3.3.3.cmml" xref="A1.4.p3.5.m5.1.1.3.3.3">𝑞</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.5.m5.1c">2^{n^{c}}\times 2^{q}=2^{n^{c}+q}</annotation></semantics></math>. Let <math id="A1.4.p3.6.m6.1" class="ltx_Math" alttext="m=n^{c}+q" display="inline"><semantics id="A1.4.p3.6.m6.1a"><mrow id="A1.4.p3.6.m6.1.1" xref="A1.4.p3.6.m6.1.1.cmml"><mi id="A1.4.p3.6.m6.1.1.2" xref="A1.4.p3.6.m6.1.1.2.cmml">m</mi><mo id="A1.4.p3.6.m6.1.1.1" xref="A1.4.p3.6.m6.1.1.1.cmml">=</mo><mrow id="A1.4.p3.6.m6.1.1.3" xref="A1.4.p3.6.m6.1.1.3.cmml"><msup id="A1.4.p3.6.m6.1.1.3.2" xref="A1.4.p3.6.m6.1.1.3.2.cmml"><mi id="A1.4.p3.6.m6.1.1.3.2.2" xref="A1.4.p3.6.m6.1.1.3.2.2.cmml">n</mi><mi id="A1.4.p3.6.m6.1.1.3.2.3" xref="A1.4.p3.6.m6.1.1.3.2.3.cmml">c</mi></msup><mo id="A1.4.p3.6.m6.1.1.3.1" xref="A1.4.p3.6.m6.1.1.3.1.cmml">+</mo><mi id="A1.4.p3.6.m6.1.1.3.3" xref="A1.4.p3.6.m6.1.1.3.3.cmml">q</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.4.p3.6.m6.1b"><apply id="A1.4.p3.6.m6.1.1.cmml" xref="A1.4.p3.6.m6.1.1"><eq id="A1.4.p3.6.m6.1.1.1.cmml" xref="A1.4.p3.6.m6.1.1.1"></eq><ci id="A1.4.p3.6.m6.1.1.2.cmml" xref="A1.4.p3.6.m6.1.1.2">𝑚</ci><apply id="A1.4.p3.6.m6.1.1.3.cmml" xref="A1.4.p3.6.m6.1.1.3"><plus id="A1.4.p3.6.m6.1.1.3.1.cmml" xref="A1.4.p3.6.m6.1.1.3.1"></plus><apply id="A1.4.p3.6.m6.1.1.3.2.cmml" xref="A1.4.p3.6.m6.1.1.3.2"><csymbol cd="ambiguous" id="A1.4.p3.6.m6.1.1.3.2.1.cmml" xref="A1.4.p3.6.m6.1.1.3.2">superscript</csymbol><ci id="A1.4.p3.6.m6.1.1.3.2.2.cmml" xref="A1.4.p3.6.m6.1.1.3.2.2">𝑛</ci><ci id="A1.4.p3.6.m6.1.1.3.2.3.cmml" xref="A1.4.p3.6.m6.1.1.3.2.3">𝑐</ci></apply><ci id="A1.4.p3.6.m6.1.1.3.3.cmml" xref="A1.4.p3.6.m6.1.1.3.3">𝑞</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.6.m6.1c">m=n^{c}+q</annotation></semantics></math>. It follows that each <math id="A1.4.p3.7.m7.1" class="ltx_Math" alttext="\psi_{i}" display="inline"><semantics id="A1.4.p3.7.m7.1a"><msub id="A1.4.p3.7.m7.1.1" xref="A1.4.p3.7.m7.1.1.cmml"><mi id="A1.4.p3.7.m7.1.1.2" xref="A1.4.p3.7.m7.1.1.2.cmml">ψ</mi><mi id="A1.4.p3.7.m7.1.1.3" xref="A1.4.p3.7.m7.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.4.p3.7.m7.1b"><apply id="A1.4.p3.7.m7.1.1.cmml" xref="A1.4.p3.7.m7.1.1"><csymbol cd="ambiguous" id="A1.4.p3.7.m7.1.1.1.cmml" xref="A1.4.p3.7.m7.1.1">subscript</csymbol><ci id="A1.4.p3.7.m7.1.1.2.cmml" xref="A1.4.p3.7.m7.1.1.2">𝜓</ci><ci id="A1.4.p3.7.m7.1.1.3.cmml" xref="A1.4.p3.7.m7.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.7.m7.1c">\psi_{i}</annotation></semantics></math> can be equivalently represented as an <math id="A1.4.p3.8.m8.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.4.p3.8.m8.1a"><mi id="A1.4.p3.8.m8.1.1" xref="A1.4.p3.8.m8.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.4.p3.8.m8.1b"><ci id="A1.4.p3.8.m8.1.1.cmml" xref="A1.4.p3.8.m8.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.8.m8.1c">m</annotation></semantics></math>-bit integer. Further, this integer can be computed by left-shifting the mantissa of <math id="A1.4.p3.9.m9.1" class="ltx_Math" alttext="\psi_{i}" display="inline"><semantics id="A1.4.p3.9.m9.1a"><msub id="A1.4.p3.9.m9.1.1" xref="A1.4.p3.9.m9.1.1.cmml"><mi id="A1.4.p3.9.m9.1.1.2" xref="A1.4.p3.9.m9.1.1.2.cmml">ψ</mi><mi id="A1.4.p3.9.m9.1.1.3" xref="A1.4.p3.9.m9.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.4.p3.9.m9.1b"><apply id="A1.4.p3.9.m9.1.1.cmml" xref="A1.4.p3.9.m9.1.1"><csymbol cd="ambiguous" id="A1.4.p3.9.m9.1.1.1.cmml" xref="A1.4.p3.9.m9.1.1">subscript</csymbol><ci id="A1.4.p3.9.m9.1.1.2.cmml" xref="A1.4.p3.9.m9.1.1.2">𝜓</ci><ci id="A1.4.p3.9.m9.1.1.3.cmml" xref="A1.4.p3.9.m9.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.9.m9.1c">\psi_{i}</annotation></semantics></math> by a number of bits equal to the value of the exponent of <math id="A1.4.p3.10.m10.1" class="ltx_Math" alttext="\psi_{i}" display="inline"><semantics id="A1.4.p3.10.m10.1a"><msub id="A1.4.p3.10.m10.1.1" xref="A1.4.p3.10.m10.1.1.cmml"><mi id="A1.4.p3.10.m10.1.1.2" xref="A1.4.p3.10.m10.1.1.2.cmml">ψ</mi><mi id="A1.4.p3.10.m10.1.1.3" xref="A1.4.p3.10.m10.1.1.3.cmml">i</mi></msub><annotation-xml encoding="MathML-Content" id="A1.4.p3.10.m10.1b"><apply id="A1.4.p3.10.m10.1.1.cmml" xref="A1.4.p3.10.m10.1.1"><csymbol cd="ambiguous" id="A1.4.p3.10.m10.1.1.1.cmml" xref="A1.4.p3.10.m10.1.1">subscript</csymbol><ci id="A1.4.p3.10.m10.1.1.2.cmml" xref="A1.4.p3.10.m10.1.1.2">𝜓</ci><ci id="A1.4.p3.10.m10.1.1.3.cmml" xref="A1.4.p3.10.m10.1.1.3">𝑖</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.10.m10.1c">\psi_{i}</annotation></semantics></math> (which is non-negative). Finally, this left-shift, and thus the <math id="A1.4.p3.11.m11.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}" display="inline"><semantics id="A1.4.p3.11.m11.1a"><msub id="A1.4.p3.11.m11.1.1" xref="A1.4.p3.11.m11.1.1.cmml"><mi id="A1.4.p3.11.m11.1.1.2" xref="A1.4.p3.11.m11.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.4.p3.11.m11.1.1.3" xref="A1.4.p3.11.m11.1.1.3.cmml">ℝ</mi></msub><annotation-xml encoding="MathML-Content" id="A1.4.p3.11.m11.1b"><apply id="A1.4.p3.11.m11.1.1.cmml" xref="A1.4.p3.11.m11.1.1"><csymbol cd="ambiguous" id="A1.4.p3.11.m11.1.1.1.cmml" xref="A1.4.p3.11.m11.1.1">subscript</csymbol><ci id="A1.4.p3.11.m11.1.1.2.cmml" xref="A1.4.p3.11.m11.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.4.p3.11.m11.1.1.3.cmml" xref="A1.4.p3.11.m11.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.11.m11.1c">\mathsf{cast}_{\mathbb{R}}</annotation></semantics></math> operation over <math id="A1.4.p3.12.m12.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.4.p3.12.m12.1a"><mi id="A1.4.p3.12.m12.1.1" xref="A1.4.p3.12.m12.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.4.p3.12.m12.1b"><ci id="A1.4.p3.12.m12.1.1.cmml" xref="A1.4.p3.12.m12.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.12.m12.1c">m</annotation></semantics></math>-precision floats, can be easily computed by an <math id="A1.4.p3.13.m13.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.4.p3.13.m13.1a"><mi id="A1.4.p3.13.m13.1.1" xref="A1.4.p3.13.m13.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.4.p3.13.m13.1b"><ci id="A1.4.p3.13.m13.1.1.cmml" xref="A1.4.p3.13.m13.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.13.m13.1c">\mathsf{L}</annotation></semantics></math>-uniform threshold circuit of size <math id="A1.4.p3.14.m14.1" class="ltx_Math" alttext="\mathrm{poly}(m)" display="inline"><semantics id="A1.4.p3.14.m14.1a"><mrow id="A1.4.p3.14.m14.1.2" xref="A1.4.p3.14.m14.1.2.cmml"><mi id="A1.4.p3.14.m14.1.2.2" xref="A1.4.p3.14.m14.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.4.p3.14.m14.1.2.1" xref="A1.4.p3.14.m14.1.2.1.cmml">​</mo><mrow id="A1.4.p3.14.m14.1.2.3.2" xref="A1.4.p3.14.m14.1.2.cmml"><mo stretchy="false" id="A1.4.p3.14.m14.1.2.3.2.1" xref="A1.4.p3.14.m14.1.2.cmml">(</mo><mi id="A1.4.p3.14.m14.1.1" xref="A1.4.p3.14.m14.1.1.cmml">m</mi><mo stretchy="false" id="A1.4.p3.14.m14.1.2.3.2.2" xref="A1.4.p3.14.m14.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.4.p3.14.m14.1b"><apply id="A1.4.p3.14.m14.1.2.cmml" xref="A1.4.p3.14.m14.1.2"><times id="A1.4.p3.14.m14.1.2.1.cmml" xref="A1.4.p3.14.m14.1.2.1"></times><ci id="A1.4.p3.14.m14.1.2.2.cmml" xref="A1.4.p3.14.m14.1.2.2">poly</ci><ci id="A1.4.p3.14.m14.1.1.cmml" xref="A1.4.p3.14.m14.1.1">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.14.m14.1c">\mathrm{poly}(m)</annotation></semantics></math>. In the other direction, casting from reals to <math id="A1.4.p3.15.m15.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.4.p3.15.m15.1a"><mi id="A1.4.p3.15.m15.1.1" xref="A1.4.p3.15.m15.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.4.p3.15.m15.1b"><ci id="A1.4.p3.15.m15.1.1.cmml" xref="A1.4.p3.15.m15.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.15.m15.1c">m</annotation></semantics></math>-precision floats can also be easily accomplished by an <math id="A1.4.p3.16.m16.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.4.p3.16.m16.1a"><mi id="A1.4.p3.16.m16.1.1" xref="A1.4.p3.16.m16.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.4.p3.16.m16.1b"><ci id="A1.4.p3.16.m16.1.1.cmml" xref="A1.4.p3.16.m16.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.16.m16.1c">\mathsf{L}</annotation></semantics></math>-uniform threshold circuit of size <math id="A1.4.p3.17.m17.1" class="ltx_Math" alttext="\mathrm{poly}(m)" display="inline"><semantics id="A1.4.p3.17.m17.1a"><mrow id="A1.4.p3.17.m17.1.2" xref="A1.4.p3.17.m17.1.2.cmml"><mi id="A1.4.p3.17.m17.1.2.2" xref="A1.4.p3.17.m17.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.4.p3.17.m17.1.2.1" xref="A1.4.p3.17.m17.1.2.1.cmml">​</mo><mrow id="A1.4.p3.17.m17.1.2.3.2" xref="A1.4.p3.17.m17.1.2.cmml"><mo stretchy="false" id="A1.4.p3.17.m17.1.2.3.2.1" xref="A1.4.p3.17.m17.1.2.cmml">(</mo><mi id="A1.4.p3.17.m17.1.1" xref="A1.4.p3.17.m17.1.1.cmml">m</mi><mo stretchy="false" id="A1.4.p3.17.m17.1.2.3.2.2" xref="A1.4.p3.17.m17.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.4.p3.17.m17.1b"><apply id="A1.4.p3.17.m17.1.2.cmml" xref="A1.4.p3.17.m17.1.2"><times id="A1.4.p3.17.m17.1.2.1.cmml" xref="A1.4.p3.17.m17.1.2.1"></times><ci id="A1.4.p3.17.m17.1.2.2.cmml" xref="A1.4.p3.17.m17.1.2.2">poly</ci><ci id="A1.4.p3.17.m17.1.1.cmml" xref="A1.4.p3.17.m17.1.1">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.4.p3.17.m17.1c">\mathrm{poly}(m)</annotation></semantics></math>.</p>
</div>
<div id="A1.5.p4" class="ltx_para">
<p id="A1.5.p4.8" class="ltx_p">Observing that <math id="A1.5.p4.1.m1.3" class="ltx_Math" alttext="\psi_{1},\ldots,\psi_{z}" display="inline"><semantics id="A1.5.p4.1.m1.3a"><mrow id="A1.5.p4.1.m1.3.3.2" xref="A1.5.p4.1.m1.3.3.3.cmml"><msub id="A1.5.p4.1.m1.2.2.1.1" xref="A1.5.p4.1.m1.2.2.1.1.cmml"><mi id="A1.5.p4.1.m1.2.2.1.1.2" xref="A1.5.p4.1.m1.2.2.1.1.2.cmml">ψ</mi><mn id="A1.5.p4.1.m1.2.2.1.1.3" xref="A1.5.p4.1.m1.2.2.1.1.3.cmml">1</mn></msub><mo id="A1.5.p4.1.m1.3.3.2.3" xref="A1.5.p4.1.m1.3.3.3.cmml">,</mo><mi mathvariant="normal" id="A1.5.p4.1.m1.1.1" xref="A1.5.p4.1.m1.1.1.cmml">…</mi><mo id="A1.5.p4.1.m1.3.3.2.4" xref="A1.5.p4.1.m1.3.3.3.cmml">,</mo><msub id="A1.5.p4.1.m1.3.3.2.2" xref="A1.5.p4.1.m1.3.3.2.2.cmml"><mi id="A1.5.p4.1.m1.3.3.2.2.2" xref="A1.5.p4.1.m1.3.3.2.2.2.cmml">ψ</mi><mi id="A1.5.p4.1.m1.3.3.2.2.3" xref="A1.5.p4.1.m1.3.3.2.2.3.cmml">z</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="A1.5.p4.1.m1.3b"><list id="A1.5.p4.1.m1.3.3.3.cmml" xref="A1.5.p4.1.m1.3.3.2"><apply id="A1.5.p4.1.m1.2.2.1.1.cmml" xref="A1.5.p4.1.m1.2.2.1.1"><csymbol cd="ambiguous" id="A1.5.p4.1.m1.2.2.1.1.1.cmml" xref="A1.5.p4.1.m1.2.2.1.1">subscript</csymbol><ci id="A1.5.p4.1.m1.2.2.1.1.2.cmml" xref="A1.5.p4.1.m1.2.2.1.1.2">𝜓</ci><cn type="integer" id="A1.5.p4.1.m1.2.2.1.1.3.cmml" xref="A1.5.p4.1.m1.2.2.1.1.3">1</cn></apply><ci id="A1.5.p4.1.m1.1.1.cmml" xref="A1.5.p4.1.m1.1.1">…</ci><apply id="A1.5.p4.1.m1.3.3.2.2.cmml" xref="A1.5.p4.1.m1.3.3.2.2"><csymbol cd="ambiguous" id="A1.5.p4.1.m1.3.3.2.2.1.cmml" xref="A1.5.p4.1.m1.3.3.2.2">subscript</csymbol><ci id="A1.5.p4.1.m1.3.3.2.2.2.cmml" xref="A1.5.p4.1.m1.3.3.2.2.2">𝜓</ci><ci id="A1.5.p4.1.m1.3.3.2.2.3.cmml" xref="A1.5.p4.1.m1.3.3.2.2.3">𝑧</ci></apply></list></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.1.m1.3c">\psi_{1},\ldots,\psi_{z}</annotation></semantics></math> is a sequence of
floats each representable as an <math id="A1.5.p4.2.m2.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.5.p4.2.m2.1a"><mi id="A1.5.p4.2.m2.1.1" xref="A1.5.p4.2.m2.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.5.p4.2.m2.1b"><ci id="A1.5.p4.2.m2.1.1.cmml" xref="A1.5.p4.2.m2.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.2.m2.1c">m</annotation></semantics></math>-bit integer,
we now apply <a href="#A1.Thmtheorem2" title="Lemma A.2 (Iterated 𝔻-product). ‣ Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">A.2</span></a> with <math id="A1.5.p4.3.m3.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.5.p4.3.m3.1a"><mi id="A1.5.p4.3.m3.1.1" xref="A1.5.p4.3.m3.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.5.p4.3.m3.1b"><ci id="A1.5.p4.3.m3.1.1.cmml" xref="A1.5.p4.3.m3.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.3.m3.1c">\mathbb{D}</annotation></semantics></math> being ‘float’ to conclude that iterated float product <math id="A1.5.p4.4.m4.1" class="ltx_Math" alttext="\tau=\bigotimes_{i=1}^{z}\psi_{i}" display="inline"><semantics id="A1.5.p4.4.m4.1a"><mrow id="A1.5.p4.4.m4.1.1" xref="A1.5.p4.4.m4.1.1.cmml"><mi id="A1.5.p4.4.m4.1.1.2" xref="A1.5.p4.4.m4.1.1.2.cmml">τ</mi><mo rspace="0.111em" id="A1.5.p4.4.m4.1.1.1" xref="A1.5.p4.4.m4.1.1.1.cmml">=</mo><mrow id="A1.5.p4.4.m4.1.1.3" xref="A1.5.p4.4.m4.1.1.3.cmml"><msubsup id="A1.5.p4.4.m4.1.1.3.1" xref="A1.5.p4.4.m4.1.1.3.1.cmml"><mo id="A1.5.p4.4.m4.1.1.3.1.2.2" xref="A1.5.p4.4.m4.1.1.3.1.2.2.cmml">⨂</mo><mrow id="A1.5.p4.4.m4.1.1.3.1.2.3" xref="A1.5.p4.4.m4.1.1.3.1.2.3.cmml"><mi id="A1.5.p4.4.m4.1.1.3.1.2.3.2" xref="A1.5.p4.4.m4.1.1.3.1.2.3.2.cmml">i</mi><mo id="A1.5.p4.4.m4.1.1.3.1.2.3.1" xref="A1.5.p4.4.m4.1.1.3.1.2.3.1.cmml">=</mo><mn id="A1.5.p4.4.m4.1.1.3.1.2.3.3" xref="A1.5.p4.4.m4.1.1.3.1.2.3.3.cmml">1</mn></mrow><mi id="A1.5.p4.4.m4.1.1.3.1.3" xref="A1.5.p4.4.m4.1.1.3.1.3.cmml">z</mi></msubsup><msub id="A1.5.p4.4.m4.1.1.3.2" xref="A1.5.p4.4.m4.1.1.3.2.cmml"><mi id="A1.5.p4.4.m4.1.1.3.2.2" xref="A1.5.p4.4.m4.1.1.3.2.2.cmml">ψ</mi><mi id="A1.5.p4.4.m4.1.1.3.2.3" xref="A1.5.p4.4.m4.1.1.3.2.3.cmml">i</mi></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.5.p4.4.m4.1b"><apply id="A1.5.p4.4.m4.1.1.cmml" xref="A1.5.p4.4.m4.1.1"><eq id="A1.5.p4.4.m4.1.1.1.cmml" xref="A1.5.p4.4.m4.1.1.1"></eq><ci id="A1.5.p4.4.m4.1.1.2.cmml" xref="A1.5.p4.4.m4.1.1.2">𝜏</ci><apply id="A1.5.p4.4.m4.1.1.3.cmml" xref="A1.5.p4.4.m4.1.1.3"><apply id="A1.5.p4.4.m4.1.1.3.1.cmml" xref="A1.5.p4.4.m4.1.1.3.1"><csymbol cd="ambiguous" id="A1.5.p4.4.m4.1.1.3.1.1.cmml" xref="A1.5.p4.4.m4.1.1.3.1">superscript</csymbol><apply id="A1.5.p4.4.m4.1.1.3.1.2.cmml" xref="A1.5.p4.4.m4.1.1.3.1"><csymbol cd="ambiguous" id="A1.5.p4.4.m4.1.1.3.1.2.1.cmml" xref="A1.5.p4.4.m4.1.1.3.1">subscript</csymbol><csymbol cd="latexml" id="A1.5.p4.4.m4.1.1.3.1.2.2.cmml" xref="A1.5.p4.4.m4.1.1.3.1.2.2">tensor-product</csymbol><apply id="A1.5.p4.4.m4.1.1.3.1.2.3.cmml" xref="A1.5.p4.4.m4.1.1.3.1.2.3"><eq id="A1.5.p4.4.m4.1.1.3.1.2.3.1.cmml" xref="A1.5.p4.4.m4.1.1.3.1.2.3.1"></eq><ci id="A1.5.p4.4.m4.1.1.3.1.2.3.2.cmml" xref="A1.5.p4.4.m4.1.1.3.1.2.3.2">𝑖</ci><cn type="integer" id="A1.5.p4.4.m4.1.1.3.1.2.3.3.cmml" xref="A1.5.p4.4.m4.1.1.3.1.2.3.3">1</cn></apply></apply><ci id="A1.5.p4.4.m4.1.1.3.1.3.cmml" xref="A1.5.p4.4.m4.1.1.3.1.3">𝑧</ci></apply><apply id="A1.5.p4.4.m4.1.1.3.2.cmml" xref="A1.5.p4.4.m4.1.1.3.2"><csymbol cd="ambiguous" id="A1.5.p4.4.m4.1.1.3.2.1.cmml" xref="A1.5.p4.4.m4.1.1.3.2">subscript</csymbol><ci id="A1.5.p4.4.m4.1.1.3.2.2.cmml" xref="A1.5.p4.4.m4.1.1.3.2.2">𝜓</ci><ci id="A1.5.p4.4.m4.1.1.3.2.3.cmml" xref="A1.5.p4.4.m4.1.1.3.2.3">𝑖</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.4.m4.1c">\tau=\bigotimes_{i=1}^{z}\psi_{i}</annotation></semantics></math> can be computed by an <math id="A1.5.p4.5.m5.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.5.p4.5.m5.1a"><mi id="A1.5.p4.5.m5.1.1" xref="A1.5.p4.5.m5.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.5.p4.5.m5.1b"><ci id="A1.5.p4.5.m5.1.1.cmml" xref="A1.5.p4.5.m5.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.5.m5.1c">\mathsf{L}</annotation></semantics></math>-uniform threshold circuit of size <math id="A1.5.p4.6.m6.1" class="ltx_Math" alttext="\mathrm{poly}(m)" display="inline"><semantics id="A1.5.p4.6.m6.1a"><mrow id="A1.5.p4.6.m6.1.2" xref="A1.5.p4.6.m6.1.2.cmml"><mi id="A1.5.p4.6.m6.1.2.2" xref="A1.5.p4.6.m6.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.5.p4.6.m6.1.2.1" xref="A1.5.p4.6.m6.1.2.1.cmml">​</mo><mrow id="A1.5.p4.6.m6.1.2.3.2" xref="A1.5.p4.6.m6.1.2.cmml"><mo stretchy="false" id="A1.5.p4.6.m6.1.2.3.2.1" xref="A1.5.p4.6.m6.1.2.cmml">(</mo><mi id="A1.5.p4.6.m6.1.1" xref="A1.5.p4.6.m6.1.1.cmml">m</mi><mo stretchy="false" id="A1.5.p4.6.m6.1.2.3.2.2" xref="A1.5.p4.6.m6.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.5.p4.6.m6.1b"><apply id="A1.5.p4.6.m6.1.2.cmml" xref="A1.5.p4.6.m6.1.2"><times id="A1.5.p4.6.m6.1.2.1.cmml" xref="A1.5.p4.6.m6.1.2.1"></times><ci id="A1.5.p4.6.m6.1.2.2.cmml" xref="A1.5.p4.6.m6.1.2.2">poly</ci><ci id="A1.5.p4.6.m6.1.1.cmml" xref="A1.5.p4.6.m6.1.1">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.6.m6.1c">\mathrm{poly}(m)</annotation></semantics></math>. Since
<math id="A1.5.p4.7.m7.1" class="ltx_Math" alttext="m\leq 2n^{c}" display="inline"><semantics id="A1.5.p4.7.m7.1a"><mrow id="A1.5.p4.7.m7.1.1" xref="A1.5.p4.7.m7.1.1.cmml"><mi id="A1.5.p4.7.m7.1.1.2" xref="A1.5.p4.7.m7.1.1.2.cmml">m</mi><mo id="A1.5.p4.7.m7.1.1.1" xref="A1.5.p4.7.m7.1.1.1.cmml">≤</mo><mrow id="A1.5.p4.7.m7.1.1.3" xref="A1.5.p4.7.m7.1.1.3.cmml"><mn id="A1.5.p4.7.m7.1.1.3.2" xref="A1.5.p4.7.m7.1.1.3.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A1.5.p4.7.m7.1.1.3.1" xref="A1.5.p4.7.m7.1.1.3.1.cmml">​</mo><msup id="A1.5.p4.7.m7.1.1.3.3" xref="A1.5.p4.7.m7.1.1.3.3.cmml"><mi id="A1.5.p4.7.m7.1.1.3.3.2" xref="A1.5.p4.7.m7.1.1.3.3.2.cmml">n</mi><mi id="A1.5.p4.7.m7.1.1.3.3.3" xref="A1.5.p4.7.m7.1.1.3.3.3.cmml">c</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.5.p4.7.m7.1b"><apply id="A1.5.p4.7.m7.1.1.cmml" xref="A1.5.p4.7.m7.1.1"><leq id="A1.5.p4.7.m7.1.1.1.cmml" xref="A1.5.p4.7.m7.1.1.1"></leq><ci id="A1.5.p4.7.m7.1.1.2.cmml" xref="A1.5.p4.7.m7.1.1.2">𝑚</ci><apply id="A1.5.p4.7.m7.1.1.3.cmml" xref="A1.5.p4.7.m7.1.1.3"><times id="A1.5.p4.7.m7.1.1.3.1.cmml" xref="A1.5.p4.7.m7.1.1.3.1"></times><cn type="integer" id="A1.5.p4.7.m7.1.1.3.2.cmml" xref="A1.5.p4.7.m7.1.1.3.2">2</cn><apply id="A1.5.p4.7.m7.1.1.3.3.cmml" xref="A1.5.p4.7.m7.1.1.3.3"><csymbol cd="ambiguous" id="A1.5.p4.7.m7.1.1.3.3.1.cmml" xref="A1.5.p4.7.m7.1.1.3.3">superscript</csymbol><ci id="A1.5.p4.7.m7.1.1.3.3.2.cmml" xref="A1.5.p4.7.m7.1.1.3.3.2">𝑛</ci><ci id="A1.5.p4.7.m7.1.1.3.3.3.cmml" xref="A1.5.p4.7.m7.1.1.3.3.3">𝑐</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.7.m7.1c">m\leq 2n^{c}</annotation></semantics></math>,
this circuit is also of size <math id="A1.5.p4.8.m8.1" class="ltx_Math" alttext="\mathrm{poly}(n)" display="inline"><semantics id="A1.5.p4.8.m8.1a"><mrow id="A1.5.p4.8.m8.1.2" xref="A1.5.p4.8.m8.1.2.cmml"><mi id="A1.5.p4.8.m8.1.2.2" xref="A1.5.p4.8.m8.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.5.p4.8.m8.1.2.1" xref="A1.5.p4.8.m8.1.2.1.cmml">​</mo><mrow id="A1.5.p4.8.m8.1.2.3.2" xref="A1.5.p4.8.m8.1.2.cmml"><mo stretchy="false" id="A1.5.p4.8.m8.1.2.3.2.1" xref="A1.5.p4.8.m8.1.2.cmml">(</mo><mi id="A1.5.p4.8.m8.1.1" xref="A1.5.p4.8.m8.1.1.cmml">n</mi><mo stretchy="false" id="A1.5.p4.8.m8.1.2.3.2.2" xref="A1.5.p4.8.m8.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.5.p4.8.m8.1b"><apply id="A1.5.p4.8.m8.1.2.cmml" xref="A1.5.p4.8.m8.1.2"><times id="A1.5.p4.8.m8.1.2.1.cmml" xref="A1.5.p4.8.m8.1.2.1"></times><ci id="A1.5.p4.8.m8.1.2.2.cmml" xref="A1.5.p4.8.m8.1.2.2">poly</ci><ci id="A1.5.p4.8.m8.1.1.cmml" xref="A1.5.p4.8.m8.1.1">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.5.p4.8.m8.1c">\mathrm{poly}(n)</annotation></semantics></math>.</p>
</div>
<div id="A1.6.p5" class="ltx_para">
<p id="A1.6.p5.12" class="ltx_p">Finally, to compute the original iterated float product <math id="A1.6.p5.1.m1.1" class="ltx_Math" alttext="\bigotimes_{i=1}^{z}\phi_{i}" display="inline"><semantics id="A1.6.p5.1.m1.1a"><mrow id="A1.6.p5.1.m1.1.1" xref="A1.6.p5.1.m1.1.1.cmml"><msubsup id="A1.6.p5.1.m1.1.1.1" xref="A1.6.p5.1.m1.1.1.1.cmml"><mo id="A1.6.p5.1.m1.1.1.1.2.2" xref="A1.6.p5.1.m1.1.1.1.2.2.cmml">⨂</mo><mrow id="A1.6.p5.1.m1.1.1.1.2.3" xref="A1.6.p5.1.m1.1.1.1.2.3.cmml"><mi id="A1.6.p5.1.m1.1.1.1.2.3.2" xref="A1.6.p5.1.m1.1.1.1.2.3.2.cmml">i</mi><mo id="A1.6.p5.1.m1.1.1.1.2.3.1" xref="A1.6.p5.1.m1.1.1.1.2.3.1.cmml">=</mo><mn id="A1.6.p5.1.m1.1.1.1.2.3.3" xref="A1.6.p5.1.m1.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="A1.6.p5.1.m1.1.1.1.3" xref="A1.6.p5.1.m1.1.1.1.3.cmml">z</mi></msubsup><msub id="A1.6.p5.1.m1.1.1.2" xref="A1.6.p5.1.m1.1.1.2.cmml"><mi id="A1.6.p5.1.m1.1.1.2.2" xref="A1.6.p5.1.m1.1.1.2.2.cmml">ϕ</mi><mi id="A1.6.p5.1.m1.1.1.2.3" xref="A1.6.p5.1.m1.1.1.2.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="A1.6.p5.1.m1.1b"><apply id="A1.6.p5.1.m1.1.1.cmml" xref="A1.6.p5.1.m1.1.1"><apply id="A1.6.p5.1.m1.1.1.1.cmml" xref="A1.6.p5.1.m1.1.1.1"><csymbol cd="ambiguous" id="A1.6.p5.1.m1.1.1.1.1.cmml" xref="A1.6.p5.1.m1.1.1.1">superscript</csymbol><apply id="A1.6.p5.1.m1.1.1.1.2.cmml" xref="A1.6.p5.1.m1.1.1.1"><csymbol cd="ambiguous" id="A1.6.p5.1.m1.1.1.1.2.1.cmml" xref="A1.6.p5.1.m1.1.1.1">subscript</csymbol><csymbol cd="latexml" id="A1.6.p5.1.m1.1.1.1.2.2.cmml" xref="A1.6.p5.1.m1.1.1.1.2.2">tensor-product</csymbol><apply id="A1.6.p5.1.m1.1.1.1.2.3.cmml" xref="A1.6.p5.1.m1.1.1.1.2.3"><eq id="A1.6.p5.1.m1.1.1.1.2.3.1.cmml" xref="A1.6.p5.1.m1.1.1.1.2.3.1"></eq><ci id="A1.6.p5.1.m1.1.1.1.2.3.2.cmml" xref="A1.6.p5.1.m1.1.1.1.2.3.2">𝑖</ci><cn type="integer" id="A1.6.p5.1.m1.1.1.1.2.3.3.cmml" xref="A1.6.p5.1.m1.1.1.1.2.3.3">1</cn></apply></apply><ci id="A1.6.p5.1.m1.1.1.1.3.cmml" xref="A1.6.p5.1.m1.1.1.1.3">𝑧</ci></apply><apply id="A1.6.p5.1.m1.1.1.2.cmml" xref="A1.6.p5.1.m1.1.1.2"><csymbol cd="ambiguous" id="A1.6.p5.1.m1.1.1.2.1.cmml" xref="A1.6.p5.1.m1.1.1.2">subscript</csymbol><ci id="A1.6.p5.1.m1.1.1.2.2.cmml" xref="A1.6.p5.1.m1.1.1.2.2">italic-ϕ</ci><ci id="A1.6.p5.1.m1.1.1.2.3.cmml" xref="A1.6.p5.1.m1.1.1.2.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.1.m1.1c">\bigotimes_{i=1}^{z}\phi_{i}</annotation></semantics></math>, we divide <math id="A1.6.p5.2.m2.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="A1.6.p5.2.m2.1a"><mi id="A1.6.p5.2.m2.1.1" xref="A1.6.p5.2.m2.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="A1.6.p5.2.m2.1b"><ci id="A1.6.p5.2.m2.1.1.cmml" xref="A1.6.p5.2.m2.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.2.m2.1c">\tau</annotation></semantics></math> by <math id="A1.6.p5.3.m3.1" class="ltx_Math" alttext="2^{qz}" display="inline"><semantics id="A1.6.p5.3.m3.1a"><msup id="A1.6.p5.3.m3.1.1" xref="A1.6.p5.3.m3.1.1.cmml"><mn id="A1.6.p5.3.m3.1.1.2" xref="A1.6.p5.3.m3.1.1.2.cmml">2</mn><mrow id="A1.6.p5.3.m3.1.1.3" xref="A1.6.p5.3.m3.1.1.3.cmml"><mi id="A1.6.p5.3.m3.1.1.3.2" xref="A1.6.p5.3.m3.1.1.3.2.cmml">q</mi><mo lspace="0em" rspace="0em" id="A1.6.p5.3.m3.1.1.3.1" xref="A1.6.p5.3.m3.1.1.3.1.cmml">​</mo><mi id="A1.6.p5.3.m3.1.1.3.3" xref="A1.6.p5.3.m3.1.1.3.3.cmml">z</mi></mrow></msup><annotation-xml encoding="MathML-Content" id="A1.6.p5.3.m3.1b"><apply id="A1.6.p5.3.m3.1.1.cmml" xref="A1.6.p5.3.m3.1.1"><csymbol cd="ambiguous" id="A1.6.p5.3.m3.1.1.1.cmml" xref="A1.6.p5.3.m3.1.1">superscript</csymbol><cn type="integer" id="A1.6.p5.3.m3.1.1.2.cmml" xref="A1.6.p5.3.m3.1.1.2">2</cn><apply id="A1.6.p5.3.m3.1.1.3.cmml" xref="A1.6.p5.3.m3.1.1.3"><times id="A1.6.p5.3.m3.1.1.3.1.cmml" xref="A1.6.p5.3.m3.1.1.3.1"></times><ci id="A1.6.p5.3.m3.1.1.3.2.cmml" xref="A1.6.p5.3.m3.1.1.3.2">𝑞</ci><ci id="A1.6.p5.3.m3.1.1.3.3.cmml" xref="A1.6.p5.3.m3.1.1.3.3">𝑧</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.3.m3.1c">2^{qz}</annotation></semantics></math>. This can be accomplished by subtracting <math id="A1.6.p5.4.m4.1" class="ltx_Math" alttext="qz" display="inline"><semantics id="A1.6.p5.4.m4.1a"><mrow id="A1.6.p5.4.m4.1.1" xref="A1.6.p5.4.m4.1.1.cmml"><mi id="A1.6.p5.4.m4.1.1.2" xref="A1.6.p5.4.m4.1.1.2.cmml">q</mi><mo lspace="0em" rspace="0em" id="A1.6.p5.4.m4.1.1.1" xref="A1.6.p5.4.m4.1.1.1.cmml">​</mo><mi id="A1.6.p5.4.m4.1.1.3" xref="A1.6.p5.4.m4.1.1.3.cmml">z</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.6.p5.4.m4.1b"><apply id="A1.6.p5.4.m4.1.1.cmml" xref="A1.6.p5.4.m4.1.1"><times id="A1.6.p5.4.m4.1.1.1.cmml" xref="A1.6.p5.4.m4.1.1.1"></times><ci id="A1.6.p5.4.m4.1.1.2.cmml" xref="A1.6.p5.4.m4.1.1.2">𝑞</ci><ci id="A1.6.p5.4.m4.1.1.3.cmml" xref="A1.6.p5.4.m4.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.4.m4.1c">qz</annotation></semantics></math> from the exponent of <math id="A1.6.p5.5.m5.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="A1.6.p5.5.m5.1a"><mi id="A1.6.p5.5.m5.1.1" xref="A1.6.p5.5.m5.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="A1.6.p5.5.m5.1b"><ci id="A1.6.p5.5.m5.1.1.cmml" xref="A1.6.p5.5.m5.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.5.m5.1c">\tau</annotation></semantics></math>; again, we do this computation exactly, using up to <math id="A1.6.p5.6.m6.1" class="ltx_Math" alttext="(c+1)\log n" display="inline"><semantics id="A1.6.p5.6.m6.1a"><mrow id="A1.6.p5.6.m6.1.1" xref="A1.6.p5.6.m6.1.1.cmml"><mrow id="A1.6.p5.6.m6.1.1.1.1" xref="A1.6.p5.6.m6.1.1.1.1.1.cmml"><mo stretchy="false" id="A1.6.p5.6.m6.1.1.1.1.2" xref="A1.6.p5.6.m6.1.1.1.1.1.cmml">(</mo><mrow id="A1.6.p5.6.m6.1.1.1.1.1" xref="A1.6.p5.6.m6.1.1.1.1.1.cmml"><mi id="A1.6.p5.6.m6.1.1.1.1.1.2" xref="A1.6.p5.6.m6.1.1.1.1.1.2.cmml">c</mi><mo id="A1.6.p5.6.m6.1.1.1.1.1.1" xref="A1.6.p5.6.m6.1.1.1.1.1.1.cmml">+</mo><mn id="A1.6.p5.6.m6.1.1.1.1.1.3" xref="A1.6.p5.6.m6.1.1.1.1.1.3.cmml">1</mn></mrow><mo stretchy="false" id="A1.6.p5.6.m6.1.1.1.1.3" xref="A1.6.p5.6.m6.1.1.1.1.1.cmml">)</mo></mrow><mo lspace="0.167em" rspace="0em" id="A1.6.p5.6.m6.1.1.2" xref="A1.6.p5.6.m6.1.1.2.cmml">​</mo><mrow id="A1.6.p5.6.m6.1.1.3" xref="A1.6.p5.6.m6.1.1.3.cmml"><mi id="A1.6.p5.6.m6.1.1.3.1" xref="A1.6.p5.6.m6.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.6.p5.6.m6.1.1.3a" xref="A1.6.p5.6.m6.1.1.3.cmml">⁡</mo><mi id="A1.6.p5.6.m6.1.1.3.2" xref="A1.6.p5.6.m6.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.6.p5.6.m6.1b"><apply id="A1.6.p5.6.m6.1.1.cmml" xref="A1.6.p5.6.m6.1.1"><times id="A1.6.p5.6.m6.1.1.2.cmml" xref="A1.6.p5.6.m6.1.1.2"></times><apply id="A1.6.p5.6.m6.1.1.1.1.1.cmml" xref="A1.6.p5.6.m6.1.1.1.1"><plus id="A1.6.p5.6.m6.1.1.1.1.1.1.cmml" xref="A1.6.p5.6.m6.1.1.1.1.1.1"></plus><ci id="A1.6.p5.6.m6.1.1.1.1.1.2.cmml" xref="A1.6.p5.6.m6.1.1.1.1.1.2">𝑐</ci><cn type="integer" id="A1.6.p5.6.m6.1.1.1.1.1.3.cmml" xref="A1.6.p5.6.m6.1.1.1.1.1.3">1</cn></apply><apply id="A1.6.p5.6.m6.1.1.3.cmml" xref="A1.6.p5.6.m6.1.1.3"><log id="A1.6.p5.6.m6.1.1.3.1.cmml" xref="A1.6.p5.6.m6.1.1.3.1"></log><ci id="A1.6.p5.6.m6.1.1.3.2.cmml" xref="A1.6.p5.6.m6.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.6.m6.1c">(c+1)\log n</annotation></semantics></math> additional bits in the exponent if necessary. We then cast the resulting float back to a <math id="A1.6.p5.7.m7.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.6.p5.7.m7.1a"><mrow id="A1.6.p5.7.m7.1.1" xref="A1.6.p5.7.m7.1.1.cmml"><mi id="A1.6.p5.7.m7.1.1.2" xref="A1.6.p5.7.m7.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.6.p5.7.m7.1.1.1" xref="A1.6.p5.7.m7.1.1.1.cmml">​</mo><mrow id="A1.6.p5.7.m7.1.1.3" xref="A1.6.p5.7.m7.1.1.3.cmml"><mi id="A1.6.p5.7.m7.1.1.3.1" xref="A1.6.p5.7.m7.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.6.p5.7.m7.1.1.3a" xref="A1.6.p5.7.m7.1.1.3.cmml">⁡</mo><mi id="A1.6.p5.7.m7.1.1.3.2" xref="A1.6.p5.7.m7.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.6.p5.7.m7.1b"><apply id="A1.6.p5.7.m7.1.1.cmml" xref="A1.6.p5.7.m7.1.1"><times id="A1.6.p5.7.m7.1.1.1.cmml" xref="A1.6.p5.7.m7.1.1.1"></times><ci id="A1.6.p5.7.m7.1.1.2.cmml" xref="A1.6.p5.7.m7.1.1.2">𝑐</ci><apply id="A1.6.p5.7.m7.1.1.3.cmml" xref="A1.6.p5.7.m7.1.1.3"><log id="A1.6.p5.7.m7.1.1.3.1.cmml" xref="A1.6.p5.7.m7.1.1.3.1"></log><ci id="A1.6.p5.7.m7.1.1.3.2.cmml" xref="A1.6.p5.7.m7.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.7.m7.1c">c\log n</annotation></semantics></math> precision float. All this can be done in <math id="A1.6.p5.8.m8.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.6.p5.8.m8.1a"><mi id="A1.6.p5.8.m8.1.1" xref="A1.6.p5.8.m8.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.6.p5.8.m8.1b"><ci id="A1.6.p5.8.m8.1.1.cmml" xref="A1.6.p5.8.m8.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.8.m8.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.6.p5.9.m9.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.6.p5.9.m9.1a"><msup id="A1.6.p5.9.m9.1.1" xref="A1.6.p5.9.m9.1.1.cmml"><mi id="A1.6.p5.9.m9.1.1.2" xref="A1.6.p5.9.m9.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.6.p5.9.m9.1.1.3" xref="A1.6.p5.9.m9.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.6.p5.9.m9.1b"><apply id="A1.6.p5.9.m9.1.1.cmml" xref="A1.6.p5.9.m9.1.1"><csymbol cd="ambiguous" id="A1.6.p5.9.m9.1.1.1.cmml" xref="A1.6.p5.9.m9.1.1">superscript</csymbol><ci id="A1.6.p5.9.m9.1.1.2.cmml" xref="A1.6.p5.9.m9.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.6.p5.9.m9.1.1.3.cmml" xref="A1.6.p5.9.m9.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.9.m9.1c">\mathsf{TC}^{0}</annotation></semantics></math>, finishing the proof that <math id="A1.6.p5.10.m10.1" class="ltx_Math" alttext="\bigotimes_{i=1}^{z}\phi_{i}" display="inline"><semantics id="A1.6.p5.10.m10.1a"><mrow id="A1.6.p5.10.m10.1.1" xref="A1.6.p5.10.m10.1.1.cmml"><msubsup id="A1.6.p5.10.m10.1.1.1" xref="A1.6.p5.10.m10.1.1.1.cmml"><mo id="A1.6.p5.10.m10.1.1.1.2.2" xref="A1.6.p5.10.m10.1.1.1.2.2.cmml">⨂</mo><mrow id="A1.6.p5.10.m10.1.1.1.2.3" xref="A1.6.p5.10.m10.1.1.1.2.3.cmml"><mi id="A1.6.p5.10.m10.1.1.1.2.3.2" xref="A1.6.p5.10.m10.1.1.1.2.3.2.cmml">i</mi><mo id="A1.6.p5.10.m10.1.1.1.2.3.1" xref="A1.6.p5.10.m10.1.1.1.2.3.1.cmml">=</mo><mn id="A1.6.p5.10.m10.1.1.1.2.3.3" xref="A1.6.p5.10.m10.1.1.1.2.3.3.cmml">1</mn></mrow><mi id="A1.6.p5.10.m10.1.1.1.3" xref="A1.6.p5.10.m10.1.1.1.3.cmml">z</mi></msubsup><msub id="A1.6.p5.10.m10.1.1.2" xref="A1.6.p5.10.m10.1.1.2.cmml"><mi id="A1.6.p5.10.m10.1.1.2.2" xref="A1.6.p5.10.m10.1.1.2.2.cmml">ϕ</mi><mi id="A1.6.p5.10.m10.1.1.2.3" xref="A1.6.p5.10.m10.1.1.2.3.cmml">i</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="A1.6.p5.10.m10.1b"><apply id="A1.6.p5.10.m10.1.1.cmml" xref="A1.6.p5.10.m10.1.1"><apply id="A1.6.p5.10.m10.1.1.1.cmml" xref="A1.6.p5.10.m10.1.1.1"><csymbol cd="ambiguous" id="A1.6.p5.10.m10.1.1.1.1.cmml" xref="A1.6.p5.10.m10.1.1.1">superscript</csymbol><apply id="A1.6.p5.10.m10.1.1.1.2.cmml" xref="A1.6.p5.10.m10.1.1.1"><csymbol cd="ambiguous" id="A1.6.p5.10.m10.1.1.1.2.1.cmml" xref="A1.6.p5.10.m10.1.1.1">subscript</csymbol><csymbol cd="latexml" id="A1.6.p5.10.m10.1.1.1.2.2.cmml" xref="A1.6.p5.10.m10.1.1.1.2.2">tensor-product</csymbol><apply id="A1.6.p5.10.m10.1.1.1.2.3.cmml" xref="A1.6.p5.10.m10.1.1.1.2.3"><eq id="A1.6.p5.10.m10.1.1.1.2.3.1.cmml" xref="A1.6.p5.10.m10.1.1.1.2.3.1"></eq><ci id="A1.6.p5.10.m10.1.1.1.2.3.2.cmml" xref="A1.6.p5.10.m10.1.1.1.2.3.2">𝑖</ci><cn type="integer" id="A1.6.p5.10.m10.1.1.1.2.3.3.cmml" xref="A1.6.p5.10.m10.1.1.1.2.3.3">1</cn></apply></apply><ci id="A1.6.p5.10.m10.1.1.1.3.cmml" xref="A1.6.p5.10.m10.1.1.1.3">𝑧</ci></apply><apply id="A1.6.p5.10.m10.1.1.2.cmml" xref="A1.6.p5.10.m10.1.1.2"><csymbol cd="ambiguous" id="A1.6.p5.10.m10.1.1.2.1.cmml" xref="A1.6.p5.10.m10.1.1.2">subscript</csymbol><ci id="A1.6.p5.10.m10.1.1.2.2.cmml" xref="A1.6.p5.10.m10.1.1.2.2">italic-ϕ</ci><ci id="A1.6.p5.10.m10.1.1.2.3.cmml" xref="A1.6.p5.10.m10.1.1.2.3">𝑖</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.10.m10.1c">\bigotimes_{i=1}^{z}\phi_{i}</annotation></semantics></math> can be computed in <math id="A1.6.p5.11.m11.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.6.p5.11.m11.1a"><mi id="A1.6.p5.11.m11.1.1" xref="A1.6.p5.11.m11.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.6.p5.11.m11.1b"><ci id="A1.6.p5.11.m11.1.1.cmml" xref="A1.6.p5.11.m11.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.11.m11.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.6.p5.12.m12.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.6.p5.12.m12.1a"><msup id="A1.6.p5.12.m12.1.1" xref="A1.6.p5.12.m12.1.1.cmml"><mi id="A1.6.p5.12.m12.1.1.2" xref="A1.6.p5.12.m12.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.6.p5.12.m12.1.1.3" xref="A1.6.p5.12.m12.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.6.p5.12.m12.1b"><apply id="A1.6.p5.12.m12.1.1.cmml" xref="A1.6.p5.12.m12.1.1"><csymbol cd="ambiguous" id="A1.6.p5.12.m12.1.1.1.cmml" xref="A1.6.p5.12.m12.1.1">superscript</csymbol><ci id="A1.6.p5.12.m12.1.1.2.cmml" xref="A1.6.p5.12.m12.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.6.p5.12.m12.1.1.3.cmml" xref="A1.6.p5.12.m12.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.6.p5.12.m12.1c">\mathsf{TC}^{0}</annotation></semantics></math>.
∎</p>
</div>
</div>
<div id="A1.p3" class="ltx_para">
<p id="A1.p3.1" class="ltx_p">We now extend these notions and results to matrix powering.</p>
</div>
<div id="A1.Thmtheorem3" class="ltx_theorem ltx_theorem_definition">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="A1.Thmtheorem3.2.1.1" class="ltx_text ltx_font_bold">Definition A.3</span></span><span id="A1.Thmtheorem3.3.2" class="ltx_text ltx_font_bold"> </span>(<math id="A1.Thmtheorem3.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem3.1.m1.1b"><mi id="A1.Thmtheorem3.1.m1.1.1" xref="A1.Thmtheorem3.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem3.1.m1.1c"><ci id="A1.Thmtheorem3.1.m1.1.1.cmml" xref="A1.Thmtheorem3.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem3.1.m1.1d">\mathbb{D}</annotation></semantics></math>-matrix power)<span id="A1.Thmtheorem3.4.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="A1.Thmtheorem3.p1" class="ltx_para">
<p id="A1.Thmtheorem3.p1.4" class="ltx_p">For a matrix <math id="A1.Thmtheorem3.p1.1.m1.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.Thmtheorem3.p1.1.m1.1a"><mi id="A1.Thmtheorem3.p1.1.m1.1.1" xref="A1.Thmtheorem3.p1.1.m1.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem3.p1.1.m1.1b"><ci id="A1.Thmtheorem3.p1.1.m1.1.1.cmml" xref="A1.Thmtheorem3.p1.1.m1.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem3.p1.1.m1.1c">M</annotation></semantics></math> over <math id="A1.Thmtheorem3.p1.2.m2.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem3.p1.2.m2.1a"><mi id="A1.Thmtheorem3.p1.2.m2.1.1" xref="A1.Thmtheorem3.p1.2.m2.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem3.p1.2.m2.1b"><ci id="A1.Thmtheorem3.p1.2.m2.1.1.cmml" xref="A1.Thmtheorem3.p1.2.m2.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem3.p1.2.m2.1c">\mathbb{D}</annotation></semantics></math> and <math id="A1.Thmtheorem3.p1.3.m3.1" class="ltx_Math" alttext="z\in\mathbb{Z}^{+}" display="inline"><semantics id="A1.Thmtheorem3.p1.3.m3.1a"><mrow id="A1.Thmtheorem3.p1.3.m3.1.1" xref="A1.Thmtheorem3.p1.3.m3.1.1.cmml"><mi id="A1.Thmtheorem3.p1.3.m3.1.1.2" xref="A1.Thmtheorem3.p1.3.m3.1.1.2.cmml">z</mi><mo id="A1.Thmtheorem3.p1.3.m3.1.1.1" xref="A1.Thmtheorem3.p1.3.m3.1.1.1.cmml">∈</mo><msup id="A1.Thmtheorem3.p1.3.m3.1.1.3" xref="A1.Thmtheorem3.p1.3.m3.1.1.3.cmml"><mi id="A1.Thmtheorem3.p1.3.m3.1.1.3.2" xref="A1.Thmtheorem3.p1.3.m3.1.1.3.2.cmml">ℤ</mi><mo id="A1.Thmtheorem3.p1.3.m3.1.1.3.3" xref="A1.Thmtheorem3.p1.3.m3.1.1.3.3.cmml">+</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem3.p1.3.m3.1b"><apply id="A1.Thmtheorem3.p1.3.m3.1.1.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1"><in id="A1.Thmtheorem3.p1.3.m3.1.1.1.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1.1"></in><ci id="A1.Thmtheorem3.p1.3.m3.1.1.2.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1.2">𝑧</ci><apply id="A1.Thmtheorem3.p1.3.m3.1.1.3.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1.3"><csymbol cd="ambiguous" id="A1.Thmtheorem3.p1.3.m3.1.1.3.1.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1.3">superscript</csymbol><ci id="A1.Thmtheorem3.p1.3.m3.1.1.3.2.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1.3.2">ℤ</ci><plus id="A1.Thmtheorem3.p1.3.m3.1.1.3.3.cmml" xref="A1.Thmtheorem3.p1.3.m3.1.1.3.3"></plus></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem3.p1.3.m3.1c">z\in\mathbb{Z}^{+}</annotation></semantics></math>, <em id="A1.Thmtheorem3.p1.4.1" class="ltx_emph ltx_font_italic"><math id="A1.Thmtheorem3.p1.4.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem3.p1.4.1.m1.1a"><mi id="A1.Thmtheorem3.p1.4.1.m1.1.1" xref="A1.Thmtheorem3.p1.4.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem3.p1.4.1.m1.1b"><ci id="A1.Thmtheorem3.p1.4.1.m1.1.1.cmml" xref="A1.Thmtheorem3.p1.4.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem3.p1.4.1.m1.1c">\mathbb{D}</annotation></semantics></math>-matrix power</em> is defined as</p>
<table id="A1.E9" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="A1.E9.m1.2" class="ltx_Math" alttext="M^{z}\,\triangleq\,\mathsf{cast}_{\mathbb{D}}(\mathsf{cast}_{\mathbb{R}}(M)^{z})." display="block"><semantics id="A1.E9.m1.2a"><mrow id="A1.E9.m1.2.2.1" xref="A1.E9.m1.2.2.1.1.cmml"><mrow id="A1.E9.m1.2.2.1.1" xref="A1.E9.m1.2.2.1.1.cmml"><msup id="A1.E9.m1.2.2.1.1.3" xref="A1.E9.m1.2.2.1.1.3.cmml"><mi id="A1.E9.m1.2.2.1.1.3.2" xref="A1.E9.m1.2.2.1.1.3.2.cmml">M</mi><mi id="A1.E9.m1.2.2.1.1.3.3" xref="A1.E9.m1.2.2.1.1.3.3.cmml">z</mi></msup><mo lspace="0.448em" rspace="0.448em" id="A1.E9.m1.2.2.1.1.2" xref="A1.E9.m1.2.2.1.1.2.cmml">≜</mo><mrow id="A1.E9.m1.2.2.1.1.1" xref="A1.E9.m1.2.2.1.1.1.cmml"><msub id="A1.E9.m1.2.2.1.1.1.3" xref="A1.E9.m1.2.2.1.1.1.3.cmml"><mi id="A1.E9.m1.2.2.1.1.1.3.2" xref="A1.E9.m1.2.2.1.1.1.3.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.E9.m1.2.2.1.1.1.3.3" xref="A1.E9.m1.2.2.1.1.1.3.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="A1.E9.m1.2.2.1.1.1.2" xref="A1.E9.m1.2.2.1.1.1.2.cmml">​</mo><mrow id="A1.E9.m1.2.2.1.1.1.1.1" xref="A1.E9.m1.2.2.1.1.1.1.1.1.cmml"><mo stretchy="false" id="A1.E9.m1.2.2.1.1.1.1.1.2" xref="A1.E9.m1.2.2.1.1.1.1.1.1.cmml">(</mo><mrow id="A1.E9.m1.2.2.1.1.1.1.1.1" xref="A1.E9.m1.2.2.1.1.1.1.1.1.cmml"><msub id="A1.E9.m1.2.2.1.1.1.1.1.1.2" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2.cmml"><mi id="A1.E9.m1.2.2.1.1.1.1.1.1.2.2" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.E9.m1.2.2.1.1.1.1.1.1.2.3" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2.3.cmml">ℝ</mi></msub><mo lspace="0em" rspace="0em" id="A1.E9.m1.2.2.1.1.1.1.1.1.1" xref="A1.E9.m1.2.2.1.1.1.1.1.1.1.cmml">​</mo><msup id="A1.E9.m1.2.2.1.1.1.1.1.1.3" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3.cmml"><mrow id="A1.E9.m1.2.2.1.1.1.1.1.1.3.2.2" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3.cmml"><mo stretchy="false" id="A1.E9.m1.2.2.1.1.1.1.1.1.3.2.2.1" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3.cmml">(</mo><mi id="A1.E9.m1.1.1" xref="A1.E9.m1.1.1.cmml">M</mi><mo stretchy="false" id="A1.E9.m1.2.2.1.1.1.1.1.1.3.2.2.2" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3.cmml">)</mo></mrow><mi id="A1.E9.m1.2.2.1.1.1.1.1.1.3.3" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3.3.cmml">z</mi></msup></mrow><mo stretchy="false" id="A1.E9.m1.2.2.1.1.1.1.1.3" xref="A1.E9.m1.2.2.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="A1.E9.m1.2.2.1.2" xref="A1.E9.m1.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="A1.E9.m1.2b"><apply id="A1.E9.m1.2.2.1.1.cmml" xref="A1.E9.m1.2.2.1"><ci id="A1.E9.m1.2.2.1.1.2.cmml" xref="A1.E9.m1.2.2.1.1.2">≜</ci><apply id="A1.E9.m1.2.2.1.1.3.cmml" xref="A1.E9.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="A1.E9.m1.2.2.1.1.3.1.cmml" xref="A1.E9.m1.2.2.1.1.3">superscript</csymbol><ci id="A1.E9.m1.2.2.1.1.3.2.cmml" xref="A1.E9.m1.2.2.1.1.3.2">𝑀</ci><ci id="A1.E9.m1.2.2.1.1.3.3.cmml" xref="A1.E9.m1.2.2.1.1.3.3">𝑧</ci></apply><apply id="A1.E9.m1.2.2.1.1.1.cmml" xref="A1.E9.m1.2.2.1.1.1"><times id="A1.E9.m1.2.2.1.1.1.2.cmml" xref="A1.E9.m1.2.2.1.1.1.2"></times><apply id="A1.E9.m1.2.2.1.1.1.3.cmml" xref="A1.E9.m1.2.2.1.1.1.3"><csymbol cd="ambiguous" id="A1.E9.m1.2.2.1.1.1.3.1.cmml" xref="A1.E9.m1.2.2.1.1.1.3">subscript</csymbol><ci id="A1.E9.m1.2.2.1.1.1.3.2.cmml" xref="A1.E9.m1.2.2.1.1.1.3.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.E9.m1.2.2.1.1.1.3.3.cmml" xref="A1.E9.m1.2.2.1.1.1.3.3">𝔻</ci></apply><apply id="A1.E9.m1.2.2.1.1.1.1.1.1.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1"><times id="A1.E9.m1.2.2.1.1.1.1.1.1.1.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.1"></times><apply id="A1.E9.m1.2.2.1.1.1.1.1.1.2.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="A1.E9.m1.2.2.1.1.1.1.1.1.2.1.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2">subscript</csymbol><ci id="A1.E9.m1.2.2.1.1.1.1.1.1.2.2.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.E9.m1.2.2.1.1.1.1.1.1.2.3.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.2.3">ℝ</ci></apply><apply id="A1.E9.m1.2.2.1.1.1.1.1.1.3.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3"><csymbol cd="ambiguous" id="A1.E9.m1.2.2.1.1.1.1.1.1.3.1.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3">superscript</csymbol><ci id="A1.E9.m1.1.1.cmml" xref="A1.E9.m1.1.1">𝑀</ci><ci id="A1.E9.m1.2.2.1.1.1.1.1.1.3.3.cmml" xref="A1.E9.m1.2.2.1.1.1.1.1.1.3.3">𝑧</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.E9.m1.2c">M^{z}\,\triangleq\,\mathsf{cast}_{\mathbb{D}}(\mathsf{cast}_{\mathbb{R}}(M)^{z}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(9)</span></td>
</tr></tbody>
</table>
</div>
</div>
<div id="A1.p4" class="ltx_para">
<p id="A1.p4.14" class="ltx_p"><cite class="ltx_cite ltx_citemacro_citet">Mereghetti &amp; Palano (<a href="#bib.bib23" title="" class="ltx_ref">2000</a>)</cite> showed that when the datatype <math id="A1.p4.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.p4.1.m1.1a"><mi id="A1.p4.1.m1.1.1" xref="A1.p4.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.p4.1.m1.1b"><ci id="A1.p4.1.m1.1.1.cmml" xref="A1.p4.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.1.m1.1c">\mathbb{D}</annotation></semantics></math> is <math id="A1.p4.2.m2.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.p4.2.m2.1a"><mi id="A1.p4.2.m2.1.1" xref="A1.p4.2.m2.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.p4.2.m2.1b"><ci id="A1.p4.2.m2.1.1.cmml" xref="A1.p4.2.m2.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.2.m2.1c">n</annotation></semantics></math>-bit integers, one can compute <math id="A1.p4.3.m3.1" class="ltx_Math" alttext="M^{n}" display="inline"><semantics id="A1.p4.3.m3.1a"><msup id="A1.p4.3.m3.1.1" xref="A1.p4.3.m3.1.1.cmml"><mi id="A1.p4.3.m3.1.1.2" xref="A1.p4.3.m3.1.1.2.cmml">M</mi><mi id="A1.p4.3.m3.1.1.3" xref="A1.p4.3.m3.1.1.3.cmml">n</mi></msup><annotation-xml encoding="MathML-Content" id="A1.p4.3.m3.1b"><apply id="A1.p4.3.m3.1.1.cmml" xref="A1.p4.3.m3.1.1"><csymbol cd="ambiguous" id="A1.p4.3.m3.1.1.1.cmml" xref="A1.p4.3.m3.1.1">superscript</csymbol><ci id="A1.p4.3.m3.1.1.2.cmml" xref="A1.p4.3.m3.1.1.2">𝑀</ci><ci id="A1.p4.3.m3.1.1.3.cmml" xref="A1.p4.3.m3.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.3.m3.1c">M^{n}</annotation></semantics></math> in <math id="A1.p4.4.m4.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.p4.4.m4.1a"><msup id="A1.p4.4.m4.1.1" xref="A1.p4.4.m4.1.1.cmml"><mi id="A1.p4.4.m4.1.1.2" xref="A1.p4.4.m4.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.p4.4.m4.1.1.3" xref="A1.p4.4.m4.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.p4.4.m4.1b"><apply id="A1.p4.4.m4.1.1.cmml" xref="A1.p4.4.m4.1.1"><csymbol cd="ambiguous" id="A1.p4.4.m4.1.1.1.cmml" xref="A1.p4.4.m4.1.1">superscript</csymbol><ci id="A1.p4.4.m4.1.1.2.cmml" xref="A1.p4.4.m4.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.p4.4.m4.1.1.3.cmml" xref="A1.p4.4.m4.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.4.m4.1c">\mathsf{TC}^{0}</annotation></semantics></math>. We note that their construction also works for computing <math id="A1.p4.5.m5.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="A1.p4.5.m5.1a"><msup id="A1.p4.5.m5.1.1" xref="A1.p4.5.m5.1.1.cmml"><mi id="A1.p4.5.m5.1.1.2" xref="A1.p4.5.m5.1.1.2.cmml">M</mi><mi id="A1.p4.5.m5.1.1.3" xref="A1.p4.5.m5.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.p4.5.m5.1b"><apply id="A1.p4.5.m5.1.1.cmml" xref="A1.p4.5.m5.1.1"><csymbol cd="ambiguous" id="A1.p4.5.m5.1.1.1.cmml" xref="A1.p4.5.m5.1.1">superscript</csymbol><ci id="A1.p4.5.m5.1.1.2.cmml" xref="A1.p4.5.m5.1.1.2">𝑀</ci><ci id="A1.p4.5.m5.1.1.3.cmml" xref="A1.p4.5.m5.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.5.m5.1c">M^{z}</annotation></semantics></math> for any <math id="A1.p4.6.m6.2" class="ltx_Math" alttext="z\leq n,z\in\mathbb{Z}^{+}" display="inline"><semantics id="A1.p4.6.m6.2a"><mrow id="A1.p4.6.m6.2.2.2" xref="A1.p4.6.m6.2.2.3.cmml"><mrow id="A1.p4.6.m6.1.1.1.1" xref="A1.p4.6.m6.1.1.1.1.cmml"><mi id="A1.p4.6.m6.1.1.1.1.2" xref="A1.p4.6.m6.1.1.1.1.2.cmml">z</mi><mo id="A1.p4.6.m6.1.1.1.1.1" xref="A1.p4.6.m6.1.1.1.1.1.cmml">≤</mo><mi id="A1.p4.6.m6.1.1.1.1.3" xref="A1.p4.6.m6.1.1.1.1.3.cmml">n</mi></mrow><mo id="A1.p4.6.m6.2.2.2.3" xref="A1.p4.6.m6.2.2.3a.cmml">,</mo><mrow id="A1.p4.6.m6.2.2.2.2" xref="A1.p4.6.m6.2.2.2.2.cmml"><mi id="A1.p4.6.m6.2.2.2.2.2" xref="A1.p4.6.m6.2.2.2.2.2.cmml">z</mi><mo id="A1.p4.6.m6.2.2.2.2.1" xref="A1.p4.6.m6.2.2.2.2.1.cmml">∈</mo><msup id="A1.p4.6.m6.2.2.2.2.3" xref="A1.p4.6.m6.2.2.2.2.3.cmml"><mi id="A1.p4.6.m6.2.2.2.2.3.2" xref="A1.p4.6.m6.2.2.2.2.3.2.cmml">ℤ</mi><mo id="A1.p4.6.m6.2.2.2.2.3.3" xref="A1.p4.6.m6.2.2.2.2.3.3.cmml">+</mo></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.p4.6.m6.2b"><apply id="A1.p4.6.m6.2.2.3.cmml" xref="A1.p4.6.m6.2.2.2"><csymbol cd="ambiguous" id="A1.p4.6.m6.2.2.3a.cmml" xref="A1.p4.6.m6.2.2.2.3">formulae-sequence</csymbol><apply id="A1.p4.6.m6.1.1.1.1.cmml" xref="A1.p4.6.m6.1.1.1.1"><leq id="A1.p4.6.m6.1.1.1.1.1.cmml" xref="A1.p4.6.m6.1.1.1.1.1"></leq><ci id="A1.p4.6.m6.1.1.1.1.2.cmml" xref="A1.p4.6.m6.1.1.1.1.2">𝑧</ci><ci id="A1.p4.6.m6.1.1.1.1.3.cmml" xref="A1.p4.6.m6.1.1.1.1.3">𝑛</ci></apply><apply id="A1.p4.6.m6.2.2.2.2.cmml" xref="A1.p4.6.m6.2.2.2.2"><in id="A1.p4.6.m6.2.2.2.2.1.cmml" xref="A1.p4.6.m6.2.2.2.2.1"></in><ci id="A1.p4.6.m6.2.2.2.2.2.cmml" xref="A1.p4.6.m6.2.2.2.2.2">𝑧</ci><apply id="A1.p4.6.m6.2.2.2.2.3.cmml" xref="A1.p4.6.m6.2.2.2.2.3"><csymbol cd="ambiguous" id="A1.p4.6.m6.2.2.2.2.3.1.cmml" xref="A1.p4.6.m6.2.2.2.2.3">superscript</csymbol><ci id="A1.p4.6.m6.2.2.2.2.3.2.cmml" xref="A1.p4.6.m6.2.2.2.2.3.2">ℤ</ci><plus id="A1.p4.6.m6.2.2.2.2.3.3.cmml" xref="A1.p4.6.m6.2.2.2.2.3.3"></plus></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.6.m6.2c">z\leq n,z\in\mathbb{Z}^{+}</annotation></semantics></math>. Further, as they remark, their construction can, in fact, be done in <em id="A1.p4.14.1" class="ltx_emph ltx_font_italic">uniform</em> <math id="A1.p4.7.m7.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.p4.7.m7.1a"><msup id="A1.p4.7.m7.1.1" xref="A1.p4.7.m7.1.1.cmml"><mi id="A1.p4.7.m7.1.1.2" xref="A1.p4.7.m7.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.p4.7.m7.1.1.3" xref="A1.p4.7.m7.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.p4.7.m7.1b"><apply id="A1.p4.7.m7.1.1.cmml" xref="A1.p4.7.m7.1.1"><csymbol cd="ambiguous" id="A1.p4.7.m7.1.1.1.cmml" xref="A1.p4.7.m7.1.1">superscript</csymbol><ci id="A1.p4.7.m7.1.1.2.cmml" xref="A1.p4.7.m7.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.p4.7.m7.1.1.3.cmml" xref="A1.p4.7.m7.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.7.m7.1c">\mathsf{TC}^{0}</annotation></semantics></math>. Specifically, we observe most of their construction involves sums and products of constantly many <math id="A1.p4.8.m8.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.p4.8.m8.1a"><mi id="A1.p4.8.m8.1.1" xref="A1.p4.8.m8.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.p4.8.m8.1b"><ci id="A1.p4.8.m8.1.1.cmml" xref="A1.p4.8.m8.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.8.m8.1c">n</annotation></semantics></math>-bit integers, which can be done in <math id="A1.p4.9.m9.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.p4.9.m9.1a"><mi id="A1.p4.9.m9.1.1" xref="A1.p4.9.m9.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.p4.9.m9.1b"><ci id="A1.p4.9.m9.1.1.cmml" xref="A1.p4.9.m9.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.9.m9.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.p4.10.m10.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.p4.10.m10.1a"><msup id="A1.p4.10.m10.1.1" xref="A1.p4.10.m10.1.1.cmml"><mi id="A1.p4.10.m10.1.1.2" xref="A1.p4.10.m10.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.p4.10.m10.1.1.3" xref="A1.p4.10.m10.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.p4.10.m10.1b"><apply id="A1.p4.10.m10.1.1.cmml" xref="A1.p4.10.m10.1.1"><csymbol cd="ambiguous" id="A1.p4.10.m10.1.1.1.cmml" xref="A1.p4.10.m10.1.1">superscript</csymbol><ci id="A1.p4.10.m10.1.1.2.cmml" xref="A1.p4.10.m10.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.p4.10.m10.1.1.3.cmml" xref="A1.p4.10.m10.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.10.m10.1c">\mathsf{TC}^{0}</annotation></semantics></math>. The only involved step is dividing a polynomial of degree (up to) <math id="A1.p4.11.m11.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.p4.11.m11.1a"><mi id="A1.p4.11.m11.1.1" xref="A1.p4.11.m11.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.p4.11.m11.1b"><ci id="A1.p4.11.m11.1.1.cmml" xref="A1.p4.11.m11.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.11.m11.1c">n</annotation></semantics></math> by a polynomial of degree (up to) <math id="A1.p4.12.m12.1" class="ltx_Math" alttext="k-1" display="inline"><semantics id="A1.p4.12.m12.1a"><mrow id="A1.p4.12.m12.1.1" xref="A1.p4.12.m12.1.1.cmml"><mi id="A1.p4.12.m12.1.1.2" xref="A1.p4.12.m12.1.1.2.cmml">k</mi><mo id="A1.p4.12.m12.1.1.1" xref="A1.p4.12.m12.1.1.1.cmml">−</mo><mn id="A1.p4.12.m12.1.1.3" xref="A1.p4.12.m12.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="A1.p4.12.m12.1b"><apply id="A1.p4.12.m12.1.1.cmml" xref="A1.p4.12.m12.1.1"><minus id="A1.p4.12.m12.1.1.1.cmml" xref="A1.p4.12.m12.1.1.1"></minus><ci id="A1.p4.12.m12.1.1.2.cmml" xref="A1.p4.12.m12.1.1.2">𝑘</ci><cn type="integer" id="A1.p4.12.m12.1.1.3.cmml" xref="A1.p4.12.m12.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.12.m12.1c">k-1</annotation></semantics></math> and returning the remainder. It turns out that this ‘polynomial division with remainder’ operation can also be performed in <math id="A1.p4.13.m13.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.p4.13.m13.1a"><mi id="A1.p4.13.m13.1.1" xref="A1.p4.13.m13.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.p4.13.m13.1b"><ci id="A1.p4.13.m13.1.1.cmml" xref="A1.p4.13.m13.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.13.m13.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.p4.14.m14.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.p4.14.m14.1a"><msup id="A1.p4.14.m14.1.1" xref="A1.p4.14.m14.1.1.cmml"><mi id="A1.p4.14.m14.1.1.2" xref="A1.p4.14.m14.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.p4.14.m14.1.1.3" xref="A1.p4.14.m14.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.p4.14.m14.1b"><apply id="A1.p4.14.m14.1.1.cmml" xref="A1.p4.14.m14.1.1"><csymbol cd="ambiguous" id="A1.p4.14.m14.1.1.1.cmml" xref="A1.p4.14.m14.1.1">superscript</csymbol><ci id="A1.p4.14.m14.1.1.2.cmml" xref="A1.p4.14.m14.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.p4.14.m14.1.1.3.cmml" xref="A1.p4.14.m14.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.p4.14.m14.1c">\mathsf{TC}^{0}</annotation></semantics></math> (see Corollary 6.5 of <cite class="ltx_cite ltx_citemacro_citep">Hesse et al., <a href="#bib.bib15" title="" class="ltx_ref">2002</a></cite> and an explanation in <a href="#A1.SS1" title="A.1 𝖫-Uniformity of Polynomial Division in 𝖳𝖢⁰ ‣ Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Section</span> <span class="ltx_text ltx_ref_tag">A.1</span></a>). We thus have the following extension of <cite class="ltx_cite ltx_citemacro_citeauthor"><a href="#bib.bib23" title="" class="ltx_ref">Mereghetti &amp; Palano</a></cite>’s result:</p>
</div>
<div id="A1.Thmtheorem4" class="ltx_theorem ltx_theorem_lemma">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="A1.Thmtheorem4.1.1.1" class="ltx_text ltx_font_bold">Lemma A.4</span></span><span id="A1.Thmtheorem4.2.2" class="ltx_text ltx_font_bold"> </span>(Integer matrix power, derived from <cite class="ltx_cite ltx_citemacro_citep">Mereghetti &amp; Palano, <a href="#bib.bib23" title="" class="ltx_ref">2000</a></cite>)<span id="A1.Thmtheorem4.3.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="A1.Thmtheorem4.p1" class="ltx_para">
<p id="A1.Thmtheorem4.p1.8" class="ltx_p"><span id="A1.Thmtheorem4.p1.8.8" class="ltx_text ltx_font_italic">Let <math id="A1.Thmtheorem4.p1.1.1.m1.1" class="ltx_Math" alttext="k\in\mathbb{Z}^{+}" display="inline"><semantics id="A1.Thmtheorem4.p1.1.1.m1.1a"><mrow id="A1.Thmtheorem4.p1.1.1.m1.1.1" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.cmml"><mi id="A1.Thmtheorem4.p1.1.1.m1.1.1.2" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.2.cmml">k</mi><mo id="A1.Thmtheorem4.p1.1.1.m1.1.1.1" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.1.cmml">∈</mo><msup id="A1.Thmtheorem4.p1.1.1.m1.1.1.3" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3.cmml"><mi id="A1.Thmtheorem4.p1.1.1.m1.1.1.3.2" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3.2.cmml">ℤ</mi><mo id="A1.Thmtheorem4.p1.1.1.m1.1.1.3.3" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3.3.cmml">+</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.1.1.m1.1b"><apply id="A1.Thmtheorem4.p1.1.1.m1.1.1.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1"><in id="A1.Thmtheorem4.p1.1.1.m1.1.1.1.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.1"></in><ci id="A1.Thmtheorem4.p1.1.1.m1.1.1.2.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.2">𝑘</ci><apply id="A1.Thmtheorem4.p1.1.1.m1.1.1.3.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3"><csymbol cd="ambiguous" id="A1.Thmtheorem4.p1.1.1.m1.1.1.3.1.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3">superscript</csymbol><ci id="A1.Thmtheorem4.p1.1.1.m1.1.1.3.2.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3.2">ℤ</ci><plus id="A1.Thmtheorem4.p1.1.1.m1.1.1.3.3.cmml" xref="A1.Thmtheorem4.p1.1.1.m1.1.1.3.3"></plus></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.1.1.m1.1c">k\in\mathbb{Z}^{+}</annotation></semantics></math> be a fixed constant.
Let <math id="A1.Thmtheorem4.p1.2.2.m2.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.Thmtheorem4.p1.2.2.m2.1a"><mi id="A1.Thmtheorem4.p1.2.2.m2.1.1" xref="A1.Thmtheorem4.p1.2.2.m2.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.2.2.m2.1b"><ci id="A1.Thmtheorem4.p1.2.2.m2.1.1.cmml" xref="A1.Thmtheorem4.p1.2.2.m2.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.2.2.m2.1c">M</annotation></semantics></math> be a <math id="A1.Thmtheorem4.p1.3.3.m3.1" class="ltx_Math" alttext="k\times k" display="inline"><semantics id="A1.Thmtheorem4.p1.3.3.m3.1a"><mrow id="A1.Thmtheorem4.p1.3.3.m3.1.1" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.cmml"><mi id="A1.Thmtheorem4.p1.3.3.m3.1.1.2" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.2.cmml">k</mi><mo lspace="0.222em" rspace="0.222em" id="A1.Thmtheorem4.p1.3.3.m3.1.1.1" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.1.cmml">×</mo><mi id="A1.Thmtheorem4.p1.3.3.m3.1.1.3" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.3.cmml">k</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.3.3.m3.1b"><apply id="A1.Thmtheorem4.p1.3.3.m3.1.1.cmml" xref="A1.Thmtheorem4.p1.3.3.m3.1.1"><times id="A1.Thmtheorem4.p1.3.3.m3.1.1.1.cmml" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.1"></times><ci id="A1.Thmtheorem4.p1.3.3.m3.1.1.2.cmml" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.2">𝑘</ci><ci id="A1.Thmtheorem4.p1.3.3.m3.1.1.3.cmml" xref="A1.Thmtheorem4.p1.3.3.m3.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.3.3.m3.1c">k\times k</annotation></semantics></math> matrix over <math id="A1.Thmtheorem4.p1.4.4.m4.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.Thmtheorem4.p1.4.4.m4.1a"><mi id="A1.Thmtheorem4.p1.4.4.m4.1.1" xref="A1.Thmtheorem4.p1.4.4.m4.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.4.4.m4.1b"><ci id="A1.Thmtheorem4.p1.4.4.m4.1.1.cmml" xref="A1.Thmtheorem4.p1.4.4.m4.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.4.4.m4.1c">n</annotation></semantics></math>-bit integers and <math id="A1.Thmtheorem4.p1.5.5.m5.2" class="ltx_Math" alttext="z\leq n,z\in\mathbb{Z}^{+}" display="inline"><semantics id="A1.Thmtheorem4.p1.5.5.m5.2a"><mrow id="A1.Thmtheorem4.p1.5.5.m5.2.2.2" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.3.cmml"><mrow id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.cmml"><mi id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.2" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.2.cmml">z</mi><mo id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.1" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.1.cmml">≤</mo><mi id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.3" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.3.cmml">n</mi></mrow><mo id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.3" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.3a.cmml">,</mo><mrow id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.cmml"><mi id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.2" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.2.cmml">z</mi><mo id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.1" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.1.cmml">∈</mo><msup id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.cmml"><mi id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2.cmml">ℤ</mi><mo id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3.cmml">+</mo></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.5.5.m5.2b"><apply id="A1.Thmtheorem4.p1.5.5.m5.2.2.3.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2"><csymbol cd="ambiguous" id="A1.Thmtheorem4.p1.5.5.m5.2.2.3a.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.3">formulae-sequence</csymbol><apply id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1"><leq id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.1.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.1"></leq><ci id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.2.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.2">𝑧</ci><ci id="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.3.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.1.1.1.1.3">𝑛</ci></apply><apply id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2"><in id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.1.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.1"></in><ci id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.2.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.2">𝑧</ci><apply id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3"><csymbol cd="ambiguous" id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.1.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3">superscript</csymbol><ci id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.2">ℤ</ci><plus id="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3.cmml" xref="A1.Thmtheorem4.p1.5.5.m5.2.2.2.2.3.3"></plus></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.5.5.m5.2c">z\leq n,z\in\mathbb{Z}^{+}</annotation></semantics></math>. Then integer matrix power <math id="A1.Thmtheorem4.p1.6.6.m6.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="A1.Thmtheorem4.p1.6.6.m6.1a"><msup id="A1.Thmtheorem4.p1.6.6.m6.1.1" xref="A1.Thmtheorem4.p1.6.6.m6.1.1.cmml"><mi id="A1.Thmtheorem4.p1.6.6.m6.1.1.2" xref="A1.Thmtheorem4.p1.6.6.m6.1.1.2.cmml">M</mi><mi id="A1.Thmtheorem4.p1.6.6.m6.1.1.3" xref="A1.Thmtheorem4.p1.6.6.m6.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.6.6.m6.1b"><apply id="A1.Thmtheorem4.p1.6.6.m6.1.1.cmml" xref="A1.Thmtheorem4.p1.6.6.m6.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem4.p1.6.6.m6.1.1.1.cmml" xref="A1.Thmtheorem4.p1.6.6.m6.1.1">superscript</csymbol><ci id="A1.Thmtheorem4.p1.6.6.m6.1.1.2.cmml" xref="A1.Thmtheorem4.p1.6.6.m6.1.1.2">𝑀</ci><ci id="A1.Thmtheorem4.p1.6.6.m6.1.1.3.cmml" xref="A1.Thmtheorem4.p1.6.6.m6.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.6.6.m6.1c">M^{z}</annotation></semantics></math> can be computed in <math id="A1.Thmtheorem4.p1.7.7.m7.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.Thmtheorem4.p1.7.7.m7.1a"><mi id="A1.Thmtheorem4.p1.7.7.m7.1.1" xref="A1.Thmtheorem4.p1.7.7.m7.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.7.7.m7.1b"><ci id="A1.Thmtheorem4.p1.7.7.m7.1.1.cmml" xref="A1.Thmtheorem4.p1.7.7.m7.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.7.7.m7.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.Thmtheorem4.p1.8.8.m8.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.Thmtheorem4.p1.8.8.m8.1a"><msup id="A1.Thmtheorem4.p1.8.8.m8.1.1" xref="A1.Thmtheorem4.p1.8.8.m8.1.1.cmml"><mi id="A1.Thmtheorem4.p1.8.8.m8.1.1.2" xref="A1.Thmtheorem4.p1.8.8.m8.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.Thmtheorem4.p1.8.8.m8.1.1.3" xref="A1.Thmtheorem4.p1.8.8.m8.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem4.p1.8.8.m8.1b"><apply id="A1.Thmtheorem4.p1.8.8.m8.1.1.cmml" xref="A1.Thmtheorem4.p1.8.8.m8.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem4.p1.8.8.m8.1.1.1.cmml" xref="A1.Thmtheorem4.p1.8.8.m8.1.1">superscript</csymbol><ci id="A1.Thmtheorem4.p1.8.8.m8.1.1.2.cmml" xref="A1.Thmtheorem4.p1.8.8.m8.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.Thmtheorem4.p1.8.8.m8.1.1.3.cmml" xref="A1.Thmtheorem4.p1.8.8.m8.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem4.p1.8.8.m8.1c">\mathsf{TC}^{0}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="A1.p5" class="ltx_para">
<p id="A1.p5.1" class="ltx_p">We extend this result to matrix powers over <math id="A1.p5.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.p5.1.m1.1a"><mi id="A1.p5.1.m1.1.1" xref="A1.p5.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.p5.1.m1.1b"><ci id="A1.p5.1.m1.1.1.cmml" xref="A1.p5.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p5.1.m1.1c">\mathbb{D}</annotation></semantics></math> rather than integers:</p>
</div>
<div id="A1.Thmtheorem5" class="ltx_theorem ltx_theorem_lemma">
<h6 class="ltx_title ltx_runin ltx_title_theorem">
<span class="ltx_tag ltx_tag_theorem"><span id="A1.Thmtheorem5.2.1.1" class="ltx_text ltx_font_bold">Lemma A.5</span></span><span id="A1.Thmtheorem5.3.2" class="ltx_text ltx_font_bold"> </span>(<math id="A1.Thmtheorem5.1.m1.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem5.1.m1.1b"><mi id="A1.Thmtheorem5.1.m1.1.1" xref="A1.Thmtheorem5.1.m1.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.1.m1.1c"><ci id="A1.Thmtheorem5.1.m1.1.1.cmml" xref="A1.Thmtheorem5.1.m1.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.1.m1.1d">\mathbb{D}</annotation></semantics></math>-matrix power)<span id="A1.Thmtheorem5.4.3" class="ltx_text ltx_font_bold">.</span>
</h6>
<div id="A1.Thmtheorem5.p1" class="ltx_para">
<p id="A1.Thmtheorem5.p1.14" class="ltx_p"><span id="A1.Thmtheorem5.p1.14.14" class="ltx_text ltx_font_italic">Let <math id="A1.Thmtheorem5.p1.1.1.m1.1" class="ltx_Math" alttext="k\in\mathbb{Z}^{+}" display="inline"><semantics id="A1.Thmtheorem5.p1.1.1.m1.1a"><mrow id="A1.Thmtheorem5.p1.1.1.m1.1.1" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.cmml"><mi id="A1.Thmtheorem5.p1.1.1.m1.1.1.2" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.2.cmml">k</mi><mo id="A1.Thmtheorem5.p1.1.1.m1.1.1.1" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.1.cmml">∈</mo><msup id="A1.Thmtheorem5.p1.1.1.m1.1.1.3" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3.cmml"><mi id="A1.Thmtheorem5.p1.1.1.m1.1.1.3.2" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3.2.cmml">ℤ</mi><mo id="A1.Thmtheorem5.p1.1.1.m1.1.1.3.3" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3.3.cmml">+</mo></msup></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.1.1.m1.1b"><apply id="A1.Thmtheorem5.p1.1.1.m1.1.1.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1"><in id="A1.Thmtheorem5.p1.1.1.m1.1.1.1.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.1"></in><ci id="A1.Thmtheorem5.p1.1.1.m1.1.1.2.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.2">𝑘</ci><apply id="A1.Thmtheorem5.p1.1.1.m1.1.1.3.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.1.1.m1.1.1.3.1.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3">superscript</csymbol><ci id="A1.Thmtheorem5.p1.1.1.m1.1.1.3.2.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3.2">ℤ</ci><plus id="A1.Thmtheorem5.p1.1.1.m1.1.1.3.3.cmml" xref="A1.Thmtheorem5.p1.1.1.m1.1.1.3.3"></plus></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.1.1.m1.1c">k\in\mathbb{Z}^{+}</annotation></semantics></math> be a fixed constant.
Let <math id="A1.Thmtheorem5.p1.2.2.m2.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.Thmtheorem5.p1.2.2.m2.1a"><mi id="A1.Thmtheorem5.p1.2.2.m2.1.1" xref="A1.Thmtheorem5.p1.2.2.m2.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.2.2.m2.1b"><ci id="A1.Thmtheorem5.p1.2.2.m2.1.1.cmml" xref="A1.Thmtheorem5.p1.2.2.m2.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.2.2.m2.1c">M</annotation></semantics></math> be a <math id="A1.Thmtheorem5.p1.3.3.m3.1" class="ltx_Math" alttext="k\times k" display="inline"><semantics id="A1.Thmtheorem5.p1.3.3.m3.1a"><mrow id="A1.Thmtheorem5.p1.3.3.m3.1.1" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.cmml"><mi id="A1.Thmtheorem5.p1.3.3.m3.1.1.2" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.2.cmml">k</mi><mo lspace="0.222em" rspace="0.222em" id="A1.Thmtheorem5.p1.3.3.m3.1.1.1" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.1.cmml">×</mo><mi id="A1.Thmtheorem5.p1.3.3.m3.1.1.3" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.3.cmml">k</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.3.3.m3.1b"><apply id="A1.Thmtheorem5.p1.3.3.m3.1.1.cmml" xref="A1.Thmtheorem5.p1.3.3.m3.1.1"><times id="A1.Thmtheorem5.p1.3.3.m3.1.1.1.cmml" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.1"></times><ci id="A1.Thmtheorem5.p1.3.3.m3.1.1.2.cmml" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.2">𝑘</ci><ci id="A1.Thmtheorem5.p1.3.3.m3.1.1.3.cmml" xref="A1.Thmtheorem5.p1.3.3.m3.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.3.3.m3.1c">k\times k</annotation></semantics></math> matrix over a datatype <math id="A1.Thmtheorem5.p1.4.4.m4.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem5.p1.4.4.m4.1a"><mi id="A1.Thmtheorem5.p1.4.4.m4.1.1" xref="A1.Thmtheorem5.p1.4.4.m4.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.4.4.m4.1b"><ci id="A1.Thmtheorem5.p1.4.4.m4.1.1.cmml" xref="A1.Thmtheorem5.p1.4.4.m4.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.4.4.m4.1c">\mathbb{D}</annotation></semantics></math> with entries equivalently representable as <math id="A1.Thmtheorem5.p1.5.5.m5.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.Thmtheorem5.p1.5.5.m5.1a"><mi id="A1.Thmtheorem5.p1.5.5.m5.1.1" xref="A1.Thmtheorem5.p1.5.5.m5.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.5.5.m5.1b"><ci id="A1.Thmtheorem5.p1.5.5.m5.1.1.cmml" xref="A1.Thmtheorem5.p1.5.5.m5.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.5.5.m5.1c">n</annotation></semantics></math>-bit integers. Let <math id="A1.Thmtheorem5.p1.6.6.m6.2" class="ltx_Math" alttext="z\leq n,z\in\mathbb{Z}^{+}" display="inline"><semantics id="A1.Thmtheorem5.p1.6.6.m6.2a"><mrow id="A1.Thmtheorem5.p1.6.6.m6.2.2.2" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.3.cmml"><mrow id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.cmml"><mi id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.2" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.2.cmml">z</mi><mo id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.1" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.1.cmml">≤</mo><mi id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.3" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.3.cmml">n</mi></mrow><mo id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.3" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.3a.cmml">,</mo><mrow id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.cmml"><mi id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.2" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.2.cmml">z</mi><mo id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.1" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.1.cmml">∈</mo><msup id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.cmml"><mi id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.2" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.2.cmml">ℤ</mi><mo id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.3" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.3.cmml">+</mo></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.6.6.m6.2b"><apply id="A1.Thmtheorem5.p1.6.6.m6.2.2.3.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.6.6.m6.2.2.3a.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.3">formulae-sequence</csymbol><apply id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1"><leq id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.1.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.1"></leq><ci id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.2.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.2">𝑧</ci><ci id="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.3.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.1.1.1.1.3">𝑛</ci></apply><apply id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2"><in id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.1.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.1"></in><ci id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.2.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.2">𝑧</ci><apply id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.1.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3">superscript</csymbol><ci id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.2.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.2">ℤ</ci><plus id="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.3.cmml" xref="A1.Thmtheorem5.p1.6.6.m6.2.2.2.2.3.3"></plus></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.6.6.m6.2c">z\leq n,z\in\mathbb{Z}^{+}</annotation></semantics></math>. If operators <math id="A1.Thmtheorem5.p1.7.7.m7.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{D}}" display="inline"><semantics id="A1.Thmtheorem5.p1.7.7.m7.1a"><msub id="A1.Thmtheorem5.p1.7.7.m7.1.1" xref="A1.Thmtheorem5.p1.7.7.m7.1.1.cmml"><mi id="A1.Thmtheorem5.p1.7.7.m7.1.1.2" xref="A1.Thmtheorem5.p1.7.7.m7.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.Thmtheorem5.p1.7.7.m7.1.1.3" xref="A1.Thmtheorem5.p1.7.7.m7.1.1.3.cmml">𝔻</mi></msub><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.7.7.m7.1b"><apply id="A1.Thmtheorem5.p1.7.7.m7.1.1.cmml" xref="A1.Thmtheorem5.p1.7.7.m7.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.7.7.m7.1.1.1.cmml" xref="A1.Thmtheorem5.p1.7.7.m7.1.1">subscript</csymbol><ci id="A1.Thmtheorem5.p1.7.7.m7.1.1.2.cmml" xref="A1.Thmtheorem5.p1.7.7.m7.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.Thmtheorem5.p1.7.7.m7.1.1.3.cmml" xref="A1.Thmtheorem5.p1.7.7.m7.1.1.3">𝔻</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.7.7.m7.1c">\mathsf{cast}_{\mathbb{D}}</annotation></semantics></math> and <math id="A1.Thmtheorem5.p1.8.8.m8.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}" display="inline"><semantics id="A1.Thmtheorem5.p1.8.8.m8.1a"><msub id="A1.Thmtheorem5.p1.8.8.m8.1.1" xref="A1.Thmtheorem5.p1.8.8.m8.1.1.cmml"><mi id="A1.Thmtheorem5.p1.8.8.m8.1.1.2" xref="A1.Thmtheorem5.p1.8.8.m8.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.Thmtheorem5.p1.8.8.m8.1.1.3" xref="A1.Thmtheorem5.p1.8.8.m8.1.1.3.cmml">ℝ</mi></msub><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.8.8.m8.1b"><apply id="A1.Thmtheorem5.p1.8.8.m8.1.1.cmml" xref="A1.Thmtheorem5.p1.8.8.m8.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.8.8.m8.1.1.1.cmml" xref="A1.Thmtheorem5.p1.8.8.m8.1.1">subscript</csymbol><ci id="A1.Thmtheorem5.p1.8.8.m8.1.1.2.cmml" xref="A1.Thmtheorem5.p1.8.8.m8.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.Thmtheorem5.p1.8.8.m8.1.1.3.cmml" xref="A1.Thmtheorem5.p1.8.8.m8.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.8.8.m8.1c">\mathsf{cast}_{\mathbb{R}}</annotation></semantics></math> are in <math id="A1.Thmtheorem5.p1.9.9.m9.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.Thmtheorem5.p1.9.9.m9.1a"><mi id="A1.Thmtheorem5.p1.9.9.m9.1.1" xref="A1.Thmtheorem5.p1.9.9.m9.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.9.9.m9.1b"><ci id="A1.Thmtheorem5.p1.9.9.m9.1.1.cmml" xref="A1.Thmtheorem5.p1.9.9.m9.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.9.9.m9.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.Thmtheorem5.p1.10.10.m10.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.Thmtheorem5.p1.10.10.m10.1a"><msup id="A1.Thmtheorem5.p1.10.10.m10.1.1" xref="A1.Thmtheorem5.p1.10.10.m10.1.1.cmml"><mi id="A1.Thmtheorem5.p1.10.10.m10.1.1.2" xref="A1.Thmtheorem5.p1.10.10.m10.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.Thmtheorem5.p1.10.10.m10.1.1.3" xref="A1.Thmtheorem5.p1.10.10.m10.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.10.10.m10.1b"><apply id="A1.Thmtheorem5.p1.10.10.m10.1.1.cmml" xref="A1.Thmtheorem5.p1.10.10.m10.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.10.10.m10.1.1.1.cmml" xref="A1.Thmtheorem5.p1.10.10.m10.1.1">superscript</csymbol><ci id="A1.Thmtheorem5.p1.10.10.m10.1.1.2.cmml" xref="A1.Thmtheorem5.p1.10.10.m10.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.Thmtheorem5.p1.10.10.m10.1.1.3.cmml" xref="A1.Thmtheorem5.p1.10.10.m10.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.10.10.m10.1c">\mathsf{TC}^{0}</annotation></semantics></math>, then <math id="A1.Thmtheorem5.p1.11.11.m11.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.Thmtheorem5.p1.11.11.m11.1a"><mi id="A1.Thmtheorem5.p1.11.11.m11.1.1" xref="A1.Thmtheorem5.p1.11.11.m11.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.11.11.m11.1b"><ci id="A1.Thmtheorem5.p1.11.11.m11.1.1.cmml" xref="A1.Thmtheorem5.p1.11.11.m11.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.11.11.m11.1c">\mathbb{D}</annotation></semantics></math>-matrix power <math id="A1.Thmtheorem5.p1.12.12.m12.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="A1.Thmtheorem5.p1.12.12.m12.1a"><msup id="A1.Thmtheorem5.p1.12.12.m12.1.1" xref="A1.Thmtheorem5.p1.12.12.m12.1.1.cmml"><mi id="A1.Thmtheorem5.p1.12.12.m12.1.1.2" xref="A1.Thmtheorem5.p1.12.12.m12.1.1.2.cmml">M</mi><mi id="A1.Thmtheorem5.p1.12.12.m12.1.1.3" xref="A1.Thmtheorem5.p1.12.12.m12.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.12.12.m12.1b"><apply id="A1.Thmtheorem5.p1.12.12.m12.1.1.cmml" xref="A1.Thmtheorem5.p1.12.12.m12.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.12.12.m12.1.1.1.cmml" xref="A1.Thmtheorem5.p1.12.12.m12.1.1">superscript</csymbol><ci id="A1.Thmtheorem5.p1.12.12.m12.1.1.2.cmml" xref="A1.Thmtheorem5.p1.12.12.m12.1.1.2">𝑀</ci><ci id="A1.Thmtheorem5.p1.12.12.m12.1.1.3.cmml" xref="A1.Thmtheorem5.p1.12.12.m12.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.12.12.m12.1c">M^{z}</annotation></semantics></math> can be computed in <math id="A1.Thmtheorem5.p1.13.13.m13.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.Thmtheorem5.p1.13.13.m13.1a"><mi id="A1.Thmtheorem5.p1.13.13.m13.1.1" xref="A1.Thmtheorem5.p1.13.13.m13.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.13.13.m13.1b"><ci id="A1.Thmtheorem5.p1.13.13.m13.1.1.cmml" xref="A1.Thmtheorem5.p1.13.13.m13.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.13.13.m13.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.Thmtheorem5.p1.14.14.m14.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.Thmtheorem5.p1.14.14.m14.1a"><msup id="A1.Thmtheorem5.p1.14.14.m14.1.1" xref="A1.Thmtheorem5.p1.14.14.m14.1.1.cmml"><mi id="A1.Thmtheorem5.p1.14.14.m14.1.1.2" xref="A1.Thmtheorem5.p1.14.14.m14.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.Thmtheorem5.p1.14.14.m14.1.1.3" xref="A1.Thmtheorem5.p1.14.14.m14.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.Thmtheorem5.p1.14.14.m14.1b"><apply id="A1.Thmtheorem5.p1.14.14.m14.1.1.cmml" xref="A1.Thmtheorem5.p1.14.14.m14.1.1"><csymbol cd="ambiguous" id="A1.Thmtheorem5.p1.14.14.m14.1.1.1.cmml" xref="A1.Thmtheorem5.p1.14.14.m14.1.1">superscript</csymbol><ci id="A1.Thmtheorem5.p1.14.14.m14.1.1.2.cmml" xref="A1.Thmtheorem5.p1.14.14.m14.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.Thmtheorem5.p1.14.14.m14.1.1.3.cmml" xref="A1.Thmtheorem5.p1.14.14.m14.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.Thmtheorem5.p1.14.14.m14.1c">\mathsf{TC}^{0}</annotation></semantics></math>.</span></p>
</div>
</div>
<div id="A1.7" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="A1.7.p1" class="ltx_para">
<p id="A1.7.p1.16" class="ltx_p">By preconditions of the lemma, we can compute <math id="A1.7.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}(M)" display="inline"><semantics id="A1.7.p1.1.m1.1a"><mrow id="A1.7.p1.1.m1.1.2" xref="A1.7.p1.1.m1.1.2.cmml"><msub id="A1.7.p1.1.m1.1.2.2" xref="A1.7.p1.1.m1.1.2.2.cmml"><mi id="A1.7.p1.1.m1.1.2.2.2" xref="A1.7.p1.1.m1.1.2.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.7.p1.1.m1.1.2.2.3" xref="A1.7.p1.1.m1.1.2.2.3.cmml">ℝ</mi></msub><mo lspace="0em" rspace="0em" id="A1.7.p1.1.m1.1.2.1" xref="A1.7.p1.1.m1.1.2.1.cmml">​</mo><mrow id="A1.7.p1.1.m1.1.2.3.2" xref="A1.7.p1.1.m1.1.2.cmml"><mo stretchy="false" id="A1.7.p1.1.m1.1.2.3.2.1" xref="A1.7.p1.1.m1.1.2.cmml">(</mo><mi id="A1.7.p1.1.m1.1.1" xref="A1.7.p1.1.m1.1.1.cmml">M</mi><mo stretchy="false" id="A1.7.p1.1.m1.1.2.3.2.2" xref="A1.7.p1.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.7.p1.1.m1.1b"><apply id="A1.7.p1.1.m1.1.2.cmml" xref="A1.7.p1.1.m1.1.2"><times id="A1.7.p1.1.m1.1.2.1.cmml" xref="A1.7.p1.1.m1.1.2.1"></times><apply id="A1.7.p1.1.m1.1.2.2.cmml" xref="A1.7.p1.1.m1.1.2.2"><csymbol cd="ambiguous" id="A1.7.p1.1.m1.1.2.2.1.cmml" xref="A1.7.p1.1.m1.1.2.2">subscript</csymbol><ci id="A1.7.p1.1.m1.1.2.2.2.cmml" xref="A1.7.p1.1.m1.1.2.2.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.7.p1.1.m1.1.2.2.3.cmml" xref="A1.7.p1.1.m1.1.2.2.3">ℝ</ci></apply><ci id="A1.7.p1.1.m1.1.1.cmml" xref="A1.7.p1.1.m1.1.1">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.1.m1.1c">\mathsf{cast}_{\mathbb{R}}(M)</annotation></semantics></math> in <math id="A1.7.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.7.p1.2.m2.1a"><mi id="A1.7.p1.2.m2.1.1" xref="A1.7.p1.2.m2.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.2.m2.1b"><ci id="A1.7.p1.2.m2.1.1.cmml" xref="A1.7.p1.2.m2.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.2.m2.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.7.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.7.p1.3.m3.1a"><msup id="A1.7.p1.3.m3.1.1" xref="A1.7.p1.3.m3.1.1.cmml"><mi id="A1.7.p1.3.m3.1.1.2" xref="A1.7.p1.3.m3.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.7.p1.3.m3.1.1.3" xref="A1.7.p1.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.7.p1.3.m3.1b"><apply id="A1.7.p1.3.m3.1.1.cmml" xref="A1.7.p1.3.m3.1.1"><csymbol cd="ambiguous" id="A1.7.p1.3.m3.1.1.1.cmml" xref="A1.7.p1.3.m3.1.1">superscript</csymbol><ci id="A1.7.p1.3.m3.1.1.2.cmml" xref="A1.7.p1.3.m3.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.7.p1.3.m3.1.1.3.cmml" xref="A1.7.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.3.m3.1c">\mathsf{TC}^{0}</annotation></semantics></math>. Since the entries of <math id="A1.7.p1.4.m4.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.7.p1.4.m4.1a"><mi id="A1.7.p1.4.m4.1.1" xref="A1.7.p1.4.m4.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.4.m4.1b"><ci id="A1.7.p1.4.m4.1.1.cmml" xref="A1.7.p1.4.m4.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.4.m4.1c">M</annotation></semantics></math> are equivalent to <math id="A1.7.p1.5.m5.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.7.p1.5.m5.1a"><mi id="A1.7.p1.5.m5.1.1" xref="A1.7.p1.5.m5.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.5.m5.1b"><ci id="A1.7.p1.5.m5.1.1.cmml" xref="A1.7.p1.5.m5.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.5.m5.1c">n</annotation></semantics></math>-bit integers, <math id="A1.7.p1.6.m6.1" class="ltx_Math" alttext="\mathsf{cast}_{R}(M)" display="inline"><semantics id="A1.7.p1.6.m6.1a"><mrow id="A1.7.p1.6.m6.1.2" xref="A1.7.p1.6.m6.1.2.cmml"><msub id="A1.7.p1.6.m6.1.2.2" xref="A1.7.p1.6.m6.1.2.2.cmml"><mi id="A1.7.p1.6.m6.1.2.2.2" xref="A1.7.p1.6.m6.1.2.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.7.p1.6.m6.1.2.2.3" xref="A1.7.p1.6.m6.1.2.2.3.cmml">R</mi></msub><mo lspace="0em" rspace="0em" id="A1.7.p1.6.m6.1.2.1" xref="A1.7.p1.6.m6.1.2.1.cmml">​</mo><mrow id="A1.7.p1.6.m6.1.2.3.2" xref="A1.7.p1.6.m6.1.2.cmml"><mo stretchy="false" id="A1.7.p1.6.m6.1.2.3.2.1" xref="A1.7.p1.6.m6.1.2.cmml">(</mo><mi id="A1.7.p1.6.m6.1.1" xref="A1.7.p1.6.m6.1.1.cmml">M</mi><mo stretchy="false" id="A1.7.p1.6.m6.1.2.3.2.2" xref="A1.7.p1.6.m6.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.7.p1.6.m6.1b"><apply id="A1.7.p1.6.m6.1.2.cmml" xref="A1.7.p1.6.m6.1.2"><times id="A1.7.p1.6.m6.1.2.1.cmml" xref="A1.7.p1.6.m6.1.2.1"></times><apply id="A1.7.p1.6.m6.1.2.2.cmml" xref="A1.7.p1.6.m6.1.2.2"><csymbol cd="ambiguous" id="A1.7.p1.6.m6.1.2.2.1.cmml" xref="A1.7.p1.6.m6.1.2.2">subscript</csymbol><ci id="A1.7.p1.6.m6.1.2.2.2.cmml" xref="A1.7.p1.6.m6.1.2.2.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.7.p1.6.m6.1.2.2.3.cmml" xref="A1.7.p1.6.m6.1.2.2.3">𝑅</ci></apply><ci id="A1.7.p1.6.m6.1.1.cmml" xref="A1.7.p1.6.m6.1.1">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.6.m6.1c">\mathsf{cast}_{R}(M)</annotation></semantics></math> can be viewed as a <math id="A1.7.p1.7.m7.1" class="ltx_Math" alttext="k\times k" display="inline"><semantics id="A1.7.p1.7.m7.1a"><mrow id="A1.7.p1.7.m7.1.1" xref="A1.7.p1.7.m7.1.1.cmml"><mi id="A1.7.p1.7.m7.1.1.2" xref="A1.7.p1.7.m7.1.1.2.cmml">k</mi><mo lspace="0.222em" rspace="0.222em" id="A1.7.p1.7.m7.1.1.1" xref="A1.7.p1.7.m7.1.1.1.cmml">×</mo><mi id="A1.7.p1.7.m7.1.1.3" xref="A1.7.p1.7.m7.1.1.3.cmml">k</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.7.p1.7.m7.1b"><apply id="A1.7.p1.7.m7.1.1.cmml" xref="A1.7.p1.7.m7.1.1"><times id="A1.7.p1.7.m7.1.1.1.cmml" xref="A1.7.p1.7.m7.1.1.1"></times><ci id="A1.7.p1.7.m7.1.1.2.cmml" xref="A1.7.p1.7.m7.1.1.2">𝑘</ci><ci id="A1.7.p1.7.m7.1.1.3.cmml" xref="A1.7.p1.7.m7.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.7.m7.1c">k\times k</annotation></semantics></math> integer matrix of <math id="A1.7.p1.8.m8.1" class="ltx_Math" alttext="n" display="inline"><semantics id="A1.7.p1.8.m8.1a"><mi id="A1.7.p1.8.m8.1.1" xref="A1.7.p1.8.m8.1.1.cmml">n</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.8.m8.1b"><ci id="A1.7.p1.8.m8.1.1.cmml" xref="A1.7.p1.8.m8.1.1">𝑛</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.8.m8.1c">n</annotation></semantics></math>-bit integers. By <a href="#A1.Thmtheorem4" title="Lemma A.4 (Integer matrix power, derived from Mereghetti &amp; Palano, 2000). ‣ Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">A.4</span></a>, we can compute <math id="A1.7.p1.9.m9.1" class="ltx_Math" alttext="\mathsf{cast}_{R}(M)^{z}" display="inline"><semantics id="A1.7.p1.9.m9.1a"><mrow id="A1.7.p1.9.m9.1.2" xref="A1.7.p1.9.m9.1.2.cmml"><msub id="A1.7.p1.9.m9.1.2.2" xref="A1.7.p1.9.m9.1.2.2.cmml"><mi id="A1.7.p1.9.m9.1.2.2.2" xref="A1.7.p1.9.m9.1.2.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.7.p1.9.m9.1.2.2.3" xref="A1.7.p1.9.m9.1.2.2.3.cmml">R</mi></msub><mo lspace="0em" rspace="0em" id="A1.7.p1.9.m9.1.2.1" xref="A1.7.p1.9.m9.1.2.1.cmml">​</mo><msup id="A1.7.p1.9.m9.1.2.3" xref="A1.7.p1.9.m9.1.2.3.cmml"><mrow id="A1.7.p1.9.m9.1.2.3.2.2" xref="A1.7.p1.9.m9.1.2.3.cmml"><mo stretchy="false" id="A1.7.p1.9.m9.1.2.3.2.2.1" xref="A1.7.p1.9.m9.1.2.3.cmml">(</mo><mi id="A1.7.p1.9.m9.1.1" xref="A1.7.p1.9.m9.1.1.cmml">M</mi><mo stretchy="false" id="A1.7.p1.9.m9.1.2.3.2.2.2" xref="A1.7.p1.9.m9.1.2.3.cmml">)</mo></mrow><mi id="A1.7.p1.9.m9.1.2.3.3" xref="A1.7.p1.9.m9.1.2.3.3.cmml">z</mi></msup></mrow><annotation-xml encoding="MathML-Content" id="A1.7.p1.9.m9.1b"><apply id="A1.7.p1.9.m9.1.2.cmml" xref="A1.7.p1.9.m9.1.2"><times id="A1.7.p1.9.m9.1.2.1.cmml" xref="A1.7.p1.9.m9.1.2.1"></times><apply id="A1.7.p1.9.m9.1.2.2.cmml" xref="A1.7.p1.9.m9.1.2.2"><csymbol cd="ambiguous" id="A1.7.p1.9.m9.1.2.2.1.cmml" xref="A1.7.p1.9.m9.1.2.2">subscript</csymbol><ci id="A1.7.p1.9.m9.1.2.2.2.cmml" xref="A1.7.p1.9.m9.1.2.2.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.7.p1.9.m9.1.2.2.3.cmml" xref="A1.7.p1.9.m9.1.2.2.3">𝑅</ci></apply><apply id="A1.7.p1.9.m9.1.2.3.cmml" xref="A1.7.p1.9.m9.1.2.3"><csymbol cd="ambiguous" id="A1.7.p1.9.m9.1.2.3.1.cmml" xref="A1.7.p1.9.m9.1.2.3">superscript</csymbol><ci id="A1.7.p1.9.m9.1.1.cmml" xref="A1.7.p1.9.m9.1.1">𝑀</ci><ci id="A1.7.p1.9.m9.1.2.3.3.cmml" xref="A1.7.p1.9.m9.1.2.3.3">𝑧</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.9.m9.1c">\mathsf{cast}_{R}(M)^{z}</annotation></semantics></math> using an <math id="A1.7.p1.10.m10.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.7.p1.10.m10.1a"><mi id="A1.7.p1.10.m10.1.1" xref="A1.7.p1.10.m10.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.10.m10.1b"><ci id="A1.7.p1.10.m10.1.1.cmml" xref="A1.7.p1.10.m10.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.10.m10.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.7.p1.11.m11.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.7.p1.11.m11.1a"><msup id="A1.7.p1.11.m11.1.1" xref="A1.7.p1.11.m11.1.1.cmml"><mi id="A1.7.p1.11.m11.1.1.2" xref="A1.7.p1.11.m11.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.7.p1.11.m11.1.1.3" xref="A1.7.p1.11.m11.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.7.p1.11.m11.1b"><apply id="A1.7.p1.11.m11.1.1.cmml" xref="A1.7.p1.11.m11.1.1"><csymbol cd="ambiguous" id="A1.7.p1.11.m11.1.1.1.cmml" xref="A1.7.p1.11.m11.1.1">superscript</csymbol><ci id="A1.7.p1.11.m11.1.1.2.cmml" xref="A1.7.p1.11.m11.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.7.p1.11.m11.1.1.3.cmml" xref="A1.7.p1.11.m11.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.11.m11.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit. Finally, by a precondition of the lemma, we can cast the result back to <math id="A1.7.p1.12.m12.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.7.p1.12.m12.1a"><mi id="A1.7.p1.12.m12.1.1" xref="A1.7.p1.12.m12.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.12.m12.1b"><ci id="A1.7.p1.12.m12.1.1.cmml" xref="A1.7.p1.12.m12.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.12.m12.1c">\mathbb{D}</annotation></semantics></math>, i.e., compute <math id="A1.7.p1.13.m13.2" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{D}}(\mathsf{cast}_{\mathbb{R}}(M)^{z})" display="inline"><semantics id="A1.7.p1.13.m13.2a"><mrow id="A1.7.p1.13.m13.2.2" xref="A1.7.p1.13.m13.2.2.cmml"><msub id="A1.7.p1.13.m13.2.2.3" xref="A1.7.p1.13.m13.2.2.3.cmml"><mi id="A1.7.p1.13.m13.2.2.3.2" xref="A1.7.p1.13.m13.2.2.3.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.7.p1.13.m13.2.2.3.3" xref="A1.7.p1.13.m13.2.2.3.3.cmml">𝔻</mi></msub><mo lspace="0em" rspace="0em" id="A1.7.p1.13.m13.2.2.2" xref="A1.7.p1.13.m13.2.2.2.cmml">​</mo><mrow id="A1.7.p1.13.m13.2.2.1.1" xref="A1.7.p1.13.m13.2.2.1.1.1.cmml"><mo stretchy="false" id="A1.7.p1.13.m13.2.2.1.1.2" xref="A1.7.p1.13.m13.2.2.1.1.1.cmml">(</mo><mrow id="A1.7.p1.13.m13.2.2.1.1.1" xref="A1.7.p1.13.m13.2.2.1.1.1.cmml"><msub id="A1.7.p1.13.m13.2.2.1.1.1.2" xref="A1.7.p1.13.m13.2.2.1.1.1.2.cmml"><mi id="A1.7.p1.13.m13.2.2.1.1.1.2.2" xref="A1.7.p1.13.m13.2.2.1.1.1.2.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.7.p1.13.m13.2.2.1.1.1.2.3" xref="A1.7.p1.13.m13.2.2.1.1.1.2.3.cmml">ℝ</mi></msub><mo lspace="0em" rspace="0em" id="A1.7.p1.13.m13.2.2.1.1.1.1" xref="A1.7.p1.13.m13.2.2.1.1.1.1.cmml">​</mo><msup id="A1.7.p1.13.m13.2.2.1.1.1.3" xref="A1.7.p1.13.m13.2.2.1.1.1.3.cmml"><mrow id="A1.7.p1.13.m13.2.2.1.1.1.3.2.2" xref="A1.7.p1.13.m13.2.2.1.1.1.3.cmml"><mo stretchy="false" id="A1.7.p1.13.m13.2.2.1.1.1.3.2.2.1" xref="A1.7.p1.13.m13.2.2.1.1.1.3.cmml">(</mo><mi id="A1.7.p1.13.m13.1.1" xref="A1.7.p1.13.m13.1.1.cmml">M</mi><mo stretchy="false" id="A1.7.p1.13.m13.2.2.1.1.1.3.2.2.2" xref="A1.7.p1.13.m13.2.2.1.1.1.3.cmml">)</mo></mrow><mi id="A1.7.p1.13.m13.2.2.1.1.1.3.3" xref="A1.7.p1.13.m13.2.2.1.1.1.3.3.cmml">z</mi></msup></mrow><mo stretchy="false" id="A1.7.p1.13.m13.2.2.1.1.3" xref="A1.7.p1.13.m13.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.7.p1.13.m13.2b"><apply id="A1.7.p1.13.m13.2.2.cmml" xref="A1.7.p1.13.m13.2.2"><times id="A1.7.p1.13.m13.2.2.2.cmml" xref="A1.7.p1.13.m13.2.2.2"></times><apply id="A1.7.p1.13.m13.2.2.3.cmml" xref="A1.7.p1.13.m13.2.2.3"><csymbol cd="ambiguous" id="A1.7.p1.13.m13.2.2.3.1.cmml" xref="A1.7.p1.13.m13.2.2.3">subscript</csymbol><ci id="A1.7.p1.13.m13.2.2.3.2.cmml" xref="A1.7.p1.13.m13.2.2.3.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.7.p1.13.m13.2.2.3.3.cmml" xref="A1.7.p1.13.m13.2.2.3.3">𝔻</ci></apply><apply id="A1.7.p1.13.m13.2.2.1.1.1.cmml" xref="A1.7.p1.13.m13.2.2.1.1"><times id="A1.7.p1.13.m13.2.2.1.1.1.1.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.1"></times><apply id="A1.7.p1.13.m13.2.2.1.1.1.2.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.2"><csymbol cd="ambiguous" id="A1.7.p1.13.m13.2.2.1.1.1.2.1.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.2">subscript</csymbol><ci id="A1.7.p1.13.m13.2.2.1.1.1.2.2.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.2.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.7.p1.13.m13.2.2.1.1.1.2.3.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.2.3">ℝ</ci></apply><apply id="A1.7.p1.13.m13.2.2.1.1.1.3.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.3"><csymbol cd="ambiguous" id="A1.7.p1.13.m13.2.2.1.1.1.3.1.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.3">superscript</csymbol><ci id="A1.7.p1.13.m13.1.1.cmml" xref="A1.7.p1.13.m13.1.1">𝑀</ci><ci id="A1.7.p1.13.m13.2.2.1.1.1.3.3.cmml" xref="A1.7.p1.13.m13.2.2.1.1.1.3.3">𝑧</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.13.m13.2c">\mathsf{cast}_{\mathbb{D}}(\mathsf{cast}_{\mathbb{R}}(M)^{z})</annotation></semantics></math> which equals <math id="A1.7.p1.14.m14.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="A1.7.p1.14.m14.1a"><msup id="A1.7.p1.14.m14.1.1" xref="A1.7.p1.14.m14.1.1.cmml"><mi id="A1.7.p1.14.m14.1.1.2" xref="A1.7.p1.14.m14.1.1.2.cmml">M</mi><mi id="A1.7.p1.14.m14.1.1.3" xref="A1.7.p1.14.m14.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.7.p1.14.m14.1b"><apply id="A1.7.p1.14.m14.1.1.cmml" xref="A1.7.p1.14.m14.1.1"><csymbol cd="ambiguous" id="A1.7.p1.14.m14.1.1.1.cmml" xref="A1.7.p1.14.m14.1.1">superscript</csymbol><ci id="A1.7.p1.14.m14.1.1.2.cmml" xref="A1.7.p1.14.m14.1.1.2">𝑀</ci><ci id="A1.7.p1.14.m14.1.1.3.cmml" xref="A1.7.p1.14.m14.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.14.m14.1c">M^{z}</annotation></semantics></math>, with an <math id="A1.7.p1.15.m15.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.7.p1.15.m15.1a"><mi id="A1.7.p1.15.m15.1.1" xref="A1.7.p1.15.m15.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.7.p1.15.m15.1b"><ci id="A1.7.p1.15.m15.1.1.cmml" xref="A1.7.p1.15.m15.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.15.m15.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.7.p1.16.m16.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.7.p1.16.m16.1a"><msup id="A1.7.p1.16.m16.1.1" xref="A1.7.p1.16.m16.1.1.cmml"><mi id="A1.7.p1.16.m16.1.1.2" xref="A1.7.p1.16.m16.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.7.p1.16.m16.1.1.3" xref="A1.7.p1.16.m16.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.7.p1.16.m16.1b"><apply id="A1.7.p1.16.m16.1.1.cmml" xref="A1.7.p1.16.m16.1.1"><csymbol cd="ambiguous" id="A1.7.p1.16.m16.1.1.1.cmml" xref="A1.7.p1.16.m16.1.1">superscript</csymbol><ci id="A1.7.p1.16.m16.1.1.2.cmml" xref="A1.7.p1.16.m16.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.7.p1.16.m16.1.1.3.cmml" xref="A1.7.p1.16.m16.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.7.p1.16.m16.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit.
∎</p>
</div>
</div>
<div id="A1.p6" class="ltx_para">
<p id="A1.p6.1" class="ltx_p">See <a href="#S2.Thmtheorem4" title="Lemma 2.4 (Float matrix power). ‣ 2.2 Numeric Datatype ‣ 2 Background ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.4</span></a></p>
</div>
<div id="A1.12" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="A1.8.p1" class="ltx_para">
<p id="A1.8.p1.2" class="ltx_p">The idea is to convert (by scaling up) <math id="A1.8.p1.1.m1.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.8.p1.1.m1.1a"><mi id="A1.8.p1.1.m1.1.1" xref="A1.8.p1.1.m1.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.8.p1.1.m1.1b"><ci id="A1.8.p1.1.m1.1.1.cmml" xref="A1.8.p1.1.m1.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.8.p1.1.m1.1c">M</annotation></semantics></math> to another float matrix all whose entries are representable as integers, apply <a href="#A1.Thmtheorem5" title="Lemma A.5 (𝔻-matrix power). ‣ Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">A.5</span></a>, reverse the scaling, and cast the result back to <math id="A1.8.p1.2.m2.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.8.p1.2.m2.1a"><mrow id="A1.8.p1.2.m2.1.1" xref="A1.8.p1.2.m2.1.1.cmml"><mi id="A1.8.p1.2.m2.1.1.2" xref="A1.8.p1.2.m2.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.8.p1.2.m2.1.1.1" xref="A1.8.p1.2.m2.1.1.1.cmml">​</mo><mrow id="A1.8.p1.2.m2.1.1.3" xref="A1.8.p1.2.m2.1.1.3.cmml"><mi id="A1.8.p1.2.m2.1.1.3.1" xref="A1.8.p1.2.m2.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.8.p1.2.m2.1.1.3a" xref="A1.8.p1.2.m2.1.1.3.cmml">⁡</mo><mi id="A1.8.p1.2.m2.1.1.3.2" xref="A1.8.p1.2.m2.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.8.p1.2.m2.1b"><apply id="A1.8.p1.2.m2.1.1.cmml" xref="A1.8.p1.2.m2.1.1"><times id="A1.8.p1.2.m2.1.1.1.cmml" xref="A1.8.p1.2.m2.1.1.1"></times><ci id="A1.8.p1.2.m2.1.1.2.cmml" xref="A1.8.p1.2.m2.1.1.2">𝑐</ci><apply id="A1.8.p1.2.m2.1.1.3.cmml" xref="A1.8.p1.2.m2.1.1.3"><log id="A1.8.p1.2.m2.1.1.3.1.cmml" xref="A1.8.p1.2.m2.1.1.3.1"></log><ci id="A1.8.p1.2.m2.1.1.3.2.cmml" xref="A1.8.p1.2.m2.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.8.p1.2.m2.1c">c\log n</annotation></semantics></math> precision floats.</p>
</div>
<div id="A1.9.p2" class="ltx_para">
<p id="A1.9.p2.14" class="ltx_p">Let <math id="A1.9.p2.1.m1.1" class="ltx_Math" alttext="e" display="inline"><semantics id="A1.9.p2.1.m1.1a"><mi id="A1.9.p2.1.m1.1.1" xref="A1.9.p2.1.m1.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.1.m1.1b"><ci id="A1.9.p2.1.m1.1.1.cmml" xref="A1.9.p2.1.m1.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.1.m1.1c">e</annotation></semantics></math> be the smallest exponent across all float entries of <math id="A1.9.p2.2.m2.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.9.p2.2.m2.1a"><mi id="A1.9.p2.2.m2.1.1" xref="A1.9.p2.2.m2.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.2.m2.1b"><ci id="A1.9.p2.2.m2.1.1.cmml" xref="A1.9.p2.2.m2.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.2.m2.1c">M</annotation></semantics></math> and <math id="A1.9.p2.3.m3.3" class="ltx_Math" alttext="q=\max\{0,-e\}" display="inline"><semantics id="A1.9.p2.3.m3.3a"><mrow id="A1.9.p2.3.m3.3.3" xref="A1.9.p2.3.m3.3.3.cmml"><mi id="A1.9.p2.3.m3.3.3.3" xref="A1.9.p2.3.m3.3.3.3.cmml">q</mi><mo id="A1.9.p2.3.m3.3.3.2" xref="A1.9.p2.3.m3.3.3.2.cmml">=</mo><mrow id="A1.9.p2.3.m3.3.3.1.1" xref="A1.9.p2.3.m3.3.3.1.2.cmml"><mi id="A1.9.p2.3.m3.1.1" xref="A1.9.p2.3.m3.1.1.cmml">max</mi><mo id="A1.9.p2.3.m3.3.3.1.1a" xref="A1.9.p2.3.m3.3.3.1.2.cmml">⁡</mo><mrow id="A1.9.p2.3.m3.3.3.1.1.1" xref="A1.9.p2.3.m3.3.3.1.2.cmml"><mo stretchy="false" id="A1.9.p2.3.m3.3.3.1.1.1.2" xref="A1.9.p2.3.m3.3.3.1.2.cmml">{</mo><mn id="A1.9.p2.3.m3.2.2" xref="A1.9.p2.3.m3.2.2.cmml">0</mn><mo id="A1.9.p2.3.m3.3.3.1.1.1.3" xref="A1.9.p2.3.m3.3.3.1.2.cmml">,</mo><mrow id="A1.9.p2.3.m3.3.3.1.1.1.1" xref="A1.9.p2.3.m3.3.3.1.1.1.1.cmml"><mo id="A1.9.p2.3.m3.3.3.1.1.1.1a" xref="A1.9.p2.3.m3.3.3.1.1.1.1.cmml">−</mo><mi id="A1.9.p2.3.m3.3.3.1.1.1.1.2" xref="A1.9.p2.3.m3.3.3.1.1.1.1.2.cmml">e</mi></mrow><mo stretchy="false" id="A1.9.p2.3.m3.3.3.1.1.1.4" xref="A1.9.p2.3.m3.3.3.1.2.cmml">}</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.9.p2.3.m3.3b"><apply id="A1.9.p2.3.m3.3.3.cmml" xref="A1.9.p2.3.m3.3.3"><eq id="A1.9.p2.3.m3.3.3.2.cmml" xref="A1.9.p2.3.m3.3.3.2"></eq><ci id="A1.9.p2.3.m3.3.3.3.cmml" xref="A1.9.p2.3.m3.3.3.3">𝑞</ci><apply id="A1.9.p2.3.m3.3.3.1.2.cmml" xref="A1.9.p2.3.m3.3.3.1.1"><max id="A1.9.p2.3.m3.1.1.cmml" xref="A1.9.p2.3.m3.1.1"></max><cn type="integer" id="A1.9.p2.3.m3.2.2.cmml" xref="A1.9.p2.3.m3.2.2">0</cn><apply id="A1.9.p2.3.m3.3.3.1.1.1.1.cmml" xref="A1.9.p2.3.m3.3.3.1.1.1.1"><minus id="A1.9.p2.3.m3.3.3.1.1.1.1.1.cmml" xref="A1.9.p2.3.m3.3.3.1.1.1.1"></minus><ci id="A1.9.p2.3.m3.3.3.1.1.1.1.2.cmml" xref="A1.9.p2.3.m3.3.3.1.1.1.1.2">𝑒</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.3.m3.3c">q=\max\{0,-e\}</annotation></semantics></math>. Construct a re-scaled float matrix <math id="A1.9.p2.4.m4.1" class="ltx_Math" alttext="\tilde{M}=M2^{q}" display="inline"><semantics id="A1.9.p2.4.m4.1a"><mrow id="A1.9.p2.4.m4.1.1" xref="A1.9.p2.4.m4.1.1.cmml"><mover accent="true" id="A1.9.p2.4.m4.1.1.2" xref="A1.9.p2.4.m4.1.1.2.cmml"><mi id="A1.9.p2.4.m4.1.1.2.2" xref="A1.9.p2.4.m4.1.1.2.2.cmml">M</mi><mo id="A1.9.p2.4.m4.1.1.2.1" xref="A1.9.p2.4.m4.1.1.2.1.cmml">~</mo></mover><mo id="A1.9.p2.4.m4.1.1.1" xref="A1.9.p2.4.m4.1.1.1.cmml">=</mo><mrow id="A1.9.p2.4.m4.1.1.3" xref="A1.9.p2.4.m4.1.1.3.cmml"><mi id="A1.9.p2.4.m4.1.1.3.2" xref="A1.9.p2.4.m4.1.1.3.2.cmml">M</mi><mo lspace="0em" rspace="0em" id="A1.9.p2.4.m4.1.1.3.1" xref="A1.9.p2.4.m4.1.1.3.1.cmml">​</mo><msup id="A1.9.p2.4.m4.1.1.3.3" xref="A1.9.p2.4.m4.1.1.3.3.cmml"><mn id="A1.9.p2.4.m4.1.1.3.3.2" xref="A1.9.p2.4.m4.1.1.3.3.2.cmml">2</mn><mi id="A1.9.p2.4.m4.1.1.3.3.3" xref="A1.9.p2.4.m4.1.1.3.3.3.cmml">q</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.9.p2.4.m4.1b"><apply id="A1.9.p2.4.m4.1.1.cmml" xref="A1.9.p2.4.m4.1.1"><eq id="A1.9.p2.4.m4.1.1.1.cmml" xref="A1.9.p2.4.m4.1.1.1"></eq><apply id="A1.9.p2.4.m4.1.1.2.cmml" xref="A1.9.p2.4.m4.1.1.2"><ci id="A1.9.p2.4.m4.1.1.2.1.cmml" xref="A1.9.p2.4.m4.1.1.2.1">~</ci><ci id="A1.9.p2.4.m4.1.1.2.2.cmml" xref="A1.9.p2.4.m4.1.1.2.2">𝑀</ci></apply><apply id="A1.9.p2.4.m4.1.1.3.cmml" xref="A1.9.p2.4.m4.1.1.3"><times id="A1.9.p2.4.m4.1.1.3.1.cmml" xref="A1.9.p2.4.m4.1.1.3.1"></times><ci id="A1.9.p2.4.m4.1.1.3.2.cmml" xref="A1.9.p2.4.m4.1.1.3.2">𝑀</ci><apply id="A1.9.p2.4.m4.1.1.3.3.cmml" xref="A1.9.p2.4.m4.1.1.3.3"><csymbol cd="ambiguous" id="A1.9.p2.4.m4.1.1.3.3.1.cmml" xref="A1.9.p2.4.m4.1.1.3.3">superscript</csymbol><cn type="integer" id="A1.9.p2.4.m4.1.1.3.3.2.cmml" xref="A1.9.p2.4.m4.1.1.3.3.2">2</cn><ci id="A1.9.p2.4.m4.1.1.3.3.3.cmml" xref="A1.9.p2.4.m4.1.1.3.3.3">𝑞</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.4.m4.1c">\tilde{M}=M2^{q}</annotation></semantics></math> by adding <math id="A1.9.p2.5.m5.1" class="ltx_Math" alttext="q" display="inline"><semantics id="A1.9.p2.5.m5.1a"><mi id="A1.9.p2.5.m5.1.1" xref="A1.9.p2.5.m5.1.1.cmml">q</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.5.m5.1b"><ci id="A1.9.p2.5.m5.1.1.cmml" xref="A1.9.p2.5.m5.1.1">𝑞</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.5.m5.1c">q</annotation></semantics></math> to the exponent of every entry of <math id="A1.9.p2.6.m6.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.9.p2.6.m6.1a"><mi id="A1.9.p2.6.m6.1.1" xref="A1.9.p2.6.m6.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.6.m6.1b"><ci id="A1.9.p2.6.m6.1.1.cmml" xref="A1.9.p2.6.m6.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.6.m6.1c">M</annotation></semantics></math>, using up to <math id="A1.9.p2.7.m7.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.9.p2.7.m7.1a"><mrow id="A1.9.p2.7.m7.1.1" xref="A1.9.p2.7.m7.1.1.cmml"><mi id="A1.9.p2.7.m7.1.1.2" xref="A1.9.p2.7.m7.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.9.p2.7.m7.1.1.1" xref="A1.9.p2.7.m7.1.1.1.cmml">​</mo><mrow id="A1.9.p2.7.m7.1.1.3" xref="A1.9.p2.7.m7.1.1.3.cmml"><mi id="A1.9.p2.7.m7.1.1.3.1" xref="A1.9.p2.7.m7.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.9.p2.7.m7.1.1.3a" xref="A1.9.p2.7.m7.1.1.3.cmml">⁡</mo><mi id="A1.9.p2.7.m7.1.1.3.2" xref="A1.9.p2.7.m7.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.9.p2.7.m7.1b"><apply id="A1.9.p2.7.m7.1.1.cmml" xref="A1.9.p2.7.m7.1.1"><times id="A1.9.p2.7.m7.1.1.1.cmml" xref="A1.9.p2.7.m7.1.1.1"></times><ci id="A1.9.p2.7.m7.1.1.2.cmml" xref="A1.9.p2.7.m7.1.1.2">𝑐</ci><apply id="A1.9.p2.7.m7.1.1.3.cmml" xref="A1.9.p2.7.m7.1.1.3"><log id="A1.9.p2.7.m7.1.1.3.1.cmml" xref="A1.9.p2.7.m7.1.1.3.1"></log><ci id="A1.9.p2.7.m7.1.1.3.2.cmml" xref="A1.9.p2.7.m7.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.7.m7.1c">c\log n</annotation></semantics></math> additional bits in the exponent if necessary to keep the computation exact. Note that <math id="A1.9.p2.8.m8.1" class="ltx_Math" alttext="e" display="inline"><semantics id="A1.9.p2.8.m8.1a"><mi id="A1.9.p2.8.m8.1.1" xref="A1.9.p2.8.m8.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.8.m8.1b"><ci id="A1.9.p2.8.m8.1.1.cmml" xref="A1.9.p2.8.m8.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.8.m8.1c">e</annotation></semantics></math>, <math id="A1.9.p2.9.m9.1" class="ltx_Math" alttext="q" display="inline"><semantics id="A1.9.p2.9.m9.1a"><mi id="A1.9.p2.9.m9.1.1" xref="A1.9.p2.9.m9.1.1.cmml">q</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.9.m9.1b"><ci id="A1.9.p2.9.m9.1.1.cmml" xref="A1.9.p2.9.m9.1.1">𝑞</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.9.m9.1c">q</annotation></semantics></math>, and <math id="A1.9.p2.10.m10.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.9.p2.10.m10.1a"><mover accent="true" id="A1.9.p2.10.m10.1.1" xref="A1.9.p2.10.m10.1.1.cmml"><mi id="A1.9.p2.10.m10.1.1.2" xref="A1.9.p2.10.m10.1.1.2.cmml">M</mi><mo id="A1.9.p2.10.m10.1.1.1" xref="A1.9.p2.10.m10.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.9.p2.10.m10.1b"><apply id="A1.9.p2.10.m10.1.1.cmml" xref="A1.9.p2.10.m10.1.1"><ci id="A1.9.p2.10.m10.1.1.1.cmml" xref="A1.9.p2.10.m10.1.1.1">~</ci><ci id="A1.9.p2.10.m10.1.1.2.cmml" xref="A1.9.p2.10.m10.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.10.m10.1c">\tilde{M}</annotation></semantics></math> can easily be computed exactly by an <math id="A1.9.p2.11.m11.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.9.p2.11.m11.1a"><mi id="A1.9.p2.11.m11.1.1" xref="A1.9.p2.11.m11.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.9.p2.11.m11.1b"><ci id="A1.9.p2.11.m11.1.1.cmml" xref="A1.9.p2.11.m11.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.11.m11.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.9.p2.12.m12.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.9.p2.12.m12.1a"><msup id="A1.9.p2.12.m12.1.1" xref="A1.9.p2.12.m12.1.1.cmml"><mi id="A1.9.p2.12.m12.1.1.2" xref="A1.9.p2.12.m12.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.9.p2.12.m12.1.1.3" xref="A1.9.p2.12.m12.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.9.p2.12.m12.1b"><apply id="A1.9.p2.12.m12.1.1.cmml" xref="A1.9.p2.12.m12.1.1"><csymbol cd="ambiguous" id="A1.9.p2.12.m12.1.1.1.cmml" xref="A1.9.p2.12.m12.1.1">superscript</csymbol><ci id="A1.9.p2.12.m12.1.1.2.cmml" xref="A1.9.p2.12.m12.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.9.p2.12.m12.1.1.3.cmml" xref="A1.9.p2.12.m12.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.12.m12.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuit as they involve fixed-arity arithmetic operations. Further, by construction, <math id="A1.9.p2.13.m13.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.9.p2.13.m13.1a"><mover accent="true" id="A1.9.p2.13.m13.1.1" xref="A1.9.p2.13.m13.1.1.cmml"><mi id="A1.9.p2.13.m13.1.1.2" xref="A1.9.p2.13.m13.1.1.2.cmml">M</mi><mo id="A1.9.p2.13.m13.1.1.1" xref="A1.9.p2.13.m13.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.9.p2.13.m13.1b"><apply id="A1.9.p2.13.m13.1.1.cmml" xref="A1.9.p2.13.m13.1.1"><ci id="A1.9.p2.13.m13.1.1.1.cmml" xref="A1.9.p2.13.m13.1.1.1">~</ci><ci id="A1.9.p2.13.m13.1.1.2.cmml" xref="A1.9.p2.13.m13.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.13.m13.1c">\tilde{M}</annotation></semantics></math> has non-negative exponents in all its float entries. Thus, every entry of <math id="A1.9.p2.14.m14.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.9.p2.14.m14.1a"><mover accent="true" id="A1.9.p2.14.m14.1.1" xref="A1.9.p2.14.m14.1.1.cmml"><mi id="A1.9.p2.14.m14.1.1.2" xref="A1.9.p2.14.m14.1.1.2.cmml">M</mi><mo id="A1.9.p2.14.m14.1.1.1" xref="A1.9.p2.14.m14.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.9.p2.14.m14.1b"><apply id="A1.9.p2.14.m14.1.1.cmml" xref="A1.9.p2.14.m14.1.1"><ci id="A1.9.p2.14.m14.1.1.1.cmml" xref="A1.9.p2.14.m14.1.1.1">~</ci><ci id="A1.9.p2.14.m14.1.1.2.cmml" xref="A1.9.p2.14.m14.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.9.p2.14.m14.1c">\tilde{M}</annotation></semantics></math> represents an integer.</p>
</div>
<div id="A1.10.p3" class="ltx_para">
<p id="A1.10.p3.18" class="ltx_p">The maximum number representable by each <math id="A1.10.p3.1.m1.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.10.p3.1.m1.1a"><mrow id="A1.10.p3.1.m1.1.1" xref="A1.10.p3.1.m1.1.1.cmml"><mi id="A1.10.p3.1.m1.1.1.2" xref="A1.10.p3.1.m1.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.10.p3.1.m1.1.1.1" xref="A1.10.p3.1.m1.1.1.1.cmml">​</mo><mrow id="A1.10.p3.1.m1.1.1.3" xref="A1.10.p3.1.m1.1.1.3.cmml"><mi id="A1.10.p3.1.m1.1.1.3.1" xref="A1.10.p3.1.m1.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.10.p3.1.m1.1.1.3a" xref="A1.10.p3.1.m1.1.1.3.cmml">⁡</mo><mi id="A1.10.p3.1.m1.1.1.3.2" xref="A1.10.p3.1.m1.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.10.p3.1.m1.1b"><apply id="A1.10.p3.1.m1.1.1.cmml" xref="A1.10.p3.1.m1.1.1"><times id="A1.10.p3.1.m1.1.1.1.cmml" xref="A1.10.p3.1.m1.1.1.1"></times><ci id="A1.10.p3.1.m1.1.1.2.cmml" xref="A1.10.p3.1.m1.1.1.2">𝑐</ci><apply id="A1.10.p3.1.m1.1.1.3.cmml" xref="A1.10.p3.1.m1.1.1.3"><log id="A1.10.p3.1.m1.1.1.3.1.cmml" xref="A1.10.p3.1.m1.1.1.3.1"></log><ci id="A1.10.p3.1.m1.1.1.3.2.cmml" xref="A1.10.p3.1.m1.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.1.m1.1c">c\log n</annotation></semantics></math> precision float in <math id="A1.10.p3.2.m2.1" class="ltx_Math" alttext="M" display="inline"><semantics id="A1.10.p3.2.m2.1a"><mi id="A1.10.p3.2.m2.1.1" xref="A1.10.p3.2.m2.1.1.cmml">M</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.2.m2.1b"><ci id="A1.10.p3.2.m2.1.1.cmml" xref="A1.10.p3.2.m2.1.1">𝑀</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.2.m2.1c">M</annotation></semantics></math> is upper bounded by <math id="A1.10.p3.3.m3.1" class="ltx_Math" alttext="2^{n^{c}}" display="inline"><semantics id="A1.10.p3.3.m3.1a"><msup id="A1.10.p3.3.m3.1.1" xref="A1.10.p3.3.m3.1.1.cmml"><mn id="A1.10.p3.3.m3.1.1.2" xref="A1.10.p3.3.m3.1.1.2.cmml">2</mn><msup id="A1.10.p3.3.m3.1.1.3" xref="A1.10.p3.3.m3.1.1.3.cmml"><mi id="A1.10.p3.3.m3.1.1.3.2" xref="A1.10.p3.3.m3.1.1.3.2.cmml">n</mi><mi id="A1.10.p3.3.m3.1.1.3.3" xref="A1.10.p3.3.m3.1.1.3.3.cmml">c</mi></msup></msup><annotation-xml encoding="MathML-Content" id="A1.10.p3.3.m3.1b"><apply id="A1.10.p3.3.m3.1.1.cmml" xref="A1.10.p3.3.m3.1.1"><csymbol cd="ambiguous" id="A1.10.p3.3.m3.1.1.1.cmml" xref="A1.10.p3.3.m3.1.1">superscript</csymbol><cn type="integer" id="A1.10.p3.3.m3.1.1.2.cmml" xref="A1.10.p3.3.m3.1.1.2">2</cn><apply id="A1.10.p3.3.m3.1.1.3.cmml" xref="A1.10.p3.3.m3.1.1.3"><csymbol cd="ambiguous" id="A1.10.p3.3.m3.1.1.3.1.cmml" xref="A1.10.p3.3.m3.1.1.3">superscript</csymbol><ci id="A1.10.p3.3.m3.1.1.3.2.cmml" xref="A1.10.p3.3.m3.1.1.3.2">𝑛</ci><ci id="A1.10.p3.3.m3.1.1.3.3.cmml" xref="A1.10.p3.3.m3.1.1.3.3">𝑐</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.3.m3.1c">2^{n^{c}}</annotation></semantics></math>. Thus, the maximum number representable by each entry of <math id="A1.10.p3.4.m4.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.10.p3.4.m4.1a"><mover accent="true" id="A1.10.p3.4.m4.1.1" xref="A1.10.p3.4.m4.1.1.cmml"><mi id="A1.10.p3.4.m4.1.1.2" xref="A1.10.p3.4.m4.1.1.2.cmml">M</mi><mo id="A1.10.p3.4.m4.1.1.1" xref="A1.10.p3.4.m4.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.10.p3.4.m4.1b"><apply id="A1.10.p3.4.m4.1.1.cmml" xref="A1.10.p3.4.m4.1.1"><ci id="A1.10.p3.4.m4.1.1.1.cmml" xref="A1.10.p3.4.m4.1.1.1">~</ci><ci id="A1.10.p3.4.m4.1.1.2.cmml" xref="A1.10.p3.4.m4.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.4.m4.1c">\tilde{M}</annotation></semantics></math> is <math id="A1.10.p3.5.m5.1" class="ltx_Math" alttext="2^{n^{c}}\times 2^{q}=2^{n^{c}+q}" display="inline"><semantics id="A1.10.p3.5.m5.1a"><mrow id="A1.10.p3.5.m5.1.1" xref="A1.10.p3.5.m5.1.1.cmml"><mrow id="A1.10.p3.5.m5.1.1.2" xref="A1.10.p3.5.m5.1.1.2.cmml"><msup id="A1.10.p3.5.m5.1.1.2.2" xref="A1.10.p3.5.m5.1.1.2.2.cmml"><mn id="A1.10.p3.5.m5.1.1.2.2.2" xref="A1.10.p3.5.m5.1.1.2.2.2.cmml">2</mn><msup id="A1.10.p3.5.m5.1.1.2.2.3" xref="A1.10.p3.5.m5.1.1.2.2.3.cmml"><mi id="A1.10.p3.5.m5.1.1.2.2.3.2" xref="A1.10.p3.5.m5.1.1.2.2.3.2.cmml">n</mi><mi id="A1.10.p3.5.m5.1.1.2.2.3.3" xref="A1.10.p3.5.m5.1.1.2.2.3.3.cmml">c</mi></msup></msup><mo lspace="0.222em" rspace="0.222em" id="A1.10.p3.5.m5.1.1.2.1" xref="A1.10.p3.5.m5.1.1.2.1.cmml">×</mo><msup id="A1.10.p3.5.m5.1.1.2.3" xref="A1.10.p3.5.m5.1.1.2.3.cmml"><mn id="A1.10.p3.5.m5.1.1.2.3.2" xref="A1.10.p3.5.m5.1.1.2.3.2.cmml">2</mn><mi id="A1.10.p3.5.m5.1.1.2.3.3" xref="A1.10.p3.5.m5.1.1.2.3.3.cmml">q</mi></msup></mrow><mo id="A1.10.p3.5.m5.1.1.1" xref="A1.10.p3.5.m5.1.1.1.cmml">=</mo><msup id="A1.10.p3.5.m5.1.1.3" xref="A1.10.p3.5.m5.1.1.3.cmml"><mn id="A1.10.p3.5.m5.1.1.3.2" xref="A1.10.p3.5.m5.1.1.3.2.cmml">2</mn><mrow id="A1.10.p3.5.m5.1.1.3.3" xref="A1.10.p3.5.m5.1.1.3.3.cmml"><msup id="A1.10.p3.5.m5.1.1.3.3.2" xref="A1.10.p3.5.m5.1.1.3.3.2.cmml"><mi id="A1.10.p3.5.m5.1.1.3.3.2.2" xref="A1.10.p3.5.m5.1.1.3.3.2.2.cmml">n</mi><mi id="A1.10.p3.5.m5.1.1.3.3.2.3" xref="A1.10.p3.5.m5.1.1.3.3.2.3.cmml">c</mi></msup><mo id="A1.10.p3.5.m5.1.1.3.3.1" xref="A1.10.p3.5.m5.1.1.3.3.1.cmml">+</mo><mi id="A1.10.p3.5.m5.1.1.3.3.3" xref="A1.10.p3.5.m5.1.1.3.3.3.cmml">q</mi></mrow></msup></mrow><annotation-xml encoding="MathML-Content" id="A1.10.p3.5.m5.1b"><apply id="A1.10.p3.5.m5.1.1.cmml" xref="A1.10.p3.5.m5.1.1"><eq id="A1.10.p3.5.m5.1.1.1.cmml" xref="A1.10.p3.5.m5.1.1.1"></eq><apply id="A1.10.p3.5.m5.1.1.2.cmml" xref="A1.10.p3.5.m5.1.1.2"><times id="A1.10.p3.5.m5.1.1.2.1.cmml" xref="A1.10.p3.5.m5.1.1.2.1"></times><apply id="A1.10.p3.5.m5.1.1.2.2.cmml" xref="A1.10.p3.5.m5.1.1.2.2"><csymbol cd="ambiguous" id="A1.10.p3.5.m5.1.1.2.2.1.cmml" xref="A1.10.p3.5.m5.1.1.2.2">superscript</csymbol><cn type="integer" id="A1.10.p3.5.m5.1.1.2.2.2.cmml" xref="A1.10.p3.5.m5.1.1.2.2.2">2</cn><apply id="A1.10.p3.5.m5.1.1.2.2.3.cmml" xref="A1.10.p3.5.m5.1.1.2.2.3"><csymbol cd="ambiguous" id="A1.10.p3.5.m5.1.1.2.2.3.1.cmml" xref="A1.10.p3.5.m5.1.1.2.2.3">superscript</csymbol><ci id="A1.10.p3.5.m5.1.1.2.2.3.2.cmml" xref="A1.10.p3.5.m5.1.1.2.2.3.2">𝑛</ci><ci id="A1.10.p3.5.m5.1.1.2.2.3.3.cmml" xref="A1.10.p3.5.m5.1.1.2.2.3.3">𝑐</ci></apply></apply><apply id="A1.10.p3.5.m5.1.1.2.3.cmml" xref="A1.10.p3.5.m5.1.1.2.3"><csymbol cd="ambiguous" id="A1.10.p3.5.m5.1.1.2.3.1.cmml" xref="A1.10.p3.5.m5.1.1.2.3">superscript</csymbol><cn type="integer" id="A1.10.p3.5.m5.1.1.2.3.2.cmml" xref="A1.10.p3.5.m5.1.1.2.3.2">2</cn><ci id="A1.10.p3.5.m5.1.1.2.3.3.cmml" xref="A1.10.p3.5.m5.1.1.2.3.3">𝑞</ci></apply></apply><apply id="A1.10.p3.5.m5.1.1.3.cmml" xref="A1.10.p3.5.m5.1.1.3"><csymbol cd="ambiguous" id="A1.10.p3.5.m5.1.1.3.1.cmml" xref="A1.10.p3.5.m5.1.1.3">superscript</csymbol><cn type="integer" id="A1.10.p3.5.m5.1.1.3.2.cmml" xref="A1.10.p3.5.m5.1.1.3.2">2</cn><apply id="A1.10.p3.5.m5.1.1.3.3.cmml" xref="A1.10.p3.5.m5.1.1.3.3"><plus id="A1.10.p3.5.m5.1.1.3.3.1.cmml" xref="A1.10.p3.5.m5.1.1.3.3.1"></plus><apply id="A1.10.p3.5.m5.1.1.3.3.2.cmml" xref="A1.10.p3.5.m5.1.1.3.3.2"><csymbol cd="ambiguous" id="A1.10.p3.5.m5.1.1.3.3.2.1.cmml" xref="A1.10.p3.5.m5.1.1.3.3.2">superscript</csymbol><ci id="A1.10.p3.5.m5.1.1.3.3.2.2.cmml" xref="A1.10.p3.5.m5.1.1.3.3.2.2">𝑛</ci><ci id="A1.10.p3.5.m5.1.1.3.3.2.3.cmml" xref="A1.10.p3.5.m5.1.1.3.3.2.3">𝑐</ci></apply><ci id="A1.10.p3.5.m5.1.1.3.3.3.cmml" xref="A1.10.p3.5.m5.1.1.3.3.3">𝑞</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.5.m5.1c">2^{n^{c}}\times 2^{q}=2^{n^{c}+q}</annotation></semantics></math>. Let <math id="A1.10.p3.6.m6.1" class="ltx_Math" alttext="m=n^{c}+q" display="inline"><semantics id="A1.10.p3.6.m6.1a"><mrow id="A1.10.p3.6.m6.1.1" xref="A1.10.p3.6.m6.1.1.cmml"><mi id="A1.10.p3.6.m6.1.1.2" xref="A1.10.p3.6.m6.1.1.2.cmml">m</mi><mo id="A1.10.p3.6.m6.1.1.1" xref="A1.10.p3.6.m6.1.1.1.cmml">=</mo><mrow id="A1.10.p3.6.m6.1.1.3" xref="A1.10.p3.6.m6.1.1.3.cmml"><msup id="A1.10.p3.6.m6.1.1.3.2" xref="A1.10.p3.6.m6.1.1.3.2.cmml"><mi id="A1.10.p3.6.m6.1.1.3.2.2" xref="A1.10.p3.6.m6.1.1.3.2.2.cmml">n</mi><mi id="A1.10.p3.6.m6.1.1.3.2.3" xref="A1.10.p3.6.m6.1.1.3.2.3.cmml">c</mi></msup><mo id="A1.10.p3.6.m6.1.1.3.1" xref="A1.10.p3.6.m6.1.1.3.1.cmml">+</mo><mi id="A1.10.p3.6.m6.1.1.3.3" xref="A1.10.p3.6.m6.1.1.3.3.cmml">q</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.10.p3.6.m6.1b"><apply id="A1.10.p3.6.m6.1.1.cmml" xref="A1.10.p3.6.m6.1.1"><eq id="A1.10.p3.6.m6.1.1.1.cmml" xref="A1.10.p3.6.m6.1.1.1"></eq><ci id="A1.10.p3.6.m6.1.1.2.cmml" xref="A1.10.p3.6.m6.1.1.2">𝑚</ci><apply id="A1.10.p3.6.m6.1.1.3.cmml" xref="A1.10.p3.6.m6.1.1.3"><plus id="A1.10.p3.6.m6.1.1.3.1.cmml" xref="A1.10.p3.6.m6.1.1.3.1"></plus><apply id="A1.10.p3.6.m6.1.1.3.2.cmml" xref="A1.10.p3.6.m6.1.1.3.2"><csymbol cd="ambiguous" id="A1.10.p3.6.m6.1.1.3.2.1.cmml" xref="A1.10.p3.6.m6.1.1.3.2">superscript</csymbol><ci id="A1.10.p3.6.m6.1.1.3.2.2.cmml" xref="A1.10.p3.6.m6.1.1.3.2.2">𝑛</ci><ci id="A1.10.p3.6.m6.1.1.3.2.3.cmml" xref="A1.10.p3.6.m6.1.1.3.2.3">𝑐</ci></apply><ci id="A1.10.p3.6.m6.1.1.3.3.cmml" xref="A1.10.p3.6.m6.1.1.3.3">𝑞</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.6.m6.1c">m=n^{c}+q</annotation></semantics></math>. It follows that each entry <math id="A1.10.p3.7.m7.1" class="ltx_Math" alttext="\phi" display="inline"><semantics id="A1.10.p3.7.m7.1a"><mi id="A1.10.p3.7.m7.1.1" xref="A1.10.p3.7.m7.1.1.cmml">ϕ</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.7.m7.1b"><ci id="A1.10.p3.7.m7.1.1.cmml" xref="A1.10.p3.7.m7.1.1">italic-ϕ</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.7.m7.1c">\phi</annotation></semantics></math> of <math id="A1.10.p3.8.m8.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.10.p3.8.m8.1a"><mover accent="true" id="A1.10.p3.8.m8.1.1" xref="A1.10.p3.8.m8.1.1.cmml"><mi id="A1.10.p3.8.m8.1.1.2" xref="A1.10.p3.8.m8.1.1.2.cmml">M</mi><mo id="A1.10.p3.8.m8.1.1.1" xref="A1.10.p3.8.m8.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.10.p3.8.m8.1b"><apply id="A1.10.p3.8.m8.1.1.cmml" xref="A1.10.p3.8.m8.1.1"><ci id="A1.10.p3.8.m8.1.1.1.cmml" xref="A1.10.p3.8.m8.1.1.1">~</ci><ci id="A1.10.p3.8.m8.1.1.2.cmml" xref="A1.10.p3.8.m8.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.8.m8.1c">\tilde{M}</annotation></semantics></math> can be equivalently represented as an <math id="A1.10.p3.9.m9.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.10.p3.9.m9.1a"><mi id="A1.10.p3.9.m9.1.1" xref="A1.10.p3.9.m9.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.9.m9.1b"><ci id="A1.10.p3.9.m9.1.1.cmml" xref="A1.10.p3.9.m9.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.9.m9.1c">m</annotation></semantics></math>-bit integer. Further, this integer can be computed by left-shifting the mantissa of <math id="A1.10.p3.10.m10.1" class="ltx_Math" alttext="\phi" display="inline"><semantics id="A1.10.p3.10.m10.1a"><mi id="A1.10.p3.10.m10.1.1" xref="A1.10.p3.10.m10.1.1.cmml">ϕ</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.10.m10.1b"><ci id="A1.10.p3.10.m10.1.1.cmml" xref="A1.10.p3.10.m10.1.1">italic-ϕ</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.10.m10.1c">\phi</annotation></semantics></math> by a number of bits equal to the value of the exponent of <math id="A1.10.p3.11.m11.1" class="ltx_Math" alttext="\phi" display="inline"><semantics id="A1.10.p3.11.m11.1a"><mi id="A1.10.p3.11.m11.1.1" xref="A1.10.p3.11.m11.1.1.cmml">ϕ</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.11.m11.1b"><ci id="A1.10.p3.11.m11.1.1.cmml" xref="A1.10.p3.11.m11.1.1">italic-ϕ</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.11.m11.1c">\phi</annotation></semantics></math> (which is non-negative). Finally, this left-shift, and thus the <math id="A1.10.p3.12.m12.1" class="ltx_Math" alttext="\mathsf{cast}_{\mathbb{R}}" display="inline"><semantics id="A1.10.p3.12.m12.1a"><msub id="A1.10.p3.12.m12.1.1" xref="A1.10.p3.12.m12.1.1.cmml"><mi id="A1.10.p3.12.m12.1.1.2" xref="A1.10.p3.12.m12.1.1.2.cmml">𝖼𝖺𝗌𝗍</mi><mi id="A1.10.p3.12.m12.1.1.3" xref="A1.10.p3.12.m12.1.1.3.cmml">ℝ</mi></msub><annotation-xml encoding="MathML-Content" id="A1.10.p3.12.m12.1b"><apply id="A1.10.p3.12.m12.1.1.cmml" xref="A1.10.p3.12.m12.1.1"><csymbol cd="ambiguous" id="A1.10.p3.12.m12.1.1.1.cmml" xref="A1.10.p3.12.m12.1.1">subscript</csymbol><ci id="A1.10.p3.12.m12.1.1.2.cmml" xref="A1.10.p3.12.m12.1.1.2">𝖼𝖺𝗌𝗍</ci><ci id="A1.10.p3.12.m12.1.1.3.cmml" xref="A1.10.p3.12.m12.1.1.3">ℝ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.12.m12.1c">\mathsf{cast}_{\mathbb{R}}</annotation></semantics></math> operation over <math id="A1.10.p3.13.m13.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.10.p3.13.m13.1a"><mi id="A1.10.p3.13.m13.1.1" xref="A1.10.p3.13.m13.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.13.m13.1b"><ci id="A1.10.p3.13.m13.1.1.cmml" xref="A1.10.p3.13.m13.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.13.m13.1c">m</annotation></semantics></math>-precision floats, can be easily computed by an <math id="A1.10.p3.14.m14.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.10.p3.14.m14.1a"><mi id="A1.10.p3.14.m14.1.1" xref="A1.10.p3.14.m14.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.14.m14.1b"><ci id="A1.10.p3.14.m14.1.1.cmml" xref="A1.10.p3.14.m14.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.14.m14.1c">\mathsf{L}</annotation></semantics></math>-uniform threshold circuit of size <math id="A1.10.p3.15.m15.1" class="ltx_Math" alttext="\mathrm{poly}(m)" display="inline"><semantics id="A1.10.p3.15.m15.1a"><mrow id="A1.10.p3.15.m15.1.2" xref="A1.10.p3.15.m15.1.2.cmml"><mi id="A1.10.p3.15.m15.1.2.2" xref="A1.10.p3.15.m15.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.10.p3.15.m15.1.2.1" xref="A1.10.p3.15.m15.1.2.1.cmml">​</mo><mrow id="A1.10.p3.15.m15.1.2.3.2" xref="A1.10.p3.15.m15.1.2.cmml"><mo stretchy="false" id="A1.10.p3.15.m15.1.2.3.2.1" xref="A1.10.p3.15.m15.1.2.cmml">(</mo><mi id="A1.10.p3.15.m15.1.1" xref="A1.10.p3.15.m15.1.1.cmml">m</mi><mo stretchy="false" id="A1.10.p3.15.m15.1.2.3.2.2" xref="A1.10.p3.15.m15.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.10.p3.15.m15.1b"><apply id="A1.10.p3.15.m15.1.2.cmml" xref="A1.10.p3.15.m15.1.2"><times id="A1.10.p3.15.m15.1.2.1.cmml" xref="A1.10.p3.15.m15.1.2.1"></times><ci id="A1.10.p3.15.m15.1.2.2.cmml" xref="A1.10.p3.15.m15.1.2.2">poly</ci><ci id="A1.10.p3.15.m15.1.1.cmml" xref="A1.10.p3.15.m15.1.1">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.15.m15.1c">\mathrm{poly}(m)</annotation></semantics></math>. In the other direction, casting from reals to <math id="A1.10.p3.16.m16.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.10.p3.16.m16.1a"><mi id="A1.10.p3.16.m16.1.1" xref="A1.10.p3.16.m16.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.16.m16.1b"><ci id="A1.10.p3.16.m16.1.1.cmml" xref="A1.10.p3.16.m16.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.16.m16.1c">m</annotation></semantics></math>-precision floats can also be easily accomplished by an <math id="A1.10.p3.17.m17.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.10.p3.17.m17.1a"><mi id="A1.10.p3.17.m17.1.1" xref="A1.10.p3.17.m17.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.10.p3.17.m17.1b"><ci id="A1.10.p3.17.m17.1.1.cmml" xref="A1.10.p3.17.m17.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.17.m17.1c">\mathsf{L}</annotation></semantics></math>-uniform threshold circuit of size <math id="A1.10.p3.18.m18.1" class="ltx_Math" alttext="\mathrm{poly}(m)" display="inline"><semantics id="A1.10.p3.18.m18.1a"><mrow id="A1.10.p3.18.m18.1.2" xref="A1.10.p3.18.m18.1.2.cmml"><mi id="A1.10.p3.18.m18.1.2.2" xref="A1.10.p3.18.m18.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.10.p3.18.m18.1.2.1" xref="A1.10.p3.18.m18.1.2.1.cmml">​</mo><mrow id="A1.10.p3.18.m18.1.2.3.2" xref="A1.10.p3.18.m18.1.2.cmml"><mo stretchy="false" id="A1.10.p3.18.m18.1.2.3.2.1" xref="A1.10.p3.18.m18.1.2.cmml">(</mo><mi id="A1.10.p3.18.m18.1.1" xref="A1.10.p3.18.m18.1.1.cmml">m</mi><mo stretchy="false" id="A1.10.p3.18.m18.1.2.3.2.2" xref="A1.10.p3.18.m18.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.10.p3.18.m18.1b"><apply id="A1.10.p3.18.m18.1.2.cmml" xref="A1.10.p3.18.m18.1.2"><times id="A1.10.p3.18.m18.1.2.1.cmml" xref="A1.10.p3.18.m18.1.2.1"></times><ci id="A1.10.p3.18.m18.1.2.2.cmml" xref="A1.10.p3.18.m18.1.2.2">poly</ci><ci id="A1.10.p3.18.m18.1.1.cmml" xref="A1.10.p3.18.m18.1.1">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.10.p3.18.m18.1c">\mathrm{poly}(m)</annotation></semantics></math>.</p>
</div>
<div id="A1.11.p4" class="ltx_para">
<p id="A1.11.p4.13" class="ltx_p">Note that <math id="A1.11.p4.1.m1.2" class="ltx_Math" alttext="2^{q}\in[0,n^{c}]" display="inline"><semantics id="A1.11.p4.1.m1.2a"><mrow id="A1.11.p4.1.m1.2.2" xref="A1.11.p4.1.m1.2.2.cmml"><msup id="A1.11.p4.1.m1.2.2.3" xref="A1.11.p4.1.m1.2.2.3.cmml"><mn id="A1.11.p4.1.m1.2.2.3.2" xref="A1.11.p4.1.m1.2.2.3.2.cmml">2</mn><mi id="A1.11.p4.1.m1.2.2.3.3" xref="A1.11.p4.1.m1.2.2.3.3.cmml">q</mi></msup><mo id="A1.11.p4.1.m1.2.2.2" xref="A1.11.p4.1.m1.2.2.2.cmml">∈</mo><mrow id="A1.11.p4.1.m1.2.2.1.1" xref="A1.11.p4.1.m1.2.2.1.2.cmml"><mo stretchy="false" id="A1.11.p4.1.m1.2.2.1.1.2" xref="A1.11.p4.1.m1.2.2.1.2.cmml">[</mo><mn id="A1.11.p4.1.m1.1.1" xref="A1.11.p4.1.m1.1.1.cmml">0</mn><mo id="A1.11.p4.1.m1.2.2.1.1.3" xref="A1.11.p4.1.m1.2.2.1.2.cmml">,</mo><msup id="A1.11.p4.1.m1.2.2.1.1.1" xref="A1.11.p4.1.m1.2.2.1.1.1.cmml"><mi id="A1.11.p4.1.m1.2.2.1.1.1.2" xref="A1.11.p4.1.m1.2.2.1.1.1.2.cmml">n</mi><mi id="A1.11.p4.1.m1.2.2.1.1.1.3" xref="A1.11.p4.1.m1.2.2.1.1.1.3.cmml">c</mi></msup><mo stretchy="false" id="A1.11.p4.1.m1.2.2.1.1.4" xref="A1.11.p4.1.m1.2.2.1.2.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.1.m1.2b"><apply id="A1.11.p4.1.m1.2.2.cmml" xref="A1.11.p4.1.m1.2.2"><in id="A1.11.p4.1.m1.2.2.2.cmml" xref="A1.11.p4.1.m1.2.2.2"></in><apply id="A1.11.p4.1.m1.2.2.3.cmml" xref="A1.11.p4.1.m1.2.2.3"><csymbol cd="ambiguous" id="A1.11.p4.1.m1.2.2.3.1.cmml" xref="A1.11.p4.1.m1.2.2.3">superscript</csymbol><cn type="integer" id="A1.11.p4.1.m1.2.2.3.2.cmml" xref="A1.11.p4.1.m1.2.2.3.2">2</cn><ci id="A1.11.p4.1.m1.2.2.3.3.cmml" xref="A1.11.p4.1.m1.2.2.3.3">𝑞</ci></apply><interval closure="closed" id="A1.11.p4.1.m1.2.2.1.2.cmml" xref="A1.11.p4.1.m1.2.2.1.1"><cn type="integer" id="A1.11.p4.1.m1.1.1.cmml" xref="A1.11.p4.1.m1.1.1">0</cn><apply id="A1.11.p4.1.m1.2.2.1.1.1.cmml" xref="A1.11.p4.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="A1.11.p4.1.m1.2.2.1.1.1.1.cmml" xref="A1.11.p4.1.m1.2.2.1.1.1">superscript</csymbol><ci id="A1.11.p4.1.m1.2.2.1.1.1.2.cmml" xref="A1.11.p4.1.m1.2.2.1.1.1.2">𝑛</ci><ci id="A1.11.p4.1.m1.2.2.1.1.1.3.cmml" xref="A1.11.p4.1.m1.2.2.1.1.1.3">𝑐</ci></apply></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.1.m1.2c">2^{q}\in[0,n^{c}]</annotation></semantics></math> and hence <math id="A1.11.p4.2.m2.2" class="ltx_Math" alttext="m\in[n^{c},2n^{c}]" display="inline"><semantics id="A1.11.p4.2.m2.2a"><mrow id="A1.11.p4.2.m2.2.2" xref="A1.11.p4.2.m2.2.2.cmml"><mi id="A1.11.p4.2.m2.2.2.4" xref="A1.11.p4.2.m2.2.2.4.cmml">m</mi><mo id="A1.11.p4.2.m2.2.2.3" xref="A1.11.p4.2.m2.2.2.3.cmml">∈</mo><mrow id="A1.11.p4.2.m2.2.2.2.2" xref="A1.11.p4.2.m2.2.2.2.3.cmml"><mo stretchy="false" id="A1.11.p4.2.m2.2.2.2.2.3" xref="A1.11.p4.2.m2.2.2.2.3.cmml">[</mo><msup id="A1.11.p4.2.m2.1.1.1.1.1" xref="A1.11.p4.2.m2.1.1.1.1.1.cmml"><mi id="A1.11.p4.2.m2.1.1.1.1.1.2" xref="A1.11.p4.2.m2.1.1.1.1.1.2.cmml">n</mi><mi id="A1.11.p4.2.m2.1.1.1.1.1.3" xref="A1.11.p4.2.m2.1.1.1.1.1.3.cmml">c</mi></msup><mo id="A1.11.p4.2.m2.2.2.2.2.4" xref="A1.11.p4.2.m2.2.2.2.3.cmml">,</mo><mrow id="A1.11.p4.2.m2.2.2.2.2.2" xref="A1.11.p4.2.m2.2.2.2.2.2.cmml"><mn id="A1.11.p4.2.m2.2.2.2.2.2.2" xref="A1.11.p4.2.m2.2.2.2.2.2.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A1.11.p4.2.m2.2.2.2.2.2.1" xref="A1.11.p4.2.m2.2.2.2.2.2.1.cmml">​</mo><msup id="A1.11.p4.2.m2.2.2.2.2.2.3" xref="A1.11.p4.2.m2.2.2.2.2.2.3.cmml"><mi id="A1.11.p4.2.m2.2.2.2.2.2.3.2" xref="A1.11.p4.2.m2.2.2.2.2.2.3.2.cmml">n</mi><mi id="A1.11.p4.2.m2.2.2.2.2.2.3.3" xref="A1.11.p4.2.m2.2.2.2.2.2.3.3.cmml">c</mi></msup></mrow><mo stretchy="false" id="A1.11.p4.2.m2.2.2.2.2.5" xref="A1.11.p4.2.m2.2.2.2.3.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.2.m2.2b"><apply id="A1.11.p4.2.m2.2.2.cmml" xref="A1.11.p4.2.m2.2.2"><in id="A1.11.p4.2.m2.2.2.3.cmml" xref="A1.11.p4.2.m2.2.2.3"></in><ci id="A1.11.p4.2.m2.2.2.4.cmml" xref="A1.11.p4.2.m2.2.2.4">𝑚</ci><interval closure="closed" id="A1.11.p4.2.m2.2.2.2.3.cmml" xref="A1.11.p4.2.m2.2.2.2.2"><apply id="A1.11.p4.2.m2.1.1.1.1.1.cmml" xref="A1.11.p4.2.m2.1.1.1.1.1"><csymbol cd="ambiguous" id="A1.11.p4.2.m2.1.1.1.1.1.1.cmml" xref="A1.11.p4.2.m2.1.1.1.1.1">superscript</csymbol><ci id="A1.11.p4.2.m2.1.1.1.1.1.2.cmml" xref="A1.11.p4.2.m2.1.1.1.1.1.2">𝑛</ci><ci id="A1.11.p4.2.m2.1.1.1.1.1.3.cmml" xref="A1.11.p4.2.m2.1.1.1.1.1.3">𝑐</ci></apply><apply id="A1.11.p4.2.m2.2.2.2.2.2.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2"><times id="A1.11.p4.2.m2.2.2.2.2.2.1.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2.1"></times><cn type="integer" id="A1.11.p4.2.m2.2.2.2.2.2.2.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2.2">2</cn><apply id="A1.11.p4.2.m2.2.2.2.2.2.3.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2.3"><csymbol cd="ambiguous" id="A1.11.p4.2.m2.2.2.2.2.2.3.1.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2.3">superscript</csymbol><ci id="A1.11.p4.2.m2.2.2.2.2.2.3.2.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2.3.2">𝑛</ci><ci id="A1.11.p4.2.m2.2.2.2.2.2.3.3.cmml" xref="A1.11.p4.2.m2.2.2.2.2.2.3.3">𝑐</ci></apply></apply></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.2.m2.2c">m\in[n^{c},2n^{c}]</annotation></semantics></math>. In particular, <math id="A1.11.p4.3.m3.1" class="ltx_Math" alttext="m\geq n" display="inline"><semantics id="A1.11.p4.3.m3.1a"><mrow id="A1.11.p4.3.m3.1.1" xref="A1.11.p4.3.m3.1.1.cmml"><mi id="A1.11.p4.3.m3.1.1.2" xref="A1.11.p4.3.m3.1.1.2.cmml">m</mi><mo id="A1.11.p4.3.m3.1.1.1" xref="A1.11.p4.3.m3.1.1.1.cmml">≥</mo><mi id="A1.11.p4.3.m3.1.1.3" xref="A1.11.p4.3.m3.1.1.3.cmml">n</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.3.m3.1b"><apply id="A1.11.p4.3.m3.1.1.cmml" xref="A1.11.p4.3.m3.1.1"><geq id="A1.11.p4.3.m3.1.1.1.cmml" xref="A1.11.p4.3.m3.1.1.1"></geq><ci id="A1.11.p4.3.m3.1.1.2.cmml" xref="A1.11.p4.3.m3.1.1.2">𝑚</ci><ci id="A1.11.p4.3.m3.1.1.3.cmml" xref="A1.11.p4.3.m3.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.3.m3.1c">m\geq n</annotation></semantics></math>. Thus <math id="A1.11.p4.4.m4.1" class="ltx_Math" alttext="z\leq n" display="inline"><semantics id="A1.11.p4.4.m4.1a"><mrow id="A1.11.p4.4.m4.1.1" xref="A1.11.p4.4.m4.1.1.cmml"><mi id="A1.11.p4.4.m4.1.1.2" xref="A1.11.p4.4.m4.1.1.2.cmml">z</mi><mo id="A1.11.p4.4.m4.1.1.1" xref="A1.11.p4.4.m4.1.1.1.cmml">≤</mo><mi id="A1.11.p4.4.m4.1.1.3" xref="A1.11.p4.4.m4.1.1.3.cmml">n</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.4.m4.1b"><apply id="A1.11.p4.4.m4.1.1.cmml" xref="A1.11.p4.4.m4.1.1"><leq id="A1.11.p4.4.m4.1.1.1.cmml" xref="A1.11.p4.4.m4.1.1.1"></leq><ci id="A1.11.p4.4.m4.1.1.2.cmml" xref="A1.11.p4.4.m4.1.1.2">𝑧</ci><ci id="A1.11.p4.4.m4.1.1.3.cmml" xref="A1.11.p4.4.m4.1.1.3">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.4.m4.1c">z\leq n</annotation></semantics></math> (a precision) implies <math id="A1.11.p4.5.m5.1" class="ltx_Math" alttext="z\leq m" display="inline"><semantics id="A1.11.p4.5.m5.1a"><mrow id="A1.11.p4.5.m5.1.1" xref="A1.11.p4.5.m5.1.1.cmml"><mi id="A1.11.p4.5.m5.1.1.2" xref="A1.11.p4.5.m5.1.1.2.cmml">z</mi><mo id="A1.11.p4.5.m5.1.1.1" xref="A1.11.p4.5.m5.1.1.1.cmml">≤</mo><mi id="A1.11.p4.5.m5.1.1.3" xref="A1.11.p4.5.m5.1.1.3.cmml">m</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.5.m5.1b"><apply id="A1.11.p4.5.m5.1.1.cmml" xref="A1.11.p4.5.m5.1.1"><leq id="A1.11.p4.5.m5.1.1.1.cmml" xref="A1.11.p4.5.m5.1.1.1"></leq><ci id="A1.11.p4.5.m5.1.1.2.cmml" xref="A1.11.p4.5.m5.1.1.2">𝑧</ci><ci id="A1.11.p4.5.m5.1.1.3.cmml" xref="A1.11.p4.5.m5.1.1.3">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.5.m5.1c">z\leq m</annotation></semantics></math>. Observing that <math id="A1.11.p4.6.m6.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.11.p4.6.m6.1a"><mover accent="true" id="A1.11.p4.6.m6.1.1" xref="A1.11.p4.6.m6.1.1.cmml"><mi id="A1.11.p4.6.m6.1.1.2" xref="A1.11.p4.6.m6.1.1.2.cmml">M</mi><mo id="A1.11.p4.6.m6.1.1.1" xref="A1.11.p4.6.m6.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.11.p4.6.m6.1b"><apply id="A1.11.p4.6.m6.1.1.cmml" xref="A1.11.p4.6.m6.1.1"><ci id="A1.11.p4.6.m6.1.1.1.cmml" xref="A1.11.p4.6.m6.1.1.1">~</ci><ci id="A1.11.p4.6.m6.1.1.2.cmml" xref="A1.11.p4.6.m6.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.6.m6.1c">\tilde{M}</annotation></semantics></math> is a matrix of
floats each representable as an <math id="A1.11.p4.7.m7.1" class="ltx_Math" alttext="m" display="inline"><semantics id="A1.11.p4.7.m7.1a"><mi id="A1.11.p4.7.m7.1.1" xref="A1.11.p4.7.m7.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="A1.11.p4.7.m7.1b"><ci id="A1.11.p4.7.m7.1.1.cmml" xref="A1.11.p4.7.m7.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.7.m7.1c">m</annotation></semantics></math>-bit integer,
we now apply <a href="#A1.Thmtheorem5" title="Lemma A.5 (𝔻-matrix power). ‣ Appendix A Floating-Point Arithmetic ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Lemma</span> <span class="ltx_text ltx_ref_tag">A.5</span></a> with <math id="A1.11.p4.8.m8.1" class="ltx_Math" alttext="\mathbb{D}" display="inline"><semantics id="A1.11.p4.8.m8.1a"><mi id="A1.11.p4.8.m8.1.1" xref="A1.11.p4.8.m8.1.1.cmml">𝔻</mi><annotation-xml encoding="MathML-Content" id="A1.11.p4.8.m8.1b"><ci id="A1.11.p4.8.m8.1.1.cmml" xref="A1.11.p4.8.m8.1.1">𝔻</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.8.m8.1c">\mathbb{D}</annotation></semantics></math> being ‘float’ to conclude that float matrix power <math id="A1.11.p4.9.m9.1" class="ltx_Math" alttext="\tilde{M}^{z}" display="inline"><semantics id="A1.11.p4.9.m9.1a"><msup id="A1.11.p4.9.m9.1.1" xref="A1.11.p4.9.m9.1.1.cmml"><mover accent="true" id="A1.11.p4.9.m9.1.1.2" xref="A1.11.p4.9.m9.1.1.2.cmml"><mi id="A1.11.p4.9.m9.1.1.2.2" xref="A1.11.p4.9.m9.1.1.2.2.cmml">M</mi><mo id="A1.11.p4.9.m9.1.1.2.1" xref="A1.11.p4.9.m9.1.1.2.1.cmml">~</mo></mover><mi id="A1.11.p4.9.m9.1.1.3" xref="A1.11.p4.9.m9.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.11.p4.9.m9.1b"><apply id="A1.11.p4.9.m9.1.1.cmml" xref="A1.11.p4.9.m9.1.1"><csymbol cd="ambiguous" id="A1.11.p4.9.m9.1.1.1.cmml" xref="A1.11.p4.9.m9.1.1">superscript</csymbol><apply id="A1.11.p4.9.m9.1.1.2.cmml" xref="A1.11.p4.9.m9.1.1.2"><ci id="A1.11.p4.9.m9.1.1.2.1.cmml" xref="A1.11.p4.9.m9.1.1.2.1">~</ci><ci id="A1.11.p4.9.m9.1.1.2.2.cmml" xref="A1.11.p4.9.m9.1.1.2.2">𝑀</ci></apply><ci id="A1.11.p4.9.m9.1.1.3.cmml" xref="A1.11.p4.9.m9.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.9.m9.1c">\tilde{M}^{z}</annotation></semantics></math> can be computed by an <math id="A1.11.p4.10.m10.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.11.p4.10.m10.1a"><mi id="A1.11.p4.10.m10.1.1" xref="A1.11.p4.10.m10.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.11.p4.10.m10.1b"><ci id="A1.11.p4.10.m10.1.1.cmml" xref="A1.11.p4.10.m10.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.10.m10.1c">\mathsf{L}</annotation></semantics></math>-uniform threshold circuit of size <math id="A1.11.p4.11.m11.1" class="ltx_Math" alttext="\mathrm{poly}(m)" display="inline"><semantics id="A1.11.p4.11.m11.1a"><mrow id="A1.11.p4.11.m11.1.2" xref="A1.11.p4.11.m11.1.2.cmml"><mi id="A1.11.p4.11.m11.1.2.2" xref="A1.11.p4.11.m11.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.11.p4.11.m11.1.2.1" xref="A1.11.p4.11.m11.1.2.1.cmml">​</mo><mrow id="A1.11.p4.11.m11.1.2.3.2" xref="A1.11.p4.11.m11.1.2.cmml"><mo stretchy="false" id="A1.11.p4.11.m11.1.2.3.2.1" xref="A1.11.p4.11.m11.1.2.cmml">(</mo><mi id="A1.11.p4.11.m11.1.1" xref="A1.11.p4.11.m11.1.1.cmml">m</mi><mo stretchy="false" id="A1.11.p4.11.m11.1.2.3.2.2" xref="A1.11.p4.11.m11.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.11.m11.1b"><apply id="A1.11.p4.11.m11.1.2.cmml" xref="A1.11.p4.11.m11.1.2"><times id="A1.11.p4.11.m11.1.2.1.cmml" xref="A1.11.p4.11.m11.1.2.1"></times><ci id="A1.11.p4.11.m11.1.2.2.cmml" xref="A1.11.p4.11.m11.1.2.2">poly</ci><ci id="A1.11.p4.11.m11.1.1.cmml" xref="A1.11.p4.11.m11.1.1">𝑚</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.11.m11.1c">\mathrm{poly}(m)</annotation></semantics></math>. Since
<math id="A1.11.p4.12.m12.1" class="ltx_Math" alttext="m\leq 2n^{c}" display="inline"><semantics id="A1.11.p4.12.m12.1a"><mrow id="A1.11.p4.12.m12.1.1" xref="A1.11.p4.12.m12.1.1.cmml"><mi id="A1.11.p4.12.m12.1.1.2" xref="A1.11.p4.12.m12.1.1.2.cmml">m</mi><mo id="A1.11.p4.12.m12.1.1.1" xref="A1.11.p4.12.m12.1.1.1.cmml">≤</mo><mrow id="A1.11.p4.12.m12.1.1.3" xref="A1.11.p4.12.m12.1.1.3.cmml"><mn id="A1.11.p4.12.m12.1.1.3.2" xref="A1.11.p4.12.m12.1.1.3.2.cmml">2</mn><mo lspace="0em" rspace="0em" id="A1.11.p4.12.m12.1.1.3.1" xref="A1.11.p4.12.m12.1.1.3.1.cmml">​</mo><msup id="A1.11.p4.12.m12.1.1.3.3" xref="A1.11.p4.12.m12.1.1.3.3.cmml"><mi id="A1.11.p4.12.m12.1.1.3.3.2" xref="A1.11.p4.12.m12.1.1.3.3.2.cmml">n</mi><mi id="A1.11.p4.12.m12.1.1.3.3.3" xref="A1.11.p4.12.m12.1.1.3.3.3.cmml">c</mi></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.12.m12.1b"><apply id="A1.11.p4.12.m12.1.1.cmml" xref="A1.11.p4.12.m12.1.1"><leq id="A1.11.p4.12.m12.1.1.1.cmml" xref="A1.11.p4.12.m12.1.1.1"></leq><ci id="A1.11.p4.12.m12.1.1.2.cmml" xref="A1.11.p4.12.m12.1.1.2">𝑚</ci><apply id="A1.11.p4.12.m12.1.1.3.cmml" xref="A1.11.p4.12.m12.1.1.3"><times id="A1.11.p4.12.m12.1.1.3.1.cmml" xref="A1.11.p4.12.m12.1.1.3.1"></times><cn type="integer" id="A1.11.p4.12.m12.1.1.3.2.cmml" xref="A1.11.p4.12.m12.1.1.3.2">2</cn><apply id="A1.11.p4.12.m12.1.1.3.3.cmml" xref="A1.11.p4.12.m12.1.1.3.3"><csymbol cd="ambiguous" id="A1.11.p4.12.m12.1.1.3.3.1.cmml" xref="A1.11.p4.12.m12.1.1.3.3">superscript</csymbol><ci id="A1.11.p4.12.m12.1.1.3.3.2.cmml" xref="A1.11.p4.12.m12.1.1.3.3.2">𝑛</ci><ci id="A1.11.p4.12.m12.1.1.3.3.3.cmml" xref="A1.11.p4.12.m12.1.1.3.3.3">𝑐</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.12.m12.1c">m\leq 2n^{c}</annotation></semantics></math>,
this circuit is also of size <math id="A1.11.p4.13.m13.1" class="ltx_Math" alttext="\mathrm{poly}(n)" display="inline"><semantics id="A1.11.p4.13.m13.1a"><mrow id="A1.11.p4.13.m13.1.2" xref="A1.11.p4.13.m13.1.2.cmml"><mi id="A1.11.p4.13.m13.1.2.2" xref="A1.11.p4.13.m13.1.2.2.cmml">poly</mi><mo lspace="0em" rspace="0em" id="A1.11.p4.13.m13.1.2.1" xref="A1.11.p4.13.m13.1.2.1.cmml">​</mo><mrow id="A1.11.p4.13.m13.1.2.3.2" xref="A1.11.p4.13.m13.1.2.cmml"><mo stretchy="false" id="A1.11.p4.13.m13.1.2.3.2.1" xref="A1.11.p4.13.m13.1.2.cmml">(</mo><mi id="A1.11.p4.13.m13.1.1" xref="A1.11.p4.13.m13.1.1.cmml">n</mi><mo stretchy="false" id="A1.11.p4.13.m13.1.2.3.2.2" xref="A1.11.p4.13.m13.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.11.p4.13.m13.1b"><apply id="A1.11.p4.13.m13.1.2.cmml" xref="A1.11.p4.13.m13.1.2"><times id="A1.11.p4.13.m13.1.2.1.cmml" xref="A1.11.p4.13.m13.1.2.1"></times><ci id="A1.11.p4.13.m13.1.2.2.cmml" xref="A1.11.p4.13.m13.1.2.2">poly</ci><ci id="A1.11.p4.13.m13.1.1.cmml" xref="A1.11.p4.13.m13.1.1">𝑛</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.11.p4.13.m13.1c">\mathrm{poly}(n)</annotation></semantics></math>.</p>
</div>
<div id="A1.12.p5" class="ltx_para">
<p id="A1.12.p5.12" class="ltx_p">Finally, to compute <math id="A1.12.p5.1.m1.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="A1.12.p5.1.m1.1a"><msup id="A1.12.p5.1.m1.1.1" xref="A1.12.p5.1.m1.1.1.cmml"><mi id="A1.12.p5.1.m1.1.1.2" xref="A1.12.p5.1.m1.1.1.2.cmml">M</mi><mi id="A1.12.p5.1.m1.1.1.3" xref="A1.12.p5.1.m1.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.12.p5.1.m1.1b"><apply id="A1.12.p5.1.m1.1.1.cmml" xref="A1.12.p5.1.m1.1.1"><csymbol cd="ambiguous" id="A1.12.p5.1.m1.1.1.1.cmml" xref="A1.12.p5.1.m1.1.1">superscript</csymbol><ci id="A1.12.p5.1.m1.1.1.2.cmml" xref="A1.12.p5.1.m1.1.1.2">𝑀</ci><ci id="A1.12.p5.1.m1.1.1.3.cmml" xref="A1.12.p5.1.m1.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.1.m1.1c">M^{z}</annotation></semantics></math>, we first divide each entry of <math id="A1.12.p5.2.m2.1" class="ltx_Math" alttext="\tilde{M}^{z}" display="inline"><semantics id="A1.12.p5.2.m2.1a"><msup id="A1.12.p5.2.m2.1.1" xref="A1.12.p5.2.m2.1.1.cmml"><mover accent="true" id="A1.12.p5.2.m2.1.1.2" xref="A1.12.p5.2.m2.1.1.2.cmml"><mi id="A1.12.p5.2.m2.1.1.2.2" xref="A1.12.p5.2.m2.1.1.2.2.cmml">M</mi><mo id="A1.12.p5.2.m2.1.1.2.1" xref="A1.12.p5.2.m2.1.1.2.1.cmml">~</mo></mover><mi id="A1.12.p5.2.m2.1.1.3" xref="A1.12.p5.2.m2.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.12.p5.2.m2.1b"><apply id="A1.12.p5.2.m2.1.1.cmml" xref="A1.12.p5.2.m2.1.1"><csymbol cd="ambiguous" id="A1.12.p5.2.m2.1.1.1.cmml" xref="A1.12.p5.2.m2.1.1">superscript</csymbol><apply id="A1.12.p5.2.m2.1.1.2.cmml" xref="A1.12.p5.2.m2.1.1.2"><ci id="A1.12.p5.2.m2.1.1.2.1.cmml" xref="A1.12.p5.2.m2.1.1.2.1">~</ci><ci id="A1.12.p5.2.m2.1.1.2.2.cmml" xref="A1.12.p5.2.m2.1.1.2.2">𝑀</ci></apply><ci id="A1.12.p5.2.m2.1.1.3.cmml" xref="A1.12.p5.2.m2.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.2.m2.1c">\tilde{M}^{z}</annotation></semantics></math> by <math id="A1.12.p5.3.m3.1" class="ltx_Math" alttext="2^{qz}" display="inline"><semantics id="A1.12.p5.3.m3.1a"><msup id="A1.12.p5.3.m3.1.1" xref="A1.12.p5.3.m3.1.1.cmml"><mn id="A1.12.p5.3.m3.1.1.2" xref="A1.12.p5.3.m3.1.1.2.cmml">2</mn><mrow id="A1.12.p5.3.m3.1.1.3" xref="A1.12.p5.3.m3.1.1.3.cmml"><mi id="A1.12.p5.3.m3.1.1.3.2" xref="A1.12.p5.3.m3.1.1.3.2.cmml">q</mi><mo lspace="0em" rspace="0em" id="A1.12.p5.3.m3.1.1.3.1" xref="A1.12.p5.3.m3.1.1.3.1.cmml">​</mo><mi id="A1.12.p5.3.m3.1.1.3.3" xref="A1.12.p5.3.m3.1.1.3.3.cmml">z</mi></mrow></msup><annotation-xml encoding="MathML-Content" id="A1.12.p5.3.m3.1b"><apply id="A1.12.p5.3.m3.1.1.cmml" xref="A1.12.p5.3.m3.1.1"><csymbol cd="ambiguous" id="A1.12.p5.3.m3.1.1.1.cmml" xref="A1.12.p5.3.m3.1.1">superscript</csymbol><cn type="integer" id="A1.12.p5.3.m3.1.1.2.cmml" xref="A1.12.p5.3.m3.1.1.2">2</cn><apply id="A1.12.p5.3.m3.1.1.3.cmml" xref="A1.12.p5.3.m3.1.1.3"><times id="A1.12.p5.3.m3.1.1.3.1.cmml" xref="A1.12.p5.3.m3.1.1.3.1"></times><ci id="A1.12.p5.3.m3.1.1.3.2.cmml" xref="A1.12.p5.3.m3.1.1.3.2">𝑞</ci><ci id="A1.12.p5.3.m3.1.1.3.3.cmml" xref="A1.12.p5.3.m3.1.1.3.3">𝑧</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.3.m3.1c">2^{qz}</annotation></semantics></math>. This can be accomplished by subtracting <math id="A1.12.p5.4.m4.1" class="ltx_Math" alttext="qz" display="inline"><semantics id="A1.12.p5.4.m4.1a"><mrow id="A1.12.p5.4.m4.1.1" xref="A1.12.p5.4.m4.1.1.cmml"><mi id="A1.12.p5.4.m4.1.1.2" xref="A1.12.p5.4.m4.1.1.2.cmml">q</mi><mo lspace="0em" rspace="0em" id="A1.12.p5.4.m4.1.1.1" xref="A1.12.p5.4.m4.1.1.1.cmml">​</mo><mi id="A1.12.p5.4.m4.1.1.3" xref="A1.12.p5.4.m4.1.1.3.cmml">z</mi></mrow><annotation-xml encoding="MathML-Content" id="A1.12.p5.4.m4.1b"><apply id="A1.12.p5.4.m4.1.1.cmml" xref="A1.12.p5.4.m4.1.1"><times id="A1.12.p5.4.m4.1.1.1.cmml" xref="A1.12.p5.4.m4.1.1.1"></times><ci id="A1.12.p5.4.m4.1.1.2.cmml" xref="A1.12.p5.4.m4.1.1.2">𝑞</ci><ci id="A1.12.p5.4.m4.1.1.3.cmml" xref="A1.12.p5.4.m4.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.4.m4.1c">qz</annotation></semantics></math> from the exponent of each entry of <math id="A1.12.p5.5.m5.1" class="ltx_Math" alttext="\tilde{M}" display="inline"><semantics id="A1.12.p5.5.m5.1a"><mover accent="true" id="A1.12.p5.5.m5.1.1" xref="A1.12.p5.5.m5.1.1.cmml"><mi id="A1.12.p5.5.m5.1.1.2" xref="A1.12.p5.5.m5.1.1.2.cmml">M</mi><mo id="A1.12.p5.5.m5.1.1.1" xref="A1.12.p5.5.m5.1.1.1.cmml">~</mo></mover><annotation-xml encoding="MathML-Content" id="A1.12.p5.5.m5.1b"><apply id="A1.12.p5.5.m5.1.1.cmml" xref="A1.12.p5.5.m5.1.1"><ci id="A1.12.p5.5.m5.1.1.1.cmml" xref="A1.12.p5.5.m5.1.1.1">~</ci><ci id="A1.12.p5.5.m5.1.1.2.cmml" xref="A1.12.p5.5.m5.1.1.2">𝑀</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.5.m5.1c">\tilde{M}</annotation></semantics></math>; again, we do this computation exactly, using up to <math id="A1.12.p5.6.m6.1" class="ltx_Math" alttext="(c+1)\log n" display="inline"><semantics id="A1.12.p5.6.m6.1a"><mrow id="A1.12.p5.6.m6.1.1" xref="A1.12.p5.6.m6.1.1.cmml"><mrow id="A1.12.p5.6.m6.1.1.1.1" xref="A1.12.p5.6.m6.1.1.1.1.1.cmml"><mo stretchy="false" id="A1.12.p5.6.m6.1.1.1.1.2" xref="A1.12.p5.6.m6.1.1.1.1.1.cmml">(</mo><mrow id="A1.12.p5.6.m6.1.1.1.1.1" xref="A1.12.p5.6.m6.1.1.1.1.1.cmml"><mi id="A1.12.p5.6.m6.1.1.1.1.1.2" xref="A1.12.p5.6.m6.1.1.1.1.1.2.cmml">c</mi><mo id="A1.12.p5.6.m6.1.1.1.1.1.1" xref="A1.12.p5.6.m6.1.1.1.1.1.1.cmml">+</mo><mn id="A1.12.p5.6.m6.1.1.1.1.1.3" xref="A1.12.p5.6.m6.1.1.1.1.1.3.cmml">1</mn></mrow><mo stretchy="false" id="A1.12.p5.6.m6.1.1.1.1.3" xref="A1.12.p5.6.m6.1.1.1.1.1.cmml">)</mo></mrow><mo lspace="0.167em" rspace="0em" id="A1.12.p5.6.m6.1.1.2" xref="A1.12.p5.6.m6.1.1.2.cmml">​</mo><mrow id="A1.12.p5.6.m6.1.1.3" xref="A1.12.p5.6.m6.1.1.3.cmml"><mi id="A1.12.p5.6.m6.1.1.3.1" xref="A1.12.p5.6.m6.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.12.p5.6.m6.1.1.3a" xref="A1.12.p5.6.m6.1.1.3.cmml">⁡</mo><mi id="A1.12.p5.6.m6.1.1.3.2" xref="A1.12.p5.6.m6.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.12.p5.6.m6.1b"><apply id="A1.12.p5.6.m6.1.1.cmml" xref="A1.12.p5.6.m6.1.1"><times id="A1.12.p5.6.m6.1.1.2.cmml" xref="A1.12.p5.6.m6.1.1.2"></times><apply id="A1.12.p5.6.m6.1.1.1.1.1.cmml" xref="A1.12.p5.6.m6.1.1.1.1"><plus id="A1.12.p5.6.m6.1.1.1.1.1.1.cmml" xref="A1.12.p5.6.m6.1.1.1.1.1.1"></plus><ci id="A1.12.p5.6.m6.1.1.1.1.1.2.cmml" xref="A1.12.p5.6.m6.1.1.1.1.1.2">𝑐</ci><cn type="integer" id="A1.12.p5.6.m6.1.1.1.1.1.3.cmml" xref="A1.12.p5.6.m6.1.1.1.1.1.3">1</cn></apply><apply id="A1.12.p5.6.m6.1.1.3.cmml" xref="A1.12.p5.6.m6.1.1.3"><log id="A1.12.p5.6.m6.1.1.3.1.cmml" xref="A1.12.p5.6.m6.1.1.3.1"></log><ci id="A1.12.p5.6.m6.1.1.3.2.cmml" xref="A1.12.p5.6.m6.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.6.m6.1c">(c+1)\log n</annotation></semantics></math> additional bits in the exponent if necessary. We then cast all entries of the resulting matrix back to <math id="A1.12.p5.7.m7.1" class="ltx_Math" alttext="c\log n" display="inline"><semantics id="A1.12.p5.7.m7.1a"><mrow id="A1.12.p5.7.m7.1.1" xref="A1.12.p5.7.m7.1.1.cmml"><mi id="A1.12.p5.7.m7.1.1.2" xref="A1.12.p5.7.m7.1.1.2.cmml">c</mi><mo lspace="0.167em" rspace="0em" id="A1.12.p5.7.m7.1.1.1" xref="A1.12.p5.7.m7.1.1.1.cmml">​</mo><mrow id="A1.12.p5.7.m7.1.1.3" xref="A1.12.p5.7.m7.1.1.3.cmml"><mi id="A1.12.p5.7.m7.1.1.3.1" xref="A1.12.p5.7.m7.1.1.3.1.cmml">log</mi><mo lspace="0.167em" id="A1.12.p5.7.m7.1.1.3a" xref="A1.12.p5.7.m7.1.1.3.cmml">⁡</mo><mi id="A1.12.p5.7.m7.1.1.3.2" xref="A1.12.p5.7.m7.1.1.3.2.cmml">n</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="A1.12.p5.7.m7.1b"><apply id="A1.12.p5.7.m7.1.1.cmml" xref="A1.12.p5.7.m7.1.1"><times id="A1.12.p5.7.m7.1.1.1.cmml" xref="A1.12.p5.7.m7.1.1.1"></times><ci id="A1.12.p5.7.m7.1.1.2.cmml" xref="A1.12.p5.7.m7.1.1.2">𝑐</ci><apply id="A1.12.p5.7.m7.1.1.3.cmml" xref="A1.12.p5.7.m7.1.1.3"><log id="A1.12.p5.7.m7.1.1.3.1.cmml" xref="A1.12.p5.7.m7.1.1.3.1"></log><ci id="A1.12.p5.7.m7.1.1.3.2.cmml" xref="A1.12.p5.7.m7.1.1.3.2">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.7.m7.1c">c\log n</annotation></semantics></math> precision floats. All this can be done in <math id="A1.12.p5.8.m8.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.12.p5.8.m8.1a"><mi id="A1.12.p5.8.m8.1.1" xref="A1.12.p5.8.m8.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.12.p5.8.m8.1b"><ci id="A1.12.p5.8.m8.1.1.cmml" xref="A1.12.p5.8.m8.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.8.m8.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.12.p5.9.m9.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.12.p5.9.m9.1a"><msup id="A1.12.p5.9.m9.1.1" xref="A1.12.p5.9.m9.1.1.cmml"><mi id="A1.12.p5.9.m9.1.1.2" xref="A1.12.p5.9.m9.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.12.p5.9.m9.1.1.3" xref="A1.12.p5.9.m9.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.12.p5.9.m9.1b"><apply id="A1.12.p5.9.m9.1.1.cmml" xref="A1.12.p5.9.m9.1.1"><csymbol cd="ambiguous" id="A1.12.p5.9.m9.1.1.1.cmml" xref="A1.12.p5.9.m9.1.1">superscript</csymbol><ci id="A1.12.p5.9.m9.1.1.2.cmml" xref="A1.12.p5.9.m9.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.12.p5.9.m9.1.1.3.cmml" xref="A1.12.p5.9.m9.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.9.m9.1c">\mathsf{TC}^{0}</annotation></semantics></math>, finishing the proof that <math id="A1.12.p5.10.m10.1" class="ltx_Math" alttext="M^{z}" display="inline"><semantics id="A1.12.p5.10.m10.1a"><msup id="A1.12.p5.10.m10.1.1" xref="A1.12.p5.10.m10.1.1.cmml"><mi id="A1.12.p5.10.m10.1.1.2" xref="A1.12.p5.10.m10.1.1.2.cmml">M</mi><mi id="A1.12.p5.10.m10.1.1.3" xref="A1.12.p5.10.m10.1.1.3.cmml">z</mi></msup><annotation-xml encoding="MathML-Content" id="A1.12.p5.10.m10.1b"><apply id="A1.12.p5.10.m10.1.1.cmml" xref="A1.12.p5.10.m10.1.1"><csymbol cd="ambiguous" id="A1.12.p5.10.m10.1.1.1.cmml" xref="A1.12.p5.10.m10.1.1">superscript</csymbol><ci id="A1.12.p5.10.m10.1.1.2.cmml" xref="A1.12.p5.10.m10.1.1.2">𝑀</ci><ci id="A1.12.p5.10.m10.1.1.3.cmml" xref="A1.12.p5.10.m10.1.1.3">𝑧</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.10.m10.1c">M^{z}</annotation></semantics></math> can be computed in <math id="A1.12.p5.11.m11.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.12.p5.11.m11.1a"><mi id="A1.12.p5.11.m11.1.1" xref="A1.12.p5.11.m11.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.12.p5.11.m11.1b"><ci id="A1.12.p5.11.m11.1.1.cmml" xref="A1.12.p5.11.m11.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.11.m11.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.12.p5.12.m12.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.12.p5.12.m12.1a"><msup id="A1.12.p5.12.m12.1.1" xref="A1.12.p5.12.m12.1.1.cmml"><mi id="A1.12.p5.12.m12.1.1.2" xref="A1.12.p5.12.m12.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.12.p5.12.m12.1.1.3" xref="A1.12.p5.12.m12.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.12.p5.12.m12.1b"><apply id="A1.12.p5.12.m12.1.1.cmml" xref="A1.12.p5.12.m12.1.1"><csymbol cd="ambiguous" id="A1.12.p5.12.m12.1.1.1.cmml" xref="A1.12.p5.12.m12.1.1">superscript</csymbol><ci id="A1.12.p5.12.m12.1.1.2.cmml" xref="A1.12.p5.12.m12.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.12.p5.12.m12.1.1.3.cmml" xref="A1.12.p5.12.m12.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.12.p5.12.m12.1c">\mathsf{TC}^{0}</annotation></semantics></math>.
∎</p>
</div>
</div>
<section id="A1.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">A.1 </span><math id="A1.SS1.1.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.1.m1.1b"><mi id="A1.SS1.1.m1.1.1" xref="A1.SS1.1.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.1.m1.1c"><ci id="A1.SS1.1.m1.1.1.cmml" xref="A1.SS1.1.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.1.m1.1d">\mathsf{L}</annotation></semantics></math>-Uniformity of Polynomial Division in <math id="A1.SS1.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.SS1.2.m2.1b"><msup id="A1.SS1.2.m2.1.1" xref="A1.SS1.2.m2.1.1.cmml"><mi id="A1.SS1.2.m2.1.1.2" xref="A1.SS1.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.SS1.2.m2.1.1.3" xref="A1.SS1.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS1.2.m2.1c"><apply id="A1.SS1.2.m2.1.1.cmml" xref="A1.SS1.2.m2.1.1"><csymbol cd="ambiguous" id="A1.SS1.2.m2.1.1.1.cmml" xref="A1.SS1.2.m2.1.1">superscript</csymbol><ci id="A1.SS1.2.m2.1.1.2.cmml" xref="A1.SS1.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.SS1.2.m2.1.1.3.cmml" xref="A1.SS1.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.2.m2.1d">\mathsf{TC}^{0}</annotation></semantics></math>
</h3>

<div id="A1.SS1.p1" class="ltx_para">
<p id="A1.SS1.p1.2" class="ltx_p"><cite class="ltx_cite ltx_citemacro_citet">Hesse et al. (<a href="#bib.bib15" title="" class="ltx_ref">2002</a>)</cite> state that polynomial division is in <math id="A1.SS1.p1.1.m1.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.p1.1.m1.1a"><mi id="A1.SS1.p1.1.m1.1.1" xref="A1.SS1.p1.1.m1.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p1.1.m1.1b"><ci id="A1.SS1.p1.1.m1.1.1.cmml" xref="A1.SS1.p1.1.m1.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p1.1.m1.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.SS1.p1.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.SS1.p1.2.m2.1a"><msup id="A1.SS1.p1.2.m2.1.1" xref="A1.SS1.p1.2.m2.1.1.cmml"><mi id="A1.SS1.p1.2.m2.1.1.2" xref="A1.SS1.p1.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.SS1.p1.2.m2.1.1.3" xref="A1.SS1.p1.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS1.p1.2.m2.1b"><apply id="A1.SS1.p1.2.m2.1.1.cmml" xref="A1.SS1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="A1.SS1.p1.2.m2.1.1.1.cmml" xref="A1.SS1.p1.2.m2.1.1">superscript</csymbol><ci id="A1.SS1.p1.2.m2.1.1.2.cmml" xref="A1.SS1.p1.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.SS1.p1.2.m2.1.1.3.cmml" xref="A1.SS1.p1.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p1.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math> in Corollary 6.5. For historical reasons, this claim is preceded by weaker claims in older papers. We briefly clarify this situation to show why the stronger claim is valid.</p>
</div>
<div id="A1.SS1.p2" class="ltx_para">
<p id="A1.SS1.p2.10" class="ltx_p"><cite class="ltx_cite ltx_citemacro_citet">Reif &amp; Tate (<a href="#bib.bib29" title="" class="ltx_ref">1992</a>)</cite> establish that polynomial division can be done in <math id="A1.SS1.p2.1.m1.1" class="ltx_Math" alttext="\mathsf{P}" display="inline"><semantics id="A1.SS1.p2.1.m1.1a"><mi id="A1.SS1.p2.1.m1.1.1" xref="A1.SS1.p2.1.m1.1.1.cmml">𝖯</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.1.m1.1b"><ci id="A1.SS1.p2.1.m1.1.1.cmml" xref="A1.SS1.p2.1.m1.1.1">𝖯</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.1.m1.1c">\mathsf{P}</annotation></semantics></math>-uniform <math id="A1.SS1.p2.2.m2.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.SS1.p2.2.m2.1a"><msup id="A1.SS1.p2.2.m2.1.1" xref="A1.SS1.p2.2.m2.1.1.cmml"><mi id="A1.SS1.p2.2.m2.1.1.2" xref="A1.SS1.p2.2.m2.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.SS1.p2.2.m2.1.1.3" xref="A1.SS1.p2.2.m2.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.2.m2.1b"><apply id="A1.SS1.p2.2.m2.1.1.cmml" xref="A1.SS1.p2.2.m2.1.1"><csymbol cd="ambiguous" id="A1.SS1.p2.2.m2.1.1.1.cmml" xref="A1.SS1.p2.2.m2.1.1">superscript</csymbol><ci id="A1.SS1.p2.2.m2.1.1.2.cmml" xref="A1.SS1.p2.2.m2.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.SS1.p2.2.m2.1.1.3.cmml" xref="A1.SS1.p2.2.m2.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.2.m2.1c">\mathsf{TC}^{0}</annotation></semantics></math>, whereas we state our results for <math id="A1.SS1.p2.3.m3.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.p2.3.m3.1a"><mi id="A1.SS1.p2.3.m3.1.1" xref="A1.SS1.p2.3.m3.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.3.m3.1b"><ci id="A1.SS1.p2.3.m3.1.1.cmml" xref="A1.SS1.p2.3.m3.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.3.m3.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.SS1.p2.4.m4.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.SS1.p2.4.m4.1a"><msup id="A1.SS1.p2.4.m4.1.1" xref="A1.SS1.p2.4.m4.1.1.cmml"><mi id="A1.SS1.p2.4.m4.1.1.2" xref="A1.SS1.p2.4.m4.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.SS1.p2.4.m4.1.1.3" xref="A1.SS1.p2.4.m4.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.4.m4.1b"><apply id="A1.SS1.p2.4.m4.1.1.cmml" xref="A1.SS1.p2.4.m4.1.1"><csymbol cd="ambiguous" id="A1.SS1.p2.4.m4.1.1.1.cmml" xref="A1.SS1.p2.4.m4.1.1">superscript</csymbol><ci id="A1.SS1.p2.4.m4.1.1.2.cmml" xref="A1.SS1.p2.4.m4.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.SS1.p2.4.m4.1.1.3.cmml" xref="A1.SS1.p2.4.m4.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.4.m4.1c">\mathsf{TC}^{0}</annotation></semantics></math>. However, the only issue preventing these results from going through in the <math id="A1.SS1.p2.5.m5.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.p2.5.m5.1a"><mi id="A1.SS1.p2.5.m5.1.1" xref="A1.SS1.p2.5.m5.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.5.m5.1b"><ci id="A1.SS1.p2.5.m5.1.1.cmml" xref="A1.SS1.p2.5.m5.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.5.m5.1c">\mathsf{L}</annotation></semantics></math>-uniform case is that, at the time of publication, it was not known whether integer division and iterated integer multiplication were computable in <math id="A1.SS1.p2.6.m6.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.p2.6.m6.1a"><mi id="A1.SS1.p2.6.m6.1.1" xref="A1.SS1.p2.6.m6.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.6.m6.1b"><ci id="A1.SS1.p2.6.m6.1.1.cmml" xref="A1.SS1.p2.6.m6.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.6.m6.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.SS1.p2.7.m7.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.SS1.p2.7.m7.1a"><msup id="A1.SS1.p2.7.m7.1.1" xref="A1.SS1.p2.7.m7.1.1.cmml"><mi id="A1.SS1.p2.7.m7.1.1.2" xref="A1.SS1.p2.7.m7.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.SS1.p2.7.m7.1.1.3" xref="A1.SS1.p2.7.m7.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.7.m7.1b"><apply id="A1.SS1.p2.7.m7.1.1.cmml" xref="A1.SS1.p2.7.m7.1.1"><csymbol cd="ambiguous" id="A1.SS1.p2.7.m7.1.1.1.cmml" xref="A1.SS1.p2.7.m7.1.1">superscript</csymbol><ci id="A1.SS1.p2.7.m7.1.1.2.cmml" xref="A1.SS1.p2.7.m7.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.SS1.p2.7.m7.1.1.3.cmml" xref="A1.SS1.p2.7.m7.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.7.m7.1c">\mathsf{TC}^{0}</annotation></semantics></math>. However, <cite class="ltx_cite ltx_citemacro_citet">Hesse (<a href="#bib.bib14" title="" class="ltx_ref">2001</a>)</cite> later proved exactly this. Combining the two results, Theorem 3.2 from <cite class="ltx_cite ltx_citemacro_citet">Reif &amp; Tate (<a href="#bib.bib29" title="" class="ltx_ref">1992</a>)</cite> goes through with <math id="A1.SS1.p2.8.m8.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.p2.8.m8.1a"><mi id="A1.SS1.p2.8.m8.1.1" xref="A1.SS1.p2.8.m8.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.8.m8.1b"><ci id="A1.SS1.p2.8.m8.1.1.cmml" xref="A1.SS1.p2.8.m8.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.8.m8.1c">\mathsf{L}</annotation></semantics></math>-uniformity. Its Corollary 3.3 then allows us to conclude that integer polynomial division can be solved by <math id="A1.SS1.p2.9.m9.1" class="ltx_Math" alttext="\mathsf{L}" display="inline"><semantics id="A1.SS1.p2.9.m9.1a"><mi id="A1.SS1.p2.9.m9.1.1" xref="A1.SS1.p2.9.m9.1.1.cmml">𝖫</mi><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.9.m9.1b"><ci id="A1.SS1.p2.9.m9.1.1.cmml" xref="A1.SS1.p2.9.m9.1.1">𝖫</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.9.m9.1c">\mathsf{L}</annotation></semantics></math>-uniform <math id="A1.SS1.p2.10.m10.1" class="ltx_Math" alttext="\mathsf{TC}^{0}" display="inline"><semantics id="A1.SS1.p2.10.m10.1a"><msup id="A1.SS1.p2.10.m10.1.1" xref="A1.SS1.p2.10.m10.1.1.cmml"><mi id="A1.SS1.p2.10.m10.1.1.2" xref="A1.SS1.p2.10.m10.1.1.2.cmml">𝖳𝖢</mi><mn id="A1.SS1.p2.10.m10.1.1.3" xref="A1.SS1.p2.10.m10.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A1.SS1.p2.10.m10.1b"><apply id="A1.SS1.p2.10.m10.1.1.cmml" xref="A1.SS1.p2.10.m10.1.1"><csymbol cd="ambiguous" id="A1.SS1.p2.10.m10.1.1.1.cmml" xref="A1.SS1.p2.10.m10.1.1">superscript</csymbol><ci id="A1.SS1.p2.10.m10.1.1.2.cmml" xref="A1.SS1.p2.10.m10.1.1.2">𝖳𝖢</ci><cn type="integer" id="A1.SS1.p2.10.m10.1.1.3.cmml" xref="A1.SS1.p2.10.m10.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A1.SS1.p2.10.m10.1c">\mathsf{TC}^{0}</annotation></semantics></math> circuits because the output of integer polynomial division is an analytic function whose Taylor expansion has a finite number of terms <cite class="ltx_cite ltx_citemacro_citep">(Reif &amp; Tate, <a href="#bib.bib29" title="" class="ltx_ref">1992</a>)</cite>.</p>
</div>
</section>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Chess Reduction</h2>

<div id="A2.p1" class="ltx_para">
<p id="A2.p1.1" class="ltx_p">We let <math id="A2.p1.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="A2.p1.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="A2.p1.1.m1.1.1" xref="A2.p1.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="A2.p1.1.m1.1b"><ci id="A2.p1.1.m1.1.1.cmml" xref="A2.p1.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.p1.1.m1.1c">\mathcal{M}</annotation></semantics></math> denote the set of chess moves in (source square, target square) notation.</p>
</div>
<div id="A2.p2" class="ltx_para">
<p id="A2.p2.1" class="ltx_p">See <a href="#S3.Thmtheorem3" title="Proposition 3.3. ‣ 3.2 Encoding 𝑆₅ in Chess State Tracking ‣ 3 State Tracking ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.3</span></a></p>
</div>
<div id="A2.1" class="ltx_proof">
<h6 class="ltx_title ltx_runin ltx_font_italic ltx_title_proof">Proof.</h6>
<div id="A2.1.p1" class="ltx_para">
<p id="A2.1.p1.3" class="ltx_p">Without loss of generality, we consider the variant of <math id="A2.1.p1.1.m1.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="A2.1.p1.1.m1.1a"><msub id="A2.1.p1.1.m1.1.1" xref="A2.1.p1.1.m1.1.1.cmml"><mi id="A2.1.p1.1.m1.1.1.2" xref="A2.1.p1.1.m1.1.1.2.cmml">S</mi><mn id="A2.1.p1.1.m1.1.1.3" xref="A2.1.p1.1.m1.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="A2.1.p1.1.m1.1b"><apply id="A2.1.p1.1.m1.1.1.cmml" xref="A2.1.p1.1.m1.1.1"><csymbol cd="ambiguous" id="A2.1.p1.1.m1.1.1.1.cmml" xref="A2.1.p1.1.m1.1.1">subscript</csymbol><ci id="A2.1.p1.1.m1.1.1.2.cmml" xref="A2.1.p1.1.m1.1.1.2">𝑆</ci><cn type="integer" id="A2.1.p1.1.m1.1.1.3.cmml" xref="A2.1.p1.1.m1.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.1.p1.1.m1.1c">S_{5}</annotation></semantics></math> where the output is true if and only if the original first element returns to the first position after the given sequence of permutations has been applied. Given an <math id="A2.1.p1.2.m2.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="A2.1.p1.2.m2.1a"><msub id="A2.1.p1.2.m2.1.1" xref="A2.1.p1.2.m2.1.1.cmml"><mi id="A2.1.p1.2.m2.1.1.2" xref="A2.1.p1.2.m2.1.1.2.cmml">S</mi><mn id="A2.1.p1.2.m2.1.1.3" xref="A2.1.p1.2.m2.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="A2.1.p1.2.m2.1b"><apply id="A2.1.p1.2.m2.1.1.cmml" xref="A2.1.p1.2.m2.1.1"><csymbol cd="ambiguous" id="A2.1.p1.2.m2.1.1.1.cmml" xref="A2.1.p1.2.m2.1.1">subscript</csymbol><ci id="A2.1.p1.2.m2.1.1.2.cmml" xref="A2.1.p1.2.m2.1.1.2">𝑆</ci><cn type="integer" id="A2.1.p1.2.m2.1.1.3.cmml" xref="A2.1.p1.2.m2.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.1.p1.2.m2.1c">S_{5}</annotation></semantics></math> instance, we will construct an initial board state and sequence of moves such that the final chessboard state encodes the output of the <math id="A2.1.p1.3.m3.1" class="ltx_Math" alttext="S_{5}" display="inline"><semantics id="A2.1.p1.3.m3.1a"><msub id="A2.1.p1.3.m3.1.1" xref="A2.1.p1.3.m3.1.1.cmml"><mi id="A2.1.p1.3.m3.1.1.2" xref="A2.1.p1.3.m3.1.1.2.cmml">S</mi><mn id="A2.1.p1.3.m3.1.1.3" xref="A2.1.p1.3.m3.1.1.3.cmml">5</mn></msub><annotation-xml encoding="MathML-Content" id="A2.1.p1.3.m3.1b"><apply id="A2.1.p1.3.m3.1.1.cmml" xref="A2.1.p1.3.m3.1.1"><csymbol cd="ambiguous" id="A2.1.p1.3.m3.1.1.1.cmml" xref="A2.1.p1.3.m3.1.1">subscript</csymbol><ci id="A2.1.p1.3.m3.1.1.2.cmml" xref="A2.1.p1.3.m3.1.1.2">𝑆</ci><cn type="integer" id="A2.1.p1.3.m3.1.1.3.cmml" xref="A2.1.p1.3.m3.1.1.3">5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.1.p1.3.m3.1c">S_{5}</annotation></semantics></math> problem instance.</p>
</div>
</div>
<section id="A2.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Initial Board State.</h4>

<div id="A2.SS0.SSS0.Px1.p1" class="ltx_para">
<p id="A2.SS0.SSS0.Px1.p1.3" class="ltx_p">We construct a chessboard similar to <a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a> but with a black rook at <span id="A2.SS0.SSS0.Px1.p1.3.1" class="ltx_text ltx_markedasmath ltx_font_sansserif">a8</span> and black queens at <span id="A2.SS0.SSS0.Px1.p1.3.2" class="ltx_text ltx_markedasmath ltx_font_sansserif">b8</span> to <span id="A2.SS0.SSS0.Px1.p1.3.3" class="ltx_text ltx_markedasmath ltx_font_sansserif">e8</span>.</p>
</div>
</section>
<section id="A2.SS0.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Chess Move Sequence.</h4>

<div id="A2.SS0.SSS0.Px2.p1" class="ltx_para">
<p id="A2.SS0.SSS0.Px2.p1.6" class="ltx_p">We then construct a finite function <math id="A2.SS0.SSS0.Px2.p1.1.m1.1" class="ltx_Math" alttext="f:S_{5}\to\mathcal{M}^{*}" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.1.m1.1a"><mrow id="A2.SS0.SSS0.Px2.p1.1.m1.1.1" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.cmml"><mi id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.2" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.2.cmml">f</mi><mo lspace="0.278em" rspace="0.278em" id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.1" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.1.cmml">:</mo><mrow id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.cmml"><msub id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.cmml"><mi id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.2" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.2.cmml">S</mi><mn id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.3" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.3.cmml">5</mn></msub><mo stretchy="false" id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.1" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.1.cmml">→</mo><msup id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.2" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.2.cmml">ℳ</mi><mo id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.3" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.3.cmml">∗</mo></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.1.m1.1b"><apply id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1"><ci id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.1">:</ci><ci id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.2.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.2">𝑓</ci><apply id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3"><ci id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.1">→</ci><apply id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.1.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2">subscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.2.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.2">𝑆</ci><cn type="integer" id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.3.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.2.3">5</cn></apply><apply id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3">superscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.2">ℳ</ci><times id="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.3.cmml" xref="A2.SS0.SSS0.Px2.p1.1.m1.1.1.3.3.3"></times></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.1.m1.1c">f:S_{5}\to\mathcal{M}^{*}</annotation></semantics></math> that encodes a permutation <math id="A2.SS0.SSS0.Px2.p1.2.m2.1" class="ltx_Math" alttext="\pi" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.2.m2.1a"><mi id="A2.SS0.SSS0.Px2.p1.2.m2.1.1" xref="A2.SS0.SSS0.Px2.p1.2.m2.1.1.cmml">π</mi><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.2.m2.1b"><ci id="A2.SS0.SSS0.Px2.p1.2.m2.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.2.m2.1.1">𝜋</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.2.m2.1c">\pi</annotation></semantics></math> as a sequence of chess moves. We first factor each permutation <math id="A2.SS0.SSS0.Px2.p1.3.m3.1" class="ltx_Math" alttext="\pi" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.3.m3.1a"><mi id="A2.SS0.SSS0.Px2.p1.3.m3.1.1" xref="A2.SS0.SSS0.Px2.p1.3.m3.1.1.cmml">π</mi><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.3.m3.1b"><ci id="A2.SS0.SSS0.Px2.p1.3.m3.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.3.m3.1.1">𝜋</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.3.m3.1c">\pi</annotation></semantics></math> to a sequence of transpositions <math id="A2.SS0.SSS0.Px2.p1.4.m4.2" class="ltx_Math" alttext="\tau_{1}(\pi)\cdots\tau_{m_{\pi}}(\pi)" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.4.m4.2a"><mrow id="A2.SS0.SSS0.Px2.p1.4.m4.2.3" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml"><msub id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.cmml"><mi id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.2.cmml">τ</mi><mn id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.3" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.3.cmml">1</mn></msub><mo lspace="0em" rspace="0em" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1.cmml">​</mo><mrow id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.3.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml"><mo stretchy="false" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.3.2.1" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml">(</mo><mi id="A2.SS0.SSS0.Px2.p1.4.m4.1.1" xref="A2.SS0.SSS0.Px2.p1.4.m4.1.1.cmml">π</mi><mo stretchy="false" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.3.2.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml">)</mo></mrow><mo lspace="0em" rspace="0em" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1a" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1.cmml">​</mo><mi mathvariant="normal" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.4" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.4.cmml">⋯</mi><mo lspace="0em" rspace="0em" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1b" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1.cmml">​</mo><msub id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.cmml"><mi id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.2.cmml">τ</mi><msub id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.cmml"><mi id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.2.cmml">m</mi><mi id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.3" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.3.cmml">π</mi></msub></msub><mo lspace="0em" rspace="0em" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1c" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1.cmml">​</mo><mrow id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.6.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml"><mo stretchy="false" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.6.2.1" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml">(</mo><mi id="A2.SS0.SSS0.Px2.p1.4.m4.2.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.2.cmml">π</mi><mo stretchy="false" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.6.2.2" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.4.m4.2b"><apply id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3"><times id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.1"></times><apply id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.1.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2">subscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.2.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.2">𝜏</ci><cn type="integer" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.3.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.2.3">1</cn></apply><ci id="A2.SS0.SSS0.Px2.p1.4.m4.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.1.1">𝜋</ci><ci id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.4.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.4">⋯</ci><apply id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.1.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5">subscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.2.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.2">𝜏</ci><apply id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3">subscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.2">𝑚</ci><ci id="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.3.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.3.5.3.3">𝜋</ci></apply></apply><ci id="A2.SS0.SSS0.Px2.p1.4.m4.2.2.cmml" xref="A2.SS0.SSS0.Px2.p1.4.m4.2.2">𝜋</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.4.m4.2c">\tau_{1}(\pi)\cdots\tau_{m_{\pi}}(\pi)</annotation></semantics></math>.
Each transposition <math id="A2.SS0.SSS0.Px2.p1.5.m5.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.5.m5.1a"><mi id="A2.SS0.SSS0.Px2.p1.5.m5.1.1" xref="A2.SS0.SSS0.Px2.p1.5.m5.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.5.m5.1b"><ci id="A2.SS0.SSS0.Px2.p1.5.m5.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.5.m5.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.5.m5.1c">\tau</annotation></semantics></math> can in turn be expressed as a sequence of chess moves analogously to <a href="#S1.F1" title="In 1 Introduction ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">1</span></a>. For example, transposing items 1 and 3 can be expressed as the move sequence: (<span id="A2.SS0.SSS0.Px2.p1.6.1" class="ltx_text ltx_font_sansserif">a8</span>, <span id="A2.SS0.SSS0.Px2.p1.6.2" class="ltx_text ltx_font_sansserif">a7</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.3" class="ltx_text ltx_font_sansserif">a1</span>, <span id="A2.SS0.SSS0.Px2.p1.6.4" class="ltx_text ltx_font_sansserif">b1</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.5" class="ltx_text ltx_font_sansserif">c8</span>, <span id="A2.SS0.SSS0.Px2.p1.6.6" class="ltx_text ltx_font_sansserif">c6</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.7" class="ltx_text ltx_font_sansserif">b1</span>, <span id="A2.SS0.SSS0.Px2.p1.6.8" class="ltx_text ltx_font_sansserif">a1</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.9" class="ltx_text ltx_font_sansserif">a7</span>, <span id="A2.SS0.SSS0.Px2.p1.6.10" class="ltx_text ltx_font_sansserif">c7</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.11" class="ltx_text ltx_font_sansserif">a1</span>, <span id="A2.SS0.SSS0.Px2.p1.6.12" class="ltx_text ltx_font_sansserif">b1</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.13" class="ltx_text ltx_font_sansserif">c6</span>, <span id="A2.SS0.SSS0.Px2.p1.6.14" class="ltx_text ltx_font_sansserif">a6</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.15" class="ltx_text ltx_font_sansserif">b1</span>, <span id="A2.SS0.SSS0.Px2.p1.6.16" class="ltx_text ltx_font_sansserif">a1</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.17" class="ltx_text ltx_font_sansserif">c7</span>, <span id="A2.SS0.SSS0.Px2.p1.6.18" class="ltx_text ltx_font_sansserif">c8</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.19" class="ltx_text ltx_font_sansserif">a1</span>, <span id="A2.SS0.SSS0.Px2.p1.6.20" class="ltx_text ltx_font_sansserif">b1</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.21" class="ltx_text ltx_font_sansserif">a6</span>, <span id="A2.SS0.SSS0.Px2.p1.6.22" class="ltx_text ltx_font_sansserif">a8</span>), (<span id="A2.SS0.SSS0.Px2.p1.6.23" class="ltx_text ltx_font_sansserif">b1</span>, <span id="A2.SS0.SSS0.Px2.p1.6.24" class="ltx_text ltx_font_sansserif">a1</span>), which has the crucial property that it transposes <span id="A2.SS0.SSS0.Px2.p1.6.25" class="ltx_text ltx_font_sansserif">a8</span> with <span id="A2.SS0.SSS0.Px2.p1.6.26" class="ltx_text ltx_font_sansserif">c8</span>. We denote the mapping from transpositions to chess move sequences as <math id="A2.SS0.SSS0.Px2.p1.6.m6.1" class="ltx_Math" alttext="f:\mathcal{T}\to\mathcal{M}^{*}" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.6.m6.1a"><mrow id="A2.SS0.SSS0.Px2.p1.6.m6.1.1" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.cmml"><mi id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.2" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.2.cmml">f</mi><mo lspace="0.278em" rspace="0.278em" id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.1" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.1.cmml">:</mo><mrow id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.cmml"><mi class="ltx_font_mathcaligraphic" id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.2" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.2.cmml">𝒯</mi><mo stretchy="false" id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.1" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.1.cmml">→</mo><msup id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.2" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.2.cmml">ℳ</mi><mo id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.3" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.3.cmml">∗</mo></msup></mrow></mrow><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.6.m6.1b"><apply id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1"><ci id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.1">:</ci><ci id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.2.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.2">𝑓</ci><apply id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3"><ci id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.1">→</ci><ci id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.2">𝒯</ci><apply id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3">superscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.2">ℳ</ci><times id="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.3.cmml" xref="A2.SS0.SSS0.Px2.p1.6.m6.1.1.3.3.3"></times></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.6.m6.1c">f:\mathcal{T}\to\mathcal{M}^{*}</annotation></semantics></math>. Putting it all together, we have that</p>
<table id="A2.Ex7" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="A2.Ex7.m1.3" class="ltx_Math" alttext="f(\pi)=\prod_{j=1}^{m_{\pi}}f(\tau_{j}(\pi))." display="block"><semantics id="A2.Ex7.m1.3a"><mrow id="A2.Ex7.m1.3.3.1" xref="A2.Ex7.m1.3.3.1.1.cmml"><mrow id="A2.Ex7.m1.3.3.1.1" xref="A2.Ex7.m1.3.3.1.1.cmml"><mrow id="A2.Ex7.m1.3.3.1.1.3" xref="A2.Ex7.m1.3.3.1.1.3.cmml"><mi id="A2.Ex7.m1.3.3.1.1.3.2" xref="A2.Ex7.m1.3.3.1.1.3.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="A2.Ex7.m1.3.3.1.1.3.1" xref="A2.Ex7.m1.3.3.1.1.3.1.cmml">​</mo><mrow id="A2.Ex7.m1.3.3.1.1.3.3.2" xref="A2.Ex7.m1.3.3.1.1.3.cmml"><mo stretchy="false" id="A2.Ex7.m1.3.3.1.1.3.3.2.1" xref="A2.Ex7.m1.3.3.1.1.3.cmml">(</mo><mi id="A2.Ex7.m1.1.1" xref="A2.Ex7.m1.1.1.cmml">π</mi><mo stretchy="false" id="A2.Ex7.m1.3.3.1.1.3.3.2.2" xref="A2.Ex7.m1.3.3.1.1.3.cmml">)</mo></mrow></mrow><mo rspace="0.111em" id="A2.Ex7.m1.3.3.1.1.2" xref="A2.Ex7.m1.3.3.1.1.2.cmml">=</mo><mrow id="A2.Ex7.m1.3.3.1.1.1" xref="A2.Ex7.m1.3.3.1.1.1.cmml"><munderover id="A2.Ex7.m1.3.3.1.1.1.2" xref="A2.Ex7.m1.3.3.1.1.1.2.cmml"><mo movablelimits="false" id="A2.Ex7.m1.3.3.1.1.1.2.2.2" xref="A2.Ex7.m1.3.3.1.1.1.2.2.2.cmml">∏</mo><mrow id="A2.Ex7.m1.3.3.1.1.1.2.2.3" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.cmml"><mi id="A2.Ex7.m1.3.3.1.1.1.2.2.3.2" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.2.cmml">j</mi><mo id="A2.Ex7.m1.3.3.1.1.1.2.2.3.1" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.1.cmml">=</mo><mn id="A2.Ex7.m1.3.3.1.1.1.2.2.3.3" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.3.cmml">1</mn></mrow><msub id="A2.Ex7.m1.3.3.1.1.1.2.3" xref="A2.Ex7.m1.3.3.1.1.1.2.3.cmml"><mi id="A2.Ex7.m1.3.3.1.1.1.2.3.2" xref="A2.Ex7.m1.3.3.1.1.1.2.3.2.cmml">m</mi><mi id="A2.Ex7.m1.3.3.1.1.1.2.3.3" xref="A2.Ex7.m1.3.3.1.1.1.2.3.3.cmml">π</mi></msub></munderover><mrow id="A2.Ex7.m1.3.3.1.1.1.1" xref="A2.Ex7.m1.3.3.1.1.1.1.cmml"><mi id="A2.Ex7.m1.3.3.1.1.1.1.3" xref="A2.Ex7.m1.3.3.1.1.1.1.3.cmml">f</mi><mo lspace="0em" rspace="0em" id="A2.Ex7.m1.3.3.1.1.1.1.2" xref="A2.Ex7.m1.3.3.1.1.1.1.2.cmml">​</mo><mrow id="A2.Ex7.m1.3.3.1.1.1.1.1.1" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="A2.Ex7.m1.3.3.1.1.1.1.1.1.2" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml">(</mo><mrow id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml"><msub id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.cmml"><mi id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.2" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.2.cmml">τ</mi><mi id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.3" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.3.cmml">j</mi></msub><mo lspace="0em" rspace="0em" id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.1" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.1.cmml">​</mo><mrow id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.3.2" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.3.2.1" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml">(</mo><mi id="A2.Ex7.m1.2.2" xref="A2.Ex7.m1.2.2.cmml">π</mi><mo stretchy="false" id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.3.2.2" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo stretchy="false" id="A2.Ex7.m1.3.3.1.1.1.1.1.1.3" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow></mrow><mo lspace="0em" id="A2.Ex7.m1.3.3.1.2" xref="A2.Ex7.m1.3.3.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="A2.Ex7.m1.3b"><apply id="A2.Ex7.m1.3.3.1.1.cmml" xref="A2.Ex7.m1.3.3.1"><eq id="A2.Ex7.m1.3.3.1.1.2.cmml" xref="A2.Ex7.m1.3.3.1.1.2"></eq><apply id="A2.Ex7.m1.3.3.1.1.3.cmml" xref="A2.Ex7.m1.3.3.1.1.3"><times id="A2.Ex7.m1.3.3.1.1.3.1.cmml" xref="A2.Ex7.m1.3.3.1.1.3.1"></times><ci id="A2.Ex7.m1.3.3.1.1.3.2.cmml" xref="A2.Ex7.m1.3.3.1.1.3.2">𝑓</ci><ci id="A2.Ex7.m1.1.1.cmml" xref="A2.Ex7.m1.1.1">𝜋</ci></apply><apply id="A2.Ex7.m1.3.3.1.1.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1"><apply id="A2.Ex7.m1.3.3.1.1.1.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2"><csymbol cd="ambiguous" id="A2.Ex7.m1.3.3.1.1.1.2.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2">superscript</csymbol><apply id="A2.Ex7.m1.3.3.1.1.1.2.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2"><csymbol cd="ambiguous" id="A2.Ex7.m1.3.3.1.1.1.2.2.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="A2.Ex7.m1.3.3.1.1.1.2.2.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.2.2">product</csymbol><apply id="A2.Ex7.m1.3.3.1.1.1.2.2.3.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3"><eq id="A2.Ex7.m1.3.3.1.1.1.2.2.3.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.1"></eq><ci id="A2.Ex7.m1.3.3.1.1.1.2.2.3.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.2">𝑗</ci><cn type="integer" id="A2.Ex7.m1.3.3.1.1.1.2.2.3.3.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.2.3.3">1</cn></apply></apply><apply id="A2.Ex7.m1.3.3.1.1.1.2.3.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.3"><csymbol cd="ambiguous" id="A2.Ex7.m1.3.3.1.1.1.2.3.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.3">subscript</csymbol><ci id="A2.Ex7.m1.3.3.1.1.1.2.3.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.3.2">𝑚</ci><ci id="A2.Ex7.m1.3.3.1.1.1.2.3.3.cmml" xref="A2.Ex7.m1.3.3.1.1.1.2.3.3">𝜋</ci></apply></apply><apply id="A2.Ex7.m1.3.3.1.1.1.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1"><times id="A2.Ex7.m1.3.3.1.1.1.1.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.2"></times><ci id="A2.Ex7.m1.3.3.1.1.1.1.3.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.3">𝑓</ci><apply id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1"><times id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.1"></times><apply id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2"><csymbol cd="ambiguous" id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.1.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2">subscript</csymbol><ci id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.2.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.2">𝜏</ci><ci id="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.3.cmml" xref="A2.Ex7.m1.3.3.1.1.1.1.1.1.1.2.3">𝑗</ci></apply><ci id="A2.Ex7.m1.2.2.cmml" xref="A2.Ex7.m1.2.2">𝜋</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.Ex7.m1.3c">f(\pi)=\prod_{j=1}^{m_{\pi}}f(\tau_{j}(\pi)).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="A2.SS0.SSS0.Px2.p1.7" class="ltx_p">To reduce a sequence of permutations <math id="A2.SS0.SSS0.Px2.p1.7.m1.1" class="ltx_Math" alttext="w\in S_{5}^{*}" display="inline"><semantics id="A2.SS0.SSS0.Px2.p1.7.m1.1a"><mrow id="A2.SS0.SSS0.Px2.p1.7.m1.1.1" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.cmml"><mi id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.2" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.2.cmml">w</mi><mo id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.1" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.1.cmml">∈</mo><msubsup id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.cmml"><mi id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.2" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.2.cmml">S</mi><mn id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.3" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.3.cmml">5</mn><mo id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.3" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.3.cmml">∗</mo></msubsup></mrow><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px2.p1.7.m1.1b"><apply id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1"><in id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.1.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.1"></in><ci id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.2.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.2">𝑤</ci><apply id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.1.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3">superscript</csymbol><apply id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.1.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3">subscript</csymbol><ci id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.2.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.2">𝑆</ci><cn type="integer" id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.3.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.2.3">5</cn></apply><times id="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.3.cmml" xref="A2.SS0.SSS0.Px2.p1.7.m1.1.1.3.3"></times></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px2.p1.7.m1.1c">w\in S_{5}^{*}</annotation></semantics></math>, we let</p>
<table id="A2.Ex8" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="A2.Ex8.m1.2" class="ltx_Math" alttext="f(w)=\prod_{i=1}^{n}f(w_{i})." display="block"><semantics id="A2.Ex8.m1.2a"><mrow id="A2.Ex8.m1.2.2.1" xref="A2.Ex8.m1.2.2.1.1.cmml"><mrow id="A2.Ex8.m1.2.2.1.1" xref="A2.Ex8.m1.2.2.1.1.cmml"><mrow id="A2.Ex8.m1.2.2.1.1.3" xref="A2.Ex8.m1.2.2.1.1.3.cmml"><mi id="A2.Ex8.m1.2.2.1.1.3.2" xref="A2.Ex8.m1.2.2.1.1.3.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="A2.Ex8.m1.2.2.1.1.3.1" xref="A2.Ex8.m1.2.2.1.1.3.1.cmml">​</mo><mrow id="A2.Ex8.m1.2.2.1.1.3.3.2" xref="A2.Ex8.m1.2.2.1.1.3.cmml"><mo stretchy="false" id="A2.Ex8.m1.2.2.1.1.3.3.2.1" xref="A2.Ex8.m1.2.2.1.1.3.cmml">(</mo><mi id="A2.Ex8.m1.1.1" xref="A2.Ex8.m1.1.1.cmml">w</mi><mo stretchy="false" id="A2.Ex8.m1.2.2.1.1.3.3.2.2" xref="A2.Ex8.m1.2.2.1.1.3.cmml">)</mo></mrow></mrow><mo rspace="0.111em" id="A2.Ex8.m1.2.2.1.1.2" xref="A2.Ex8.m1.2.2.1.1.2.cmml">=</mo><mrow id="A2.Ex8.m1.2.2.1.1.1" xref="A2.Ex8.m1.2.2.1.1.1.cmml"><munderover id="A2.Ex8.m1.2.2.1.1.1.2" xref="A2.Ex8.m1.2.2.1.1.1.2.cmml"><mo movablelimits="false" id="A2.Ex8.m1.2.2.1.1.1.2.2.2" xref="A2.Ex8.m1.2.2.1.1.1.2.2.2.cmml">∏</mo><mrow id="A2.Ex8.m1.2.2.1.1.1.2.2.3" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.cmml"><mi id="A2.Ex8.m1.2.2.1.1.1.2.2.3.2" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.2.cmml">i</mi><mo id="A2.Ex8.m1.2.2.1.1.1.2.2.3.1" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.1.cmml">=</mo><mn id="A2.Ex8.m1.2.2.1.1.1.2.2.3.3" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.3.cmml">1</mn></mrow><mi id="A2.Ex8.m1.2.2.1.1.1.2.3" xref="A2.Ex8.m1.2.2.1.1.1.2.3.cmml">n</mi></munderover><mrow id="A2.Ex8.m1.2.2.1.1.1.1" xref="A2.Ex8.m1.2.2.1.1.1.1.cmml"><mi id="A2.Ex8.m1.2.2.1.1.1.1.3" xref="A2.Ex8.m1.2.2.1.1.1.1.3.cmml">f</mi><mo lspace="0em" rspace="0em" id="A2.Ex8.m1.2.2.1.1.1.1.2" xref="A2.Ex8.m1.2.2.1.1.1.1.2.cmml">​</mo><mrow id="A2.Ex8.m1.2.2.1.1.1.1.1.1" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.cmml"><mo stretchy="false" id="A2.Ex8.m1.2.2.1.1.1.1.1.1.2" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.cmml">(</mo><msub id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.cmml"><mi id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.2" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.2.cmml">w</mi><mi id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.3" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.3.cmml">i</mi></msub><mo stretchy="false" id="A2.Ex8.m1.2.2.1.1.1.1.1.1.3" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow></mrow><mo lspace="0em" id="A2.Ex8.m1.2.2.1.2" xref="A2.Ex8.m1.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="A2.Ex8.m1.2b"><apply id="A2.Ex8.m1.2.2.1.1.cmml" xref="A2.Ex8.m1.2.2.1"><eq id="A2.Ex8.m1.2.2.1.1.2.cmml" xref="A2.Ex8.m1.2.2.1.1.2"></eq><apply id="A2.Ex8.m1.2.2.1.1.3.cmml" xref="A2.Ex8.m1.2.2.1.1.3"><times id="A2.Ex8.m1.2.2.1.1.3.1.cmml" xref="A2.Ex8.m1.2.2.1.1.3.1"></times><ci id="A2.Ex8.m1.2.2.1.1.3.2.cmml" xref="A2.Ex8.m1.2.2.1.1.3.2">𝑓</ci><ci id="A2.Ex8.m1.1.1.cmml" xref="A2.Ex8.m1.1.1">𝑤</ci></apply><apply id="A2.Ex8.m1.2.2.1.1.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1"><apply id="A2.Ex8.m1.2.2.1.1.1.2.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2"><csymbol cd="ambiguous" id="A2.Ex8.m1.2.2.1.1.1.2.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2">superscript</csymbol><apply id="A2.Ex8.m1.2.2.1.1.1.2.2.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2"><csymbol cd="ambiguous" id="A2.Ex8.m1.2.2.1.1.1.2.2.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2">subscript</csymbol><csymbol cd="latexml" id="A2.Ex8.m1.2.2.1.1.1.2.2.2.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2.2.2">product</csymbol><apply id="A2.Ex8.m1.2.2.1.1.1.2.2.3.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3"><eq id="A2.Ex8.m1.2.2.1.1.1.2.2.3.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.1"></eq><ci id="A2.Ex8.m1.2.2.1.1.1.2.2.3.2.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.2">𝑖</ci><cn type="integer" id="A2.Ex8.m1.2.2.1.1.1.2.2.3.3.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2.2.3.3">1</cn></apply></apply><ci id="A2.Ex8.m1.2.2.1.1.1.2.3.cmml" xref="A2.Ex8.m1.2.2.1.1.1.2.3">𝑛</ci></apply><apply id="A2.Ex8.m1.2.2.1.1.1.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1"><times id="A2.Ex8.m1.2.2.1.1.1.1.2.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1.2"></times><ci id="A2.Ex8.m1.2.2.1.1.1.1.3.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1.3">𝑓</ci><apply id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1">subscript</csymbol><ci id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.2">𝑤</ci><ci id="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.3.cmml" xref="A2.Ex8.m1.2.2.1.1.1.1.1.1.1.3">𝑖</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.Ex8.m1.2c">f(w)=\prod_{i=1}^{n}f(w_{i}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
</div>
</section>
<section id="A2.SS0.SSS0.Px3" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Putting It All Together.</h4>

<div id="A2.SS0.SSS0.Px3.p1" class="ltx_para">
<p id="A2.SS0.SSS0.Px3.p1.3" class="ltx_p">We call our oracle for chess state tracking with the constructed initial board state and <math id="A2.SS0.SSS0.Px3.p1.1.m1.1" class="ltx_Math" alttext="f(w)" display="inline"><semantics id="A2.SS0.SSS0.Px3.p1.1.m1.1a"><mrow id="A2.SS0.SSS0.Px3.p1.1.m1.1.2" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.cmml"><mi id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.2" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.2.cmml">f</mi><mo lspace="0em" rspace="0em" id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.1" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.1.cmml">​</mo><mrow id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.3.2" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.cmml"><mo stretchy="false" id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.3.2.1" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.cmml">(</mo><mi id="A2.SS0.SSS0.Px3.p1.1.m1.1.1" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.1.cmml">w</mi><mo stretchy="false" id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.3.2.2" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px3.p1.1.m1.1b"><apply id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.cmml" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2"><times id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.1.cmml" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.1"></times><ci id="A2.SS0.SSS0.Px3.p1.1.m1.1.2.2.cmml" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.2.2">𝑓</ci><ci id="A2.SS0.SSS0.Px3.p1.1.m1.1.1.cmml" xref="A2.SS0.SSS0.Px3.p1.1.m1.1.1">𝑤</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px3.p1.1.m1.1c">f(w)</annotation></semantics></math> as the sequence of chess moves. By construction, we can then return true if and only if the rook is at <span id="A2.SS0.SSS0.Px3.p1.3.1" class="ltx_text ltx_markedasmath ltx_font_sansserif">a8</span>. The reduction can be implemented in <math id="A2.SS0.SSS0.Px3.p1.3.m3.1" class="ltx_Math" alttext="\mathsf{NC}^{0}" display="inline"><semantics id="A2.SS0.SSS0.Px3.p1.3.m3.1a"><msup id="A2.SS0.SSS0.Px3.p1.3.m3.1.1" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1.cmml"><mi id="A2.SS0.SSS0.Px3.p1.3.m3.1.1.2" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1.2.cmml">𝖭𝖢</mi><mn id="A2.SS0.SSS0.Px3.p1.3.m3.1.1.3" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1.3.cmml">0</mn></msup><annotation-xml encoding="MathML-Content" id="A2.SS0.SSS0.Px3.p1.3.m3.1b"><apply id="A2.SS0.SSS0.Px3.p1.3.m3.1.1.cmml" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1"><csymbol cd="ambiguous" id="A2.SS0.SSS0.Px3.p1.3.m3.1.1.1.cmml" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1">superscript</csymbol><ci id="A2.SS0.SSS0.Px3.p1.3.m3.1.1.2.cmml" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1.2">𝖭𝖢</ci><cn type="integer" id="A2.SS0.SSS0.Px3.p1.3.m3.1.1.3.cmml" xref="A2.SS0.SSS0.Px3.p1.3.m3.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="A2.SS0.SSS0.Px3.p1.3.m3.1c">\mathsf{NC}^{0}</annotation></semantics></math> because it is a simple elementwise mapping of the input tokens, and decoding from the output chessboard is a finite table lookup.
∎</p>
</div>
<div id="A2.SS0.SSS0.Px3.p2" class="ltx_para">
<p id="A2.SS0.SSS0.Px3.p2.1" class="ltx_p">As a fun aside, we note that the chess board constructed in <a href="#S3.Thmtheorem3" title="Proposition 3.3. ‣ 3.2 Encoding 𝑆₅ in Chess State Tracking ‣ 3 State Tracking ‣ The Illusion of State in State-Space Models" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Proposition</span> <span class="ltx_text ltx_ref_tag">3.3</span></a> is reachable in a standard chess game. The chess sequences encoding permutation sequences are all valid chess games, except that they ignore the fact that repeated board states in chess will technically lead to draws.</p>
</div>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2404.08818" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2404.08819" class="ar5iv-text-button ar5iv-severity-error">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2404.08819">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2404.08819" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2404.08821" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Sun May  5 21:23:09 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2404.08819
State-space models (SSMs)  have emerged as a potential alternative architecture for building large language models (LLMs)  compared to the previously ubiquitous transformer architecture. One theoretical weakness of transformers is that they cannot express certain kinds of sequential computation and state tracking, which SSMs are explicitly designed to address via their close architectural similarity to recurrent neural networks (RNNs) .But do SSMs truly have an advantage (over transformers)  in expressive power for state tracking?Surprisingly, the answer is no. Our analysis reveals that the expressive power of SSMs is limited very similarly to transformers: SSMs cannot express computation outside the complexity class. In particular, this means they cannot solve simple state-tracking problems like permutation composition. It follows that SSMs are provably unable to accurately track chess moves with certain notation, evaluate code, or track entities in a long narrative. To supplement our formal analysis, we report experiments showing that Mamba-style SSMs indeed struggle with state tracking. Thus, despite its recurrent formulation, the “state” in an SSM is an illusion: SSMs have similar expressiveness limitations to non-recurrent models like transformers, which may fundamentally limit their ability to solve real-world state-tracking problems.
abstract
The Illusion of State in State-Space Models
Abstract
2404.08819
1Introduction\chessboard[
setpieces=qa8, qb8, rc8, qd8, qe8, Pa2, Pb2, Ka1, kh1,
addpgf=,
]\pythx = [0, 0, 1, 0, 0]\pythx[1], x[3] = x[3], x[1] # Swap 1, 3Alice, Bob, Carl, Dan, and Emma each have a coin. All are dimes except Carl’s.
Alice and Carl trade coins.Figure 1:We prove that SSMs, like transformers, cannot solve inherently sequential problems like permutation composition () , which lies at the heart of state-tracking problems like tracking chess moves in source-target notation (seeSection3.2) , evaluating Python code, or entity tracking. Thus, SSMs cannot, in general, solve these problems either.Recent theoretical workhas shown that models built upon the transformer architecture are incapable of expressing inherently sequential computation. These results reveal a surprising limitation of transformers: they cannot express simple kinds ofstate trackingproblems, such as composing sequences of permutations, which even simple recurrent neural networks (RNNs)  can naturally express.
In a different line of work, state space model (SSM)  architectureshave been introduced as an alternative to transformers, with the goal of achieving similar expressive power to RNNs for handling problems that are naturally stateful and sequential.But does the seemingly stateful design of SSMs truly enable them to solve sequential and state-tracking problems that transformers cannot?If so, this would be a promising property of SSMs because state tracking is at the heart of large language model (LLM)  capabilities such as tracking entities in a narrative, playing chess under certain notation111The hardness of chess state tracking holds with (source, target)  notation, but standard notation may make state tracking easier., or evaluating code. This would motivate further research into SSM architectures and their deployment as the next generation of LLMs.In this work, we show that the apparent stateful design of SSMs is anillusionas far as their expressive power is concerned. In contrast to the suggestion by(and, perhaps, a broader belief in the community)  that SSMs have expressive power for state tracking similar to RNNs, we prove theoretically that linear and Mamba-style SSMs, like transformers, cannot express inherently sequential problems, including state-tracking problems like composing permutations that RNNs can easily express. Further, our experiments confirm our formal predictions: both transformers and these SSMs cannot learn to compose permutations with a fixed number of layers, whereas RNNs can compose permutations with just a single layer.
Our results imply that arguments that SSMs have an advantage over transformers due to being “more recurrent” or capable of tracking state are misguided. In fact, the SSM architectures we consider are just as theoretically unequipped for state tracking and recurrent computation as transformers are.We first establish the theoretical weakness of linear SSMs and near generalizations by proving they are in the complexity class-uniform, which has been previously shown for transformers. This implies these SSMs cannot solve inherently sequential problems (formally, problems that are-hard) , including state-tracking problems like permutation composition. Permutation composition is a fundamental problem at the heart of many real-world state-tracking problems such as playing chess, evaluating code, or tracking entities in a narrative (Figure1) , implying solutions to these problems, too, cannot be expressed by SSMs, at least in the worst case.ChessCodeEntitiesSSMsTransformersFigure 2:Complexity hierarchy within. Transformers can only recognize languages within, and we show the same for SSMs (Theorems4.1and4.2) . Thus, both architectures cannot express the “hard state tracking” captured by-complete problems like, whichcanbe straightforwardly expressed by RNNs. The figure assumes the widely held conjecture.At first glance, our results may appear to contradict’s claim that linear SSMs can simulate general recurrent models, which can express permutation composition. But the contradiction is resolved by a difference in assumptions:relied oninfinite depth(number of layers)  to show that SSMs could simulate RNNs. We, on the other hand, analyze the realistic setting with a bounded number of layers, under which we find that SSMs cannot simulate the recurrent state of an RNN and, in fact, suffer from similar limitations as transformers for state tracking.Our empirical investigation shows that, in practice, both SSMs with the Mamba architectureand transformers donotlearn to solve the permutation composition state-tracking problem with a fixed number of layers, while simple RNNs can do so with just one layer. This provides empirical support for our theoretical separation in expressive power for state tracking between SSMs and true recurrent models. We also find that both transformers and SSMs struggle compared to RNNs on state-tracking problems less complex than permutation composition where it is not known whether they can express a solution.
Thus, in practice, SSMs may struggle not just on the hardest state-tracking problems like permutation composition but also on easier variants.Finally, we propose two minimal extensions of linear SSMs that increase their expressive power for state tracking, allowing them to solve permutation composition. These extensions, however, may come with a cost: they may negatively impact parallelism as well as learning dynamics. We view it as an interesting open question whether it is possible to develop SSM-like models with greater expressivity for state tracking that also have strong parallelizability and learning dynamics, or whether these different goals are fundamentally at odds, assuggest.
introduction
The Illusion of State in State-Space Models
1Introduction
2404.08819
2BackgroundWe first present the SSM architectures we will analyze (Section2.1) .
Our analysis of the state tracking capabilities of SSMs borrows deeply from the circuit complexity and algebraic formal language theory literature. We thus review how circuit complexity can be used to analyze the power of neural networks (Section2.3)  and how state-tracking problems can be captured algebraically and analyzed within the circuit complexity framework (Section3.1) .2.1Architecture of State-Space ModelsSSMs are a neural network architecture for processing sequences similar in design to RNNs or linear dynamical systems.
SSMs have been suggested to have two potential advantages compared to transformers owing to their recurrent formulation: faster inference and, possibly, the ability to better express inherently sequential or stateful problems.
Several architectural variants of SSMs have been proposed, including S4and Mamba.
Recently, SSMs have been shown to achieve strong empirical performance compared to transformers in certain settings, particularly those involving a long context.SSMs consist ofstate-space layers, which can be thought of as simplified RNN layers. We consider two variants of the state-space layer: the linear SSM layer (of which S4 is a special case;)  and the extended S6 layer used by Mamba.Definition 2.1(Linear SSM layer; e.g. S4) .Given a sequence of embeddings or previous states, therecurrent formof a linear SSM layer defines a new sequence of statesusing learned parameter matricesand. For each,(1) Theconvolutional formof the linear SSM layer defines the same222The two forms express the same function overor any other distributive datatype. Over floating points (AppendixA) , they are not guaranteed to be the same, but we must assume the error is negligible for them to be well-defined and usable in practice.but is unrolled as a summation of terms:(2) The layer outputswhereand.S4 chooses a specific parameterization of a linear layer.
A full S4 model is a cascade of such layers and feedforward layers, analogous to how transformers alternate multihead-self-attention layers with feedforward layers.The S6 layer used by Mambageneralizes a linear SSM layer by adding aselection mechanisminspired by the dynamic gating in LSTMsand GRUs.Definition 2.2(S6 layer) .An S6 layer is parameterized by adiagonalmatrix, a vector, and affine projectionsand.
Letbe softplus.
The S6 layer is:(3) This implies the convolutional form:(4) Finally, to compute,is made input dependentand computed via a projection in the same manner as. The layer output is then.In practice, a crucial detail for training SSMs is the initialization of the weight matrices.
Our main results (Theorems4.1and4.2)  will apply for any linear SSM (including S4)  as well as S6, independent of the specific values of its weights.
In contrast to S4 and S6, H3is not a true SSM because the context is not represented by a single vector. Rather, its architecture resembles a transformer with SSM components. Analyzing H3 is beyond our focus, but we believe our ideas could be extended to H3 in future work.2.2Numeric DatatypeCircuit-complexity analysis of neural networks depends to some degree on low-level details about arithmetic and the underlying datatypeused in the network’s computation graph. We can think ofas parameterized by the number of bits available to represent a number in. For instance, non-negative integers inusebits, signed integers inusebits, FP16 usesbits, etc.Our main results (Theorems4.1and4.2)  will go through for any datatypefor which the following 3 operations are efficiently parallel-computable, i.e., are in the complexity class-uniform(to be defined shortly inSection2.3) :1.Iterated addition, i.e., summingnumbers in2.Iterated product, i.e., multiplyingnumbers in3.Matrix powering, i.e., computing the-th power of a fixed-sizematrix overWhenis any finite-precision datatype, i.e., has a fixed number of bits available (e.g., 16 or 64) , then these operations are easily seen to be in-uniform. Asargue, however, finite-precision datatypes severely limit the expressivity of neural architectures from a formal perspective (e.g., finite-precision transformers cannot represent uniform attention) , motivating the use of parameterized datatypes that can (approximately)  represent any number with a sufficiently large parameter. Interestingly, whenis the datatype of-bit integers, all of the above operations are known to be in-uniform. Realistically, however, neural model implementations use floating point numbers with much fewer thanbits.Concretely, we imagine working with thelog-precision floating pointmodel used byto analyze transformers, which we will show satisfy all these properties. For some fixed constant, aprecision float is a tuplewhereare signed integers together takingbits. Usingto mean the number of bits used to represent integer, this float represents the value.Unlike for integers, arithmetic operations over log-precision floats are not closed. That is, the productof two-precision floats is a well-defined number but may not be exactly representable as a-precision float.
It is thus necessary to define approximate versions of these operations when formalizing log-precision floating-point arithmetic.
To this end,define a natural notion of approximate iterated addition over log-precision floats and show that it is computable in-uniform.
Following’s definition of iterated addition for floats, we define iterated multiplication and matrix powering over any datatypeas the result of treating the numbers as reals, performing exact arithmetic, and casting the exact outputback to, denoted.InAppendixA, we extend the arguments ofandfor integers to show that iterated product and matrix powering over log-precision floats are also computable in-uniform(cf.Lemmas2.3and2.4inAppendixA) .Lemma 2.3(Iterated float product) .Letbeprecision floats and. Then the iterated float productcan be computed in-uniform.Lemma 2.4(Float matrix power) .Letbe fixed constants.
Letbe amatrix overprecision floats. Let. Then float matrix powercan be computed in-uniform.Combined with the result for iterated addition from, this establishes that the three needed properties are met for log-precision floats.2.3Limits of Transformers via Circuit ComplexityA line of recent work has used circuit complexity and logic formalisms to identify the expressiveness limitations of transformers on reasoning problems; seefor a survey. In particular,showed that transformers can only solve problems in the complexity class, which is defined as the set of problems that can be recognized by constant-depth, polynomial-size threshold circuit families. Such circuits, in addition to having standard AND, OR, and NOT gates (of arbitrary fan-in) , can also use threshold gates that outputiff at leastof the inputs are, whereis a parameter of the gate. Informally,can be thought of as the class of problems that can be solved with extremely parallelized (constant-depth)  computation.333We useto mean-uniform, meaning the circuit family is constructible by a Turing machine that runs in space logarithmic in the size of the input.
We believe our results could be extended from-uniformto-uniformusing techniques similar tofor composingcircuits in a way that preservesuniformity.Problems outside, corresponding to problems that are inherently sequential and thus cannot be parallelized, cannot be solved by transformers. No problems in polynomial time are known unconditionally to be outside, but unless the widely held conjecture thatis false, many simple-hard problems are outside. In particular, this includes simulating finite automata (-complete) , evaluating boolean formulas (-complete) , determining graph connectivity (-complete) , and solving linear equations (-complete) . These problems have already been shown to be inexpressible by transformers. By showing that SSMs can be simulated in, we will establish that they also cannot be solved by SSMs.
body
The Illusion of State in State-Space Models
2Background
2404.08819
3State TrackingInformally, a state-tracking problem is a problem where the text specifies some sequence of updates to the state of the world, and the goal of the problem is to say what the resulting world state is after the updates have been applied in sequence.
This circuit complexity view on the power of neural networks can be combined with other insights from algebraic formal language theory to analyze the kinds of state tracking that SSMs can express.
In particular, this theory comprehensively shows us which kinds of state-tracking problems are (likely)  not in. This will, in turn, allow us to find examples of hard state tracking that models like SSMs will not be able to solve.3.1State Tracking as a Monoid Word ProblemFrom the perspective of algebraic formal language theory, state tracking over a finite world can be captured as aword problemon afinite monoid.444We consider only finite monoids for simplicity, but, in principle, it would be possible to extend this approach to infinite (e.g., finitely generated)  monoids as well.Different updates to the world become different elements in the monoid, and resolving the final world state after all the updates have been applied is equivalent to computing the product of a sequence of elements (also called a “word”) .Definition 3.1(Word Problem) .Letbe a finite set, anda finite monoid (i.e.,with identity and associative multiplication) . The word problem foris
to reduce sequences inunder multiplication; that is, sendto. Solving the word problem requires reducing sequences of arbitrary length.Example 3.2.Consider the monoidwhereis addition modulo 2. The word problem involves computing the parity of a string, e.g.,. From a state-tracking perspective, this monoid captures a world with a single light switch. The identitycorresponds to no action whereasis an update that flips the switch.Modeling state tracking with word problems lets us draw connections between circuit complexity and abstract algebra to understand which word problems are “hard” to solve.established that not all word problems are created equal: some, likeExample3.2, are in, while others are-complete, requiring recurrent processing to solve. Because we will show SSMs can be simulated in, it follows that-complete state-tracking problems cannot be solved by SSMs (cf.Figure2) .Whether or not a word problem is-complete depends on the algebraic structure of the underlying monoid.555We focus on word problems ongroups, which are monoids with inverses.showed that the word problem of every finite non-solvable666Formally, a groupis solvable exactly when there is a series of subgroupssuch thatis normal inandis abelian.group is-complete. That non-solvable groups have-complete word problems is notable because of the ubiquity with which non-solvable groups show up in tasks involving state tracking. The canonical example of an-complete word problem is that of, the symmetric group on five elements that encodes the permutations over five objects. As an immediate instantiation of this, consider a document describing arbitrarily many sequences of transpositions:“swap ball 1 and 3, swap ball 3 and 5, swap ball 4 and 2, …”777Without less of generality, any permutation can be factored into a sequence of transpositions, or swaps. This means the transpositions over five elements are a generator for.. Being able to answer the question“where does ball 5 end up?”requires solving theword problem. Beyond permutations,Figure1shows how many natural state-tracking problems like tracking chess moves, evaluating code, or tracking entities also encode the structure of, meaning these state-tracking problems also cannot be expressed by a model in. Rather, in order to solve these problems, the depth of the model would have to be expanded to accommodate longer inputs.Although theword problem is canonical, in this paper we will consider the word problem on a closely related group: thealternatinggroup on five elements. We do this for simplicity:is a subgroup ofcontaining only even permutations, and is the smallest non-solvable subgroup. We will compare the word problem onto two other baseline groups:, a non-abelian but solvable group; and, an abelian group encoding mod-addition. We choose these groups as points of comparison because they all havedistinct elements, meaning that the difficulty in learning their word problems will come only from the complexity of learning the group multiplication operation.3.2Encodingin Chess State TrackingFigure1already gives some intuition into how state-tracking problems encode. Out of these examples, the most intricated case is chess. We now give a proper reduction fromto tracking chess moves, showing formally that not just, but chess state tracking as well, is-complete.We define the chess state-tracking problem as follows:•Input:Thestate of a chessboardas well as asequence of chess moves, where each move is specified as a tuple (source square, target square) . Note that this differs from the standard notation that represents a move as a piece along with target square and potential disambiguating information.•Output:The resulting board state after starting in the initial board state and applying the sequence of moves one after another, ignoring draws. If any move is illegal given its intermediate board state, we enter a special null board state.We show thatcan be reduced to chess state tracking, establishing the-completeness of chess state tracking. In other words, we can map anysequence to a sequence of chess moves and read off the answer to theinstance from the final chessboard state.Proposition 3.3.can be reduced to chess state tracking viareductions.Proof.The idea, as illustrated inFigure1, is to map eachelement to a sequence of chess moves that permutes five pieces on the chessboard.
Then, the final chessboard state will allow us to determine the composition of the permutation sequence.
We defer a detailed proof toAppendixB.
∎Sinceis-complete underreductions:Corollary 3.4.The chess state-tracking problem is-complete underreductions.Similar reductions can be constructed for evaluating Python or tracking entities in a dialog, as suggested inFigure1.
For another example of a similar reduction used to prove-completeness, we refer the reader to Theorem 3.2 of.
body
The Illusion of State in State-Space Models
3State Tracking
2404.08819
4SSMs Can be Simulated inIn this section, we show that the convolutional form of an SSM can be simulated in. Assuming the convolutional form of the model computes the same function as the recurrent form, this implies that SSMs, in whatever parameterization, cannot solve inherently sequential problems, despite their appearance of recurrence and statefulness. We first show containment infor the simple non-gated variant of generalized S4 models (Theorem4.1) , and then show that the proof goes through for generalized S6 models as well (Theorem4.2) . The main idea in both proofs is that matrix powering can be computed in, and computing the convolutional form of an SSM can essentially be reduced to matrix powering.4.1Simulating Linear SSMs with aCircuit FamilyTheorem 4.1.For any log-precision SSMwith the S4 architecture, there exists an-uniformcircuit family that computes the same function as’s convolutional form.Proof.It suffices to construct an-uniformcircuit family to simulate a single layer. Then, the full SSM can be simulate by generating this circuit multiple times, routing the output bits from one layer as the inputs to the next. This can be done with log-space overhead by simply storing a counter that tracks the index of the current input gates.Recall that the S4 convolutional form Eqn. (2)  is. Crucially, we use the fact that matrix powering over floats is in-uniform(Lemma2.4, extending)  to print, for each, acircuit family that computes.
Next, we use the fact that fixed-arity arithmetic over floats is in-uniformto print, for each, a circuit that computesSince iterated addition over floats is in-uniform(, extendingfor integers) , we can compute.
Finally, we computefromandinvia simple arithmetic operations, completing the proof.
∎4.2Simulating S6 with aCircuit FamilyTheorem 4.2.For any log-precision SSMwith the S6 architecture, there exists an-uniformcircuit family that computes the same function as’s convolutional form.Proof.The convolutional form of the S6 layer is given in Eqn. (4) .
For eachwe print an-uniformcircuit computingand.
Sinceis diagonal, iterated matrix multiplication is reducible to iterated scalar multiplication, which is in-uniform(Lemma2.4) .
so we can compute the following in:We then conclude analogously toTheorem4.1, first computingfromand, then computing, and finally computingfromandin.
∎4.3DiscussionTheorems4.1and4.2establish that SSMs, like transformers, can only express solutions to problems in the class. This means that SSMs cannot solve-hard problems like evaluating boolean formulas or graph connectivity. In particular, it shows that they are limited as far as their state tracking capabilities as they are unable to compose permutations (solve theword problem) :Corollary 4.3.Assuming, no log-precision SSM with the S4 or S6 architecture can solve the word problem foror any other-hard problem.In contrast, RNNs can easily expressvia standard constructions that encode finite-state transitions into an RNN. This shows that SSMs cannot express some kinds of state tracking and recurrence that RNNs can. This tempers the claim fromthat SSMs have the expressive power to simulate RNNs, which relied on the assumption that SSMs can haveinfinite depth. In a more realistic setting with a bounded number of layers, our results show SSMs cannot express many state-tracking problems, including those which can be solved by fixed-depth RNNs.
body
The Illusion of State in State-Space Models
4SSMs Can be Simulated in
2404.08819
5Extending the Expressive Power of SSMsWe have shown that S4 and S6, despite their seemingly “stateful” design, cannot express problems outside, which includes state tracking like. We show how SSMs can be extended to close the gap in expressive power with RNNs, allowing them to express.
Two simple extensions can bring about this increase in expressive power: adding a nonlinearity to make the SSM more like an RNN or allowing thematrix to be input-dependent to make the SSM more like a weighted finite automaton (WFA;) .5.1Via NonlinearitiesConcretely, anRNN-SSM layerwith a step activation function can be defined as follows:(5) After this change, the SSM no longer has a straightforward convolutional form. However, its recurrent form is effectively an RNN, and can therefore solve:Theorem 5.1.For any regular language(including the word problem for) , there exists a one-layer log-precision RNN-SSM that recognizes.Proof.The standard constructions for simulating automata with RNNsapply.
∎Note that adding nonlinearities to the output of an SSM layer (as in Mamba)  is not the same thing as an RNN-SSM. Rather, an RNN-SSM has nonlinearities applied after each recurrent update to the state.5.2Via Input-Dependent Transition MatricesAnother completely different way to get greater expressive power is to letmatrix to be input-dependent.
To illustate this, we define and analyze theWFA-SSM layer. Let. The recurrent form becomes:(6) This means the convolutional form computes aniterated productof a sequence of matrices rather than powering a matrix as for S4 and S6 (cf.Section2.1) :(7) Unlike matrix powers, iterated matrix products cannot, in general, be computed in.
This means that the argument fromTheorem4.1will not go through for WFA-SSMs. In fact, we can show that WFA-SSMs gains expressive power beyond:Theorem 5.2.For any regular languageover vocabulary(including the word problem for) , there exists a one-layer log-precision WFA-SSM that recognizes, whereis a special beginning-of-string symbol.Proof.It suffices to show that an WFA-SSM can simulate a deterministic finite automaton (DFA) . We do this via a transition monoid construction. For any, letbe the function mapping a state to its eventual destination state afteris read from that state. For any DFA, this set of functions forms a finite monoid (thetransition monoid)  under composition,
following from the Myhill-Nerode theorem.
Further, each monoid elementcan be represented as a boolean transition matrix, making matrix multiplication isomorphic to monoid composition.Computing the transition monoid of a DFA allows recognizing valid words: compute the monoid element for a word by multiplying the elements for its tokens and then check whether the initial state maps to an accepting state.
In fact, a standard way to solve monoid word problems (e.g., for)  with a DFA is simply to construct a DFA whose transition monoid is the monoid of interest.Fix a DFA and its transition monoid.
To complete the proof, we show that there exists an SSM that, for all, computesgiven input. We view indices inas states and defineasat each accepting state, andelsewhere. For all other, we let.
This reduces the convolutional form ofto have a single term:Now, letfor. It follows thatSince, we conclude thatis.
∎5.3DiscussionTheorems5.1and5.2show that two minimal extensions of the SSM enable expressive power outside, allowing the model to solve hard state-tracking problems:Corollary 5.3.There exist a one-layer log-precision RNN-SSM and WFA-SSM that express the word problem for(with a beginning-of-string symbol) , and these these SSMs cannot be simulated in.But would these variants of SSMs be feasible to use in practice? Besides expressive power, there are two competing practical concerns that might make these extensions problematic: parallelism and the impact on learning dynamics.Parallelism.To be used effectively in an LLM, a model architecture must be parallelizable on practical hardware. Architectures inare parallelizable by design, but architectures inmay still be parallelizable to log depth even if they cannot be parallelized to constant depth. For the WFA-SSM, the bottleneck would be computing iterated matrix product with a log-depth computation graph. Similarly, for the RNN-SSM, the bottleneck would be computing the state updates with a log-depth binary tree rather than left to right. If this could be accomplished on practical hardware, these architectures could be parallelizable enough to scale on modern hardware.Learning Dynamics.Another potential concern for these SSM variants compared to the original SSM is whether the learning dynamics are as good. In particular, for the WFA-SSM, an iterated product of matrices may lead to vanishing gradient issues. However, this is already potentially an issue for the S6 architecture, where the selective gating involves computing an iterated product of scalars.
body
The Illusion of State in State-Space Models
5Extending the Expressive Power of SSMs
2404.08819
6Can SSMs Learn Permutations in Practice?Figure 3:Minimum number of layers required to attainvalidation accuracy on group multiplication problems by sequence length and group. RNN and IDS4 models of constant depth can solve arbitrarily long sequences, while transformer, S4, and Mamba models require depths monotonically increasing in sequence length.Having established theoretical limitations of SSMs for state tracking, we now empirically test
how well SSMs can learn such tasks,
focusing on the word problem for. Since this problem is-complete and both transformers and SSMs can only express functions in, to solve instances of this problem these models should require a dynamic depth that grows with the input length.Setup.We model word problems (seeSection3.1)  as a token-tagging task. Models are given as input a sequence of elementsdrawn from one of,, or, and for each stepmust predict the result of multiplying the firstelements of the sequence together. Modeling the problem as a tagging task rather than as a sequence classification task provides the models with more supervision during training, making it as easy as possible to learn the correct function. We tokenize inputs such that each element gets a unique token. We train models on sequences of lengthfor successively larger values ofand report full-sequence accuracy on a validation set.888We always include the full set ofpairwise sequences of lengthin the training data, along with the training split of length-sequences.To validate the predictions of SSMs and transformers being depth-bounded for expressing the word problem for, we sweep over the number of layers each S4, Mamba and transformer model has, and compare these results to a single-layer simple RNN and our proposed single-layer Input-Dependent S4 (IDS4)  architecture.Results.Figure3shows that single-layer RNN and IDS4 models learn the word problem for arbitrarily long sequences from all three groups. In contrast, transformer, S4, and Mamba models require depth monotonically increasing in sequence length to attain good accuracy on a validation set on non-commutative groups. We draw three main conclusions from this:1.Mamba and S4 show the same qualitative limitations as transformers on the inherently sequential task: longersequences require deeper models. This is consistent with both transformers, S4, and Mamba being in. On the other hand, RNNs (which resemble automata;)  and IDS4 can solve-complete problemsincluding theword problem.2.S4, Mamba, and transformers require greater depth even, which can be theoretically expressed bycircuits. Although transformer and Mamba models of a given depth perform as good or better onas they on, they still require increasingly many layers to handle proportionally longer sequences. There are two possible interpretations of this. First, it could be that while these word problems are expressible in, they cannot be expressed by S4, Mamba, or transformers (which can each likely recognize only a proper subset of) . On the other hand, it is possible that these word problemsareexpressible by transformers, S4, and Mamba but that effectively learning a constant-depth solution is difficult.3.Despite this limitation, S4 and Mamba appearempirically betterat approximate state tracking on the non-commutative tasks than the transformer. For length-sequences fromor, the transformer requires at least as many (and frequently fewer)  layers as S4 or Mamba to solve the task.
body
The Illusion of State in State-Space Models
6Can SSMs Learn Permutations in Practice?
2404.08819
7ConclusionWe have shown that SSMs, like transformers, can only express computation in the complexity class-uniform. This means they cannot solve inherently sequential problems like graph connectivity, boolean formula evaluation, and—of particular interest for state tracking—the permutation composition problem.can be naturally expressed by true recurrent models like RNNs and captures the essence of hard state tracking due to its-completeness. In practice, one-layer RNNs can easily learn a task capturingSSMs require depth growing with the sequence length. These results reveal that SSMs cannot truly track state: rather, they can only solve simple state-tracking problems for which shallow shortcuts exist.We also showed that simple extensions of SSMs can express, although this comes with potential drawbacks as far as parallelism and learning dynamics.
In future work, it would be interesting to more thoroughly explore the practical viability of our SSM extensions.
Ultimately, this line of work has the potential to unlock new neural architectures that balance the parallelism of transformers and SSMs with full expressive power for state tracking, enabling LLMs that can benefit from scale while enjoying a greater capacity to reason about games, code, and language.
body
The Illusion of State in State-Space Models
7Conclusion
2404.08819
Broader ImpactThis paper aims to advance the foundational understanding of state-space architectures for deep learning.
Such work can affect the development and deployment of deep learning models in a variety of ways, which in turn can have societal impacts. However, we find it difficult to meaningfully speculate about or anticipate these downstream impacts here.
conclusion
The Illusion of State in State-Space Models
Broader Impact