<!DOCTYPE html><html prefix="dcterms: http://purl.org/dc/terms/" lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2407.01082] Min P Sampling: Balancing Creativity and Coherence at High Temperature</title><meta property="og:description" content="Large Language Models (LLMs) generate longform text by successively sampling the next token based on the probability distribution of the token vocabulary at each decoding step. Current popular truncation sampling metho…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Min P Sampling: Balancing Creativity and Coherence at High Temperature">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Min P Sampling: Balancing Creativity and Coherence at High Temperature">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2407.01082">

<!--Generated on Mon Aug  5 13:51:45 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document">
<h1 class="ltx_title ltx_title_document">Min P Sampling: Balancing Creativity and Coherence at High
Temperature</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">
<span id="id6.1.id1" class="ltx_text ltx_font_bold">Nguyen Nhat Minh<sup id="id6.1.id1.1" class="ltx_sup"><span id="id6.1.id1.1.1" class="ltx_text ltx_font_medium">1</span></sup></span>,
<span id="id7.2.id2" class="ltx_text ltx_font_bold">Andrew Baker<span id="footnotex1" class="ltx_note ltx_role_footnotemark"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotemark: </span></span></span></span><sup id="id7.2.id2.1" class="ltx_sup"><span id="id7.2.id2.1.1" class="ltx_text ltx_font_medium">2</span></sup></span>,
<span id="id8.3.id3" class="ltx_text ltx_font_bold">Andreas Kirsch<sup id="id8.3.id3.1" class="ltx_sup"><span id="id8.3.id3.1.1" class="ltx_text ltx_font_medium">2</span></sup></span>,
<span id="id9.4.id4" class="ltx_text ltx_font_bold">Clement Neo<span id="footnotex2" class="ltx_note ltx_role_footnotemark"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotemark: </span></span></span></span><sup id="id9.4.id4.1" class="ltx_sup"><span id="id9.4.id4.1.1" class="ltx_text ltx_font_medium">1</span></sup></span>,

<br class="ltx_break">
<br class="ltx_break"><sup id="id10.5.id5" class="ltx_sup">1</sup>Apart Research
<sup id="id11.6.id6" class="ltx_sup">2</sup>Independent

<br class="ltx_break">
</span><span class="ltx_author_notes">Equal Contribution. 
<br class="ltx_break">Correspondence to <span id="id12.7.id1" class="ltx_text">minh1228@gmail.com</span></span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id5.5" class="ltx_p">Large Language Models (LLMs) generate longform text by successively sampling the next token based on the probability distribution of the token vocabulary at each decoding step. Current popular truncation sampling methods such as top-<math id="id1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="id1.1.m1.1a"><mi id="id1.1.m1.1.1" xref="id1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="id1.1.m1.1b"><ci id="id1.1.m1.1.1.cmml" xref="id1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="id1.1.m1.1c">p</annotation></semantics></math> sampling, also known as nucleus sampling, often struggle to balance coherence and creativity in generating text, particularly when using higher temperatures. To address this issue, we propose min-<math id="id2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="id2.2.m2.1a"><mi id="id2.2.m2.1.1" xref="id2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="id2.2.m2.1b"><ci id="id2.2.m2.1.1.cmml" xref="id2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="id2.2.m2.1c">p</annotation></semantics></math>, a dynamic truncation sampling method, that establishes a minimum base percentage threshold for tokens, which the scales according to the probability of the top candidate token. Through experiments on several benchmarks, such as GPQA, GSM8K and AlpacaEval Creative Writing, we demonstrate that min-<math id="id3.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="id3.3.m3.1a"><mi id="id3.3.m3.1.1" xref="id3.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="id3.3.m3.1b"><ci id="id3.3.m3.1.1.cmml" xref="id3.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="id3.3.m3.1c">p</annotation></semantics></math> improves the coherence and quality of generated text even at high temperatures, while also facilitating more creative and diverse outputs compared to top-<math id="id4.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="id4.4.m4.1a"><mi id="id4.4.m4.1.1" xref="id4.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="id4.4.m4.1b"><ci id="id4.4.m4.1.1.cmml" xref="id4.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="id4.4.m4.1c">p</annotation></semantics></math> and other sampling methods. As of writing, min-<math id="id5.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="id5.5.m5.1a"><mi id="id5.5.m5.1.1" xref="id5.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="id5.5.m5.1b"><ci id="id5.5.m5.1.1.cmml" xref="id5.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="id5.5.m5.1c">p</annotation></semantics></math> has been adopted by multiple open-source LLM implementations, and have been independently assessed by members of the open-source LLM community, further validating its practical utility and potential<span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>Code, evaluation code and results available at <a target="_blank" href="https://github.com/menhguin/minp_paper/" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/menhguin/minp_paper/</a></span></span></span>.</p>
</div>
<div id="p1" class="ltx_para ltx_noindent">
<div id="p1.1" class="ltx_block ltx_align_bottom">
<p id="p1.1.1" class="ltx_p"><span id="p1.1.1.1" class="ltx_text ltx_font_bold">Min P Sampling: Balancing Creativity and Coherence at High
Temperature</span></p>
<br class="ltx_break ltx_centering">
<p id="p1.1.2" class="ltx_p ltx_align_center" style="width:433.6pt;"><span id="p1.1.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;">
<span id="p1.1.2.1.1" class="ltx_tabular ltx_align_top">
<span class="ltx_tbody">
<span id="p1.1.2.1.1.1.1" class="ltx_tr">
<span id="p1.1.2.1.1.1.1.1" class="ltx_td ltx_align_right"><span id="p1.1.2.1.1.1.1.1.1" class="ltx_text ltx_font_bold">
Nguyen Nhat Minh<span id="p1.1.2.1.1.1.1.1.1.1" class="ltx_note ltx_role_thanks"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">thanks: </span>Equal Contribution.   
Correspondence to <span id="p1.1.2.1.1.1.1.1.1.1.1" class="ltx_text">minh1228@gmail.com</span></span></span></span><sup id="p1.1.2.1.1.1.1.1.1.2" class="ltx_sup">1</sup>,
Andrew Baker<span id="footnotex3" class="ltx_note ltx_role_footnotemark"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotemark: </span></span></span></span><sup id="p1.1.2.1.1.1.1.1.1.3" class="ltx_sup">2</sup>,
Andreas Kirsch<sup id="p1.1.2.1.1.1.1.1.1.4" class="ltx_sup">2</sup>,
Clement Neo<span id="footnotex4" class="ltx_note ltx_role_footnotemark"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotemark: </span></span></span></span><sup id="p1.1.2.1.1.1.1.1.1.5" class="ltx_sup">1</sup>,</span></span></span>
<span id="p1.1.2.1.1.2.2" class="ltx_tr">
<span id="p1.1.2.1.1.2.2.1" class="ltx_td ltx_align_center"><sup id="p1.1.2.1.1.2.2.1.1" class="ltx_sup">1</sup>Apart Research
<sup id="p1.1.2.1.1.2.2.1.2" class="ltx_sup">2</sup>Independent</span></span>
</span>
</span></span></p>
<br class="ltx_break ltx_centering">
</div>
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para">
<p id="S1.p1.1" class="ltx_p">Large Language Models (LLMs) have shown a remarkable ability to generate coherent and creative text across various domains. However, the quality and diversity of the generated text depends heavily on the choice of sampling methods (<span id="S1.p1.1.1" class="ltx_text ltx_font_italic">samplers</span>) and hyperparameters, such as thresholds and sampling temperature, during the decoding process, where the model autoregressively samples the next token based on the probability distribution over its vocabulary <cite class="ltx_cite ltx_citemacro_citep">(Shi et al., <a href="#bib.bib15" title="" class="ltx_ref">2024</a>)</cite>.</p>
</div>
<figure id="S1.F1" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x1.png" id="S1.F1.g1" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="461" height="182" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x2.png" id="S1.F1.g2" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="461" height="182" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x3.png" id="S1.F1.g3" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="461" height="182" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x4.png" id="S1.F1.g4" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="461" height="182" alt="Refer to caption"></div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>Effects of top-<math id="S1.F1.8.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.F1.8.m1.1b"><mi id="S1.F1.8.m1.1.1" xref="S1.F1.8.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.F1.8.m1.1c"><ci id="S1.F1.8.m1.1.1.cmml" xref="S1.F1.8.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.8.m1.1d">p</annotation></semantics></math>, top-<math id="S1.F1.9.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S1.F1.9.m2.1b"><mi id="S1.F1.9.m2.1.1" xref="S1.F1.9.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S1.F1.9.m2.1c"><ci id="S1.F1.9.m2.1.1.cmml" xref="S1.F1.9.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.9.m2.1d">k</annotation></semantics></math>, and min-<math id="S1.F1.10.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.F1.10.m3.1b"><mi id="S1.F1.10.m3.1.1" xref="S1.F1.10.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.F1.10.m3.1c"><ci id="S1.F1.10.m3.1.1.cmml" xref="S1.F1.10.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.10.m3.1d">p</annotation></semantics></math> sampling on token probability distributions. Min-<math id="S1.F1.11.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.F1.11.m4.1b"><mi id="S1.F1.11.m4.1.1" xref="S1.F1.11.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.F1.11.m4.1c"><ci id="S1.F1.11.m4.1.1.cmml" xref="S1.F1.11.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.11.m4.1d">p</annotation></semantics></math> sampling dynamically adjusts its filtering threshold based on the model’s confidence, focusing on high-probability tokens when confident and including diverse but plausible options when uncertain. This helps min-<math id="S1.F1.12.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.F1.12.m5.1b"><mi id="S1.F1.12.m5.1.1" xref="S1.F1.12.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.F1.12.m5.1c"><ci id="S1.F1.12.m5.1.1.cmml" xref="S1.F1.12.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.12.m5.1d">p</annotation></semantics></math> balances coherence and diversity more effectively than top-<math id="S1.F1.13.m6.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.F1.13.m6.1b"><mi id="S1.F1.13.m6.1.1" xref="S1.F1.13.m6.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.F1.13.m6.1c"><ci id="S1.F1.13.m6.1.1.cmml" xref="S1.F1.13.m6.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.13.m6.1d">p</annotation></semantics></math> and top-<math id="S1.F1.14.m7.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S1.F1.14.m7.1b"><mi id="S1.F1.14.m7.1.1" xref="S1.F1.14.m7.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S1.F1.14.m7.1c"><ci id="S1.F1.14.m7.1.1.cmml" xref="S1.F1.14.m7.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.F1.14.m7.1d">k</annotation></semantics></math> sampling.</figcaption>
</figure>
<div id="S1.p2" class="ltx_para">
<p id="S1.p2.1" class="ltx_p">In particular, increasing the sampling temperature can enhance creativity by promoting more diverse token selection—but often at the cost of coherence, as the quality of the generated text tends to degrade quickly at higher temperature with nucleus sampling. This <span id="S1.p2.1.1" class="ltx_text ltx_font_italic">creativity-coherence trade-off</span> is the main reason low conservative temperatures between 0 and 1 are often chosen for sampling. In this work, we will examine whether the creativity-coherence trade-off can be relaxed, that is whether we can achieve higher creativity while preserving coherence.</p>
</div>
<div id="S1.p3" class="ltx_para">
<p id="S1.p3.3" class="ltx_p">As of writing, top-<math id="S1.p3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p3.1.m1.1a"><mi id="S1.p3.1.m1.1.1" xref="S1.p3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p3.1.m1.1b"><ci id="S1.p3.1.m1.1.1.cmml" xref="S1.p3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.1.m1.1c">p</annotation></semantics></math> sampling <cite class="ltx_cite ltx_citemacro_citep">(Holtzman et al., <a href="#bib.bib8" title="" class="ltx_ref">2020</a>)</cite> is the state-of-the-art sampling method that improves coherence by focusing on high-probability tokens and truncating the "unreliable tail" of low-probability tokens that can lead to degenerate text in top-<math id="S1.p3.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S1.p3.2.m2.1a"><mi id="S1.p3.2.m2.1.1" xref="S1.p3.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S1.p3.2.m2.1b"><ci id="S1.p3.2.m2.1.1.cmml" xref="S1.p3.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.2.m2.1c">k</annotation></semantics></math> sampling. However, top-<math id="S1.p3.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p3.3.m3.1a"><mi id="S1.p3.3.m3.1.1" xref="S1.p3.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p3.3.m3.1b"><ci id="S1.p3.3.m3.1.1.cmml" xref="S1.p3.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p3.3.m3.1c">p</annotation></semantics></math> sampling does not work well for higher temperatures: here, tokens from the "unreliable tail" can still enter the sampling pool.</p>
</div>
<div id="S1.p4" class="ltx_para">
<p id="S1.p4.2" class="ltx_p">Inspired by this observation, our proposed truncation sampling method, min-<math id="S1.p4.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p4.1.m1.1a"><mi id="S1.p4.1.m1.1.1" xref="S1.p4.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p4.1.m1.1b"><ci id="S1.p4.1.m1.1.1.cmml" xref="S1.p4.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p4.1.m1.1c">p</annotation></semantics></math>, dynamically sets a minimum probability threshold for token selection. Min-<math id="S1.p4.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p4.2.m2.1a"><mi id="S1.p4.2.m2.1.1" xref="S1.p4.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p4.2.m2.1b"><ci id="S1.p4.2.m2.1.1.cmml" xref="S1.p4.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p4.2.m2.1c">p</annotation></semantics></math> maintains coherence at higher temperatures without sacrificing creativity by eliminating the long tail.</p>
</div>
<div id="S1.p5" class="ltx_para">
<p id="S1.p5.2" class="ltx_p">In summary, our objectives for min-<math id="S1.p5.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p5.1.m1.1a"><mi id="S1.p5.1.m1.1.1" xref="S1.p5.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p5.1.m1.1b"><ci id="S1.p5.1.m1.1.1.cmml" xref="S1.p5.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p5.1.m1.1c">p</annotation></semantics></math> are to <span id="S1.p5.2.1" class="ltx_text ltx_font_bold">(1)</span> match or outperform the state-of-the-art top-<math id="S1.p5.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p5.2.m2.1a"><mi id="S1.p5.2.m2.1.1" xref="S1.p5.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p5.2.m2.1b"><ci id="S1.p5.2.m2.1.1.cmml" xref="S1.p5.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p5.2.m2.1c">p</annotation></semantics></math> sampling across various reasoning and performance benchmarks at standard temperature settings between 0 and 1, <span id="S1.p5.2.2" class="ltx_text ltx_font_bold">(2)</span> better handle the creativity-coherence trade-off at higher temperatures, and <span id="S1.p5.2.3" class="ltx_text ltx_font_bold">(3)</span> provide a simple, effective sampling method without relying on additional techniques like repetition penalties to address the externalities of high-tempature sampling.</p>
</div>
<div id="S1.p6" class="ltx_para">
<p id="S1.p6.5" class="ltx_p">To evaluate the performance of min-<math id="S1.p6.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p6.1.m1.1a"><mi id="S1.p6.1.m1.1.1" xref="S1.p6.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p6.1.m1.1b"><ci id="S1.p6.1.m1.1.1.cmml" xref="S1.p6.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p6.1.m1.1c">p</annotation></semantics></math> sampling, we conduct experiments comparing it with top-<math id="S1.p6.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p6.2.m2.1a"><mi id="S1.p6.2.m2.1.1" xref="S1.p6.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p6.2.m2.1b"><ci id="S1.p6.2.m2.1.1.cmml" xref="S1.p6.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p6.2.m2.1c">p</annotation></semantics></math> and other popular sampling methods on several LLM benchmarks, spanning open-ended question answering, grade school math, and creative writing. The results demonstrate min-<math id="S1.p6.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p6.3.m3.1a"><mi id="S1.p6.3.m3.1.1" xref="S1.p6.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p6.3.m3.1b"><ci id="S1.p6.3.m3.1.1.cmml" xref="S1.p6.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p6.3.m3.1c">p</annotation></semantics></math>’s ability to maintain coherence at higher temperatures while outperforming top-<math id="S1.p6.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p6.4.m4.1a"><mi id="S1.p6.4.m4.1.1" xref="S1.p6.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p6.4.m4.1b"><ci id="S1.p6.4.m4.1.1.cmml" xref="S1.p6.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p6.4.m4.1c">p</annotation></semantics></math> in terms of output diversity. The practical utility of min-<math id="S1.p6.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p6.5.m5.1a"><mi id="S1.p6.5.m5.1.1" xref="S1.p6.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p6.5.m5.1b"><ci id="S1.p6.5.m5.1.1.cmml" xref="S1.p6.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p6.5.m5.1c">p</annotation></semantics></math> is further validated by its adoption in multiple open-source LLM frameworks.</p>
</div>
<div id="S1.p7" class="ltx_para">
<p id="S1.p7.3" class="ltx_p">Our key contributions can be summarized as follows:</p>
<ol id="S1.I1" class="ltx_enumerate">
<li id="S1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S1.I1.i1.p1" class="ltx_para">
<p id="S1.I1.i1.p1.1" class="ltx_p">We introduce min-<math id="S1.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.I1.i1.p1.1.m1.1a"><mi id="S1.I1.i1.p1.1.m1.1.1" xref="S1.I1.i1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.I1.i1.p1.1.m1.1b"><ci id="S1.I1.i1.p1.1.m1.1.1.cmml" xref="S1.I1.i1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i1.p1.1.m1.1c">p</annotation></semantics></math>, a dynamic truncation sampling method that balances quality and diversity, leading to more creative and coherent outputs at higher temperature settings.</p>
</div>
</li>
<li id="S1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S1.I1.i2.p1" class="ltx_para">
<p id="S1.I1.i2.p1.2" class="ltx_p">We experimentally demonstrate min-<math id="S1.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.I1.i2.p1.1.m1.1a"><mi id="S1.I1.i2.p1.1.m1.1.1" xref="S1.I1.i2.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.I1.i2.p1.1.m1.1b"><ci id="S1.I1.i2.p1.1.m1.1.1.cmml" xref="S1.I1.i2.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i2.p1.1.m1.1c">p</annotation></semantics></math>’s advantages over top-<math id="S1.I1.i2.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.I1.i2.p1.2.m2.1a"><mi id="S1.I1.i2.p1.2.m2.1.1" xref="S1.I1.i2.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.I1.i2.p1.2.m2.1b"><ci id="S1.I1.i2.p1.2.m2.1.1.cmml" xref="S1.I1.i2.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.I1.i2.p1.2.m2.1c">p</annotation></semantics></math> and other samplers on LLM benchmarks, emphasizing its reduced tradeoffs between creativity and coherence at temperature settings beyond 1.</p>
</div>
</li>
</ol>
<p id="S1.p7.2" class="ltx_p">By achieving these objectives, we establish min-<math id="S1.p7.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p7.1.m1.1a"><mi id="S1.p7.1.m1.1.1" xref="S1.p7.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p7.1.m1.1b"><ci id="S1.p7.1.m1.1.1.cmml" xref="S1.p7.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p7.1.m1.1c">p</annotation></semantics></math> sampling as a viable and user-friendly alternative to top-<math id="S1.p7.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S1.p7.2.m2.1a"><mi id="S1.p7.2.m2.1.1" xref="S1.p7.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S1.p7.2.m2.1b"><ci id="S1.p7.2.m2.1.1.cmml" xref="S1.p7.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S1.p7.2.m2.1c">p</annotation></semantics></math> sampling, unlocking the benefits of high-temperature settings for enhanced creativity in LLM-generated text without sacrificing coherence.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Background</h2>

<section id="S2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.1 </span>Language Modelling</h3>

<div id="S2.SS1.p1" class="ltx_para">
<p id="S2.SS1.p1.6" class="ltx_p">Language models are probabilistic models that assign probabilities to sequences of words or tokens. An autoregressive language model predicts the probability distribution over the vocabulary for the next token, conditioned on the previous tokens, and iteratively samples tokens to generate a sequence. Formally, given a sequence of tokens <math id="S2.SS1.p1.1.m1.4" class="ltx_Math" alttext="\mathbf{x}=(x_{1},x_{2},\dots,x_{n})" display="inline"><semantics id="S2.SS1.p1.1.m1.4a"><mrow id="S2.SS1.p1.1.m1.4.4" xref="S2.SS1.p1.1.m1.4.4.cmml"><mi id="S2.SS1.p1.1.m1.4.4.5" xref="S2.SS1.p1.1.m1.4.4.5.cmml">𝐱</mi><mo id="S2.SS1.p1.1.m1.4.4.4" xref="S2.SS1.p1.1.m1.4.4.4.cmml">=</mo><mrow id="S2.SS1.p1.1.m1.4.4.3.3" xref="S2.SS1.p1.1.m1.4.4.3.4.cmml"><mo stretchy="false" id="S2.SS1.p1.1.m1.4.4.3.3.4" xref="S2.SS1.p1.1.m1.4.4.3.4.cmml">(</mo><msub id="S2.SS1.p1.1.m1.2.2.1.1.1" xref="S2.SS1.p1.1.m1.2.2.1.1.1.cmml"><mi id="S2.SS1.p1.1.m1.2.2.1.1.1.2" xref="S2.SS1.p1.1.m1.2.2.1.1.1.2.cmml">x</mi><mn id="S2.SS1.p1.1.m1.2.2.1.1.1.3" xref="S2.SS1.p1.1.m1.2.2.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.p1.1.m1.4.4.3.3.5" xref="S2.SS1.p1.1.m1.4.4.3.4.cmml">,</mo><msub id="S2.SS1.p1.1.m1.3.3.2.2.2" xref="S2.SS1.p1.1.m1.3.3.2.2.2.cmml"><mi id="S2.SS1.p1.1.m1.3.3.2.2.2.2" xref="S2.SS1.p1.1.m1.3.3.2.2.2.2.cmml">x</mi><mn id="S2.SS1.p1.1.m1.3.3.2.2.2.3" xref="S2.SS1.p1.1.m1.3.3.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.p1.1.m1.4.4.3.3.6" xref="S2.SS1.p1.1.m1.4.4.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.p1.1.m1.1.1" xref="S2.SS1.p1.1.m1.1.1.cmml">…</mi><mo id="S2.SS1.p1.1.m1.4.4.3.3.7" xref="S2.SS1.p1.1.m1.4.4.3.4.cmml">,</mo><msub id="S2.SS1.p1.1.m1.4.4.3.3.3" xref="S2.SS1.p1.1.m1.4.4.3.3.3.cmml"><mi id="S2.SS1.p1.1.m1.4.4.3.3.3.2" xref="S2.SS1.p1.1.m1.4.4.3.3.3.2.cmml">x</mi><mi id="S2.SS1.p1.1.m1.4.4.3.3.3.3" xref="S2.SS1.p1.1.m1.4.4.3.3.3.3.cmml">n</mi></msub><mo stretchy="false" id="S2.SS1.p1.1.m1.4.4.3.3.8" xref="S2.SS1.p1.1.m1.4.4.3.4.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.1.m1.4b"><apply id="S2.SS1.p1.1.m1.4.4.cmml" xref="S2.SS1.p1.1.m1.4.4"><eq id="S2.SS1.p1.1.m1.4.4.4.cmml" xref="S2.SS1.p1.1.m1.4.4.4"></eq><ci id="S2.SS1.p1.1.m1.4.4.5.cmml" xref="S2.SS1.p1.1.m1.4.4.5">𝐱</ci><vector id="S2.SS1.p1.1.m1.4.4.3.4.cmml" xref="S2.SS1.p1.1.m1.4.4.3.3"><apply id="S2.SS1.p1.1.m1.2.2.1.1.1.cmml" xref="S2.SS1.p1.1.m1.2.2.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.1.m1.2.2.1.1.1.1.cmml" xref="S2.SS1.p1.1.m1.2.2.1.1.1">subscript</csymbol><ci id="S2.SS1.p1.1.m1.2.2.1.1.1.2.cmml" xref="S2.SS1.p1.1.m1.2.2.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS1.p1.1.m1.2.2.1.1.1.3.cmml" xref="S2.SS1.p1.1.m1.2.2.1.1.1.3">1</cn></apply><apply id="S2.SS1.p1.1.m1.3.3.2.2.2.cmml" xref="S2.SS1.p1.1.m1.3.3.2.2.2"><csymbol cd="ambiguous" id="S2.SS1.p1.1.m1.3.3.2.2.2.1.cmml" xref="S2.SS1.p1.1.m1.3.3.2.2.2">subscript</csymbol><ci id="S2.SS1.p1.1.m1.3.3.2.2.2.2.cmml" xref="S2.SS1.p1.1.m1.3.3.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS1.p1.1.m1.3.3.2.2.2.3.cmml" xref="S2.SS1.p1.1.m1.3.3.2.2.2.3">2</cn></apply><ci id="S2.SS1.p1.1.m1.1.1.cmml" xref="S2.SS1.p1.1.m1.1.1">…</ci><apply id="S2.SS1.p1.1.m1.4.4.3.3.3.cmml" xref="S2.SS1.p1.1.m1.4.4.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.p1.1.m1.4.4.3.3.3.1.cmml" xref="S2.SS1.p1.1.m1.4.4.3.3.3">subscript</csymbol><ci id="S2.SS1.p1.1.m1.4.4.3.3.3.2.cmml" xref="S2.SS1.p1.1.m1.4.4.3.3.3.2">𝑥</ci><ci id="S2.SS1.p1.1.m1.4.4.3.3.3.3.cmml" xref="S2.SS1.p1.1.m1.4.4.3.3.3.3">𝑛</ci></apply></vector></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.1.m1.4c">\mathbf{x}=(x_{1},x_{2},\dots,x_{n})</annotation></semantics></math> and a finite vocabulary <math id="S2.SS1.p1.2.m2.1" class="ltx_Math" alttext="\mathcal{V}" display="inline"><semantics id="S2.SS1.p1.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S2.SS1.p1.2.m2.1.1" xref="S2.SS1.p1.2.m2.1.1.cmml">𝒱</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.2.m2.1b"><ci id="S2.SS1.p1.2.m2.1.1.cmml" xref="S2.SS1.p1.2.m2.1.1">𝒱</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.2.m2.1c">\mathcal{V}</annotation></semantics></math>, where each token <math id="S2.SS1.p1.3.m3.1" class="ltx_Math" alttext="x_{i}\in\mathcal{V}" display="inline"><semantics id="S2.SS1.p1.3.m3.1a"><mrow id="S2.SS1.p1.3.m3.1.1" xref="S2.SS1.p1.3.m3.1.1.cmml"><msub id="S2.SS1.p1.3.m3.1.1.2" xref="S2.SS1.p1.3.m3.1.1.2.cmml"><mi id="S2.SS1.p1.3.m3.1.1.2.2" xref="S2.SS1.p1.3.m3.1.1.2.2.cmml">x</mi><mi id="S2.SS1.p1.3.m3.1.1.2.3" xref="S2.SS1.p1.3.m3.1.1.2.3.cmml">i</mi></msub><mo id="S2.SS1.p1.3.m3.1.1.1" xref="S2.SS1.p1.3.m3.1.1.1.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S2.SS1.p1.3.m3.1.1.3" xref="S2.SS1.p1.3.m3.1.1.3.cmml">𝒱</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.3.m3.1b"><apply id="S2.SS1.p1.3.m3.1.1.cmml" xref="S2.SS1.p1.3.m3.1.1"><in id="S2.SS1.p1.3.m3.1.1.1.cmml" xref="S2.SS1.p1.3.m3.1.1.1"></in><apply id="S2.SS1.p1.3.m3.1.1.2.cmml" xref="S2.SS1.p1.3.m3.1.1.2"><csymbol cd="ambiguous" id="S2.SS1.p1.3.m3.1.1.2.1.cmml" xref="S2.SS1.p1.3.m3.1.1.2">subscript</csymbol><ci id="S2.SS1.p1.3.m3.1.1.2.2.cmml" xref="S2.SS1.p1.3.m3.1.1.2.2">𝑥</ci><ci id="S2.SS1.p1.3.m3.1.1.2.3.cmml" xref="S2.SS1.p1.3.m3.1.1.2.3">𝑖</ci></apply><ci id="S2.SS1.p1.3.m3.1.1.3.cmml" xref="S2.SS1.p1.3.m3.1.1.3">𝒱</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.3.m3.1c">x_{i}\in\mathcal{V}</annotation></semantics></math>, an autoregressive language model models the probability distribution <math id="S2.SS1.p1.4.m4.2" class="ltx_Math" alttext="P(x_{t}|x_{1},x_{2},\dots,x_{t-1})" display="inline"><semantics id="S2.SS1.p1.4.m4.2a"><mrow id="S2.SS1.p1.4.m4.2.2" xref="S2.SS1.p1.4.m4.2.2.cmml"><mi id="S2.SS1.p1.4.m4.2.2.3" xref="S2.SS1.p1.4.m4.2.2.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.SS1.p1.4.m4.2.2.2" xref="S2.SS1.p1.4.m4.2.2.2.cmml">​</mo><mrow id="S2.SS1.p1.4.m4.2.2.1.1" xref="S2.SS1.p1.4.m4.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.p1.4.m4.2.2.1.1.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.cmml">(</mo><mrow id="S2.SS1.p1.4.m4.2.2.1.1.1" xref="S2.SS1.p1.4.m4.2.2.1.1.1.cmml"><msub id="S2.SS1.p1.4.m4.2.2.1.1.1.5" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5.cmml"><mi id="S2.SS1.p1.4.m4.2.2.1.1.1.5.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5.2.cmml">x</mi><mi id="S2.SS1.p1.4.m4.2.2.1.1.1.5.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5.3.cmml">t</mi></msub><mo fence="false" id="S2.SS1.p1.4.m4.2.2.1.1.1.4" xref="S2.SS1.p1.4.m4.2.2.1.1.1.4.cmml">|</mo><mrow id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.4.cmml"><msub id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.cmml"><mi id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.4" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.cmml"><mi id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.2.cmml">x</mi><mn id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.5" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.p1.4.m4.1.1" xref="S2.SS1.p1.4.m4.1.1.cmml">…</mi><mo id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.6" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.cmml"><mi id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.2.cmml">x</mi><mrow id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.cmml"><mi id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.2" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.2.cmml">t</mi><mo id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.1" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.1.cmml">−</mo><mn id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S2.SS1.p1.4.m4.2.2.1.1.3" xref="S2.SS1.p1.4.m4.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.4.m4.2b"><apply id="S2.SS1.p1.4.m4.2.2.cmml" xref="S2.SS1.p1.4.m4.2.2"><times id="S2.SS1.p1.4.m4.2.2.2.cmml" xref="S2.SS1.p1.4.m4.2.2.2"></times><ci id="S2.SS1.p1.4.m4.2.2.3.cmml" xref="S2.SS1.p1.4.m4.2.2.3">𝑃</ci><apply id="S2.SS1.p1.4.m4.2.2.1.1.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1"><csymbol cd="latexml" id="S2.SS1.p1.4.m4.2.2.1.1.1.4.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.4">conditional</csymbol><apply id="S2.SS1.p1.4.m4.2.2.1.1.1.5.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5"><csymbol cd="ambiguous" id="S2.SS1.p1.4.m4.2.2.1.1.1.5.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5">subscript</csymbol><ci id="S2.SS1.p1.4.m4.2.2.1.1.1.5.2.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5.2">𝑥</ci><ci id="S2.SS1.p1.4.m4.2.2.1.1.1.5.3.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.5.3">𝑡</ci></apply><list id="S2.SS1.p1.4.m4.2.2.1.1.1.3.4.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3"><apply id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.2.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.3.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2">subscript</csymbol><ci id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.2.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.3.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.2.2.2.3">2</cn></apply><ci id="S2.SS1.p1.4.m4.1.1.cmml" xref="S2.SS1.p1.4.m4.1.1">…</ci><apply id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3">subscript</csymbol><ci id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.2.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.2">𝑥</ci><apply id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3"><minus id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.1.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.1"></minus><ci id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.2.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.3.cmml" xref="S2.SS1.p1.4.m4.2.2.1.1.1.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.4.m4.2c">P(x_{t}|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math>, for each token <math id="S2.SS1.p1.5.m5.1" class="ltx_Math" alttext="x_{t}" display="inline"><semantics id="S2.SS1.p1.5.m5.1a"><msub id="S2.SS1.p1.5.m5.1.1" xref="S2.SS1.p1.5.m5.1.1.cmml"><mi id="S2.SS1.p1.5.m5.1.1.2" xref="S2.SS1.p1.5.m5.1.1.2.cmml">x</mi><mi id="S2.SS1.p1.5.m5.1.1.3" xref="S2.SS1.p1.5.m5.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.5.m5.1b"><apply id="S2.SS1.p1.5.m5.1.1.cmml" xref="S2.SS1.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S2.SS1.p1.5.m5.1.1.1.cmml" xref="S2.SS1.p1.5.m5.1.1">subscript</csymbol><ci id="S2.SS1.p1.5.m5.1.1.2.cmml" xref="S2.SS1.p1.5.m5.1.1.2">𝑥</ci><ci id="S2.SS1.p1.5.m5.1.1.3.cmml" xref="S2.SS1.p1.5.m5.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.5.m5.1c">x_{t}</annotation></semantics></math> in the sequence. This probability distribution represents the likelihood of each token occurring at position <math id="S2.SS1.p1.6.m6.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS1.p1.6.m6.1a"><mi id="S2.SS1.p1.6.m6.1.1" xref="S2.SS1.p1.6.m6.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.p1.6.m6.1b"><ci id="S2.SS1.p1.6.m6.1.1.cmml" xref="S2.SS1.p1.6.m6.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.p1.6.m6.1c">t</annotation></semantics></math> given the preceding tokens.</p>
</div>
<section id="S2.SS1.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Greedy decoding.</h4>

<div id="S2.SS1.SSS0.Px1.p1" class="ltx_para">
<p id="S2.SS1.SSS0.Px1.p1.4" class="ltx_p">A deterministic way to sample tokens is <span id="S2.SS1.SSS0.Px1.p1.4.1" class="ltx_text ltx_font_italic">greedy decoding</span>, which selects the token with the highest probability at each step. Given the probability distribution <math id="S2.SS1.SSS0.Px1.p1.1.m1.2" class="ltx_Math" alttext="P(x_{t}|x_{1},x_{2},\dots,x_{t-1})" display="inline"><semantics id="S2.SS1.SSS0.Px1.p1.1.m1.2a"><mrow id="S2.SS1.SSS0.Px1.p1.1.m1.2.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.cmml"><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.2.cmml">​</mo><mrow id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.cmml">(</mo><mrow id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.cmml"><msub id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.cmml"><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.2.cmml">x</mi><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.3.cmml">t</mi></msub><mo fence="false" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.4" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.4.cmml">|</mo><mrow id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.4.cmml"><msub id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.cmml"><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.4" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.cmml"><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.2.cmml">x</mi><mn id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.5" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.SSS0.Px1.p1.1.m1.1.1" xref="S2.SS1.SSS0.Px1.p1.1.m1.1.1.cmml">…</mi><mo id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.6" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.cmml"><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.2.cmml">x</mi><mrow id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.cmml"><mi id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.2" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.2.cmml">t</mi><mo id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.1" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.1.cmml">−</mo><mn id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.3" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px1.p1.1.m1.2b"><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2"><times id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.2"></times><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.3">𝑃</ci><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1"><csymbol cd="latexml" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.4.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.4">conditional</csymbol><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5">subscript</csymbol><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.2">𝑥</ci><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.5.3">𝑡</ci></apply><list id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.4.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3"><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2">subscript</csymbol><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.2.2.2.3">2</cn></apply><ci id="S2.SS1.SSS0.Px1.p1.1.m1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.1.1">…</ci><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3">subscript</csymbol><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.2">𝑥</ci><apply id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3"><minus id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.1.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.1"></minus><ci id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.2.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.3.cmml" xref="S2.SS1.SSS0.Px1.p1.1.m1.2.2.1.1.1.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px1.p1.1.m1.2c">P(x_{t}|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math> over the vocabulary <math id="S2.SS1.SSS0.Px1.p1.2.m2.1" class="ltx_Math" alttext="\mathcal{V}" display="inline"><semantics id="S2.SS1.SSS0.Px1.p1.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S2.SS1.SSS0.Px1.p1.2.m2.1.1" xref="S2.SS1.SSS0.Px1.p1.2.m2.1.1.cmml">𝒱</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px1.p1.2.m2.1b"><ci id="S2.SS1.SSS0.Px1.p1.2.m2.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.2.m2.1.1">𝒱</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px1.p1.2.m2.1c">\mathcal{V}</annotation></semantics></math> at position <math id="S2.SS1.SSS0.Px1.p1.3.m3.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS1.SSS0.Px1.p1.3.m3.1a"><mi id="S2.SS1.SSS0.Px1.p1.3.m3.1.1" xref="S2.SS1.SSS0.Px1.p1.3.m3.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px1.p1.3.m3.1b"><ci id="S2.SS1.SSS0.Px1.p1.3.m3.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.3.m3.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px1.p1.3.m3.1c">t</annotation></semantics></math>, it chooses the token <math id="S2.SS1.SSS0.Px1.p1.4.m4.1" class="ltx_Math" alttext="\hat{x}_{t}" display="inline"><semantics id="S2.SS1.SSS0.Px1.p1.4.m4.1a"><msub id="S2.SS1.SSS0.Px1.p1.4.m4.1.1" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.cmml"><mover accent="true" id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.cmml"><mi id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.2" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.2.cmml">x</mi><mo id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.1" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.3" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px1.p1.4.m4.1b"><apply id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1">subscript</csymbol><apply id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.cmml" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2"><ci id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.1.cmml" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.1">^</ci><ci id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.2.2">𝑥</ci></apply><ci id="S2.SS1.SSS0.Px1.p1.4.m4.1.1.3.cmml" xref="S2.SS1.SSS0.Px1.p1.4.m4.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px1.p1.4.m4.1c">\hat{x}_{t}</annotation></semantics></math> that maximizes this probability:</p>
<table id="S2.E1" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E1.m1.2" class="ltx_Math" alttext="\hat{x}_{t}=\operatorname*{arg\,max}_{v\in\mathcal{V}}P(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})." display="block"><semantics id="S2.E1.m1.2a"><mrow id="S2.E1.m1.2.2.1" xref="S2.E1.m1.2.2.1.1.cmml"><mrow id="S2.E1.m1.2.2.1.1" xref="S2.E1.m1.2.2.1.1.cmml"><msub id="S2.E1.m1.2.2.1.1.3" xref="S2.E1.m1.2.2.1.1.3.cmml"><mover accent="true" id="S2.E1.m1.2.2.1.1.3.2" xref="S2.E1.m1.2.2.1.1.3.2.cmml"><mi id="S2.E1.m1.2.2.1.1.3.2.2" xref="S2.E1.m1.2.2.1.1.3.2.2.cmml">x</mi><mo id="S2.E1.m1.2.2.1.1.3.2.1" xref="S2.E1.m1.2.2.1.1.3.2.1.cmml">^</mo></mover><mi id="S2.E1.m1.2.2.1.1.3.3" xref="S2.E1.m1.2.2.1.1.3.3.cmml">t</mi></msub><mo id="S2.E1.m1.2.2.1.1.2" xref="S2.E1.m1.2.2.1.1.2.cmml">=</mo><mrow id="S2.E1.m1.2.2.1.1.1" xref="S2.E1.m1.2.2.1.1.1.cmml"><mrow id="S2.E1.m1.2.2.1.1.1.3" xref="S2.E1.m1.2.2.1.1.1.3.cmml"><munder id="S2.E1.m1.2.2.1.1.1.3.1" xref="S2.E1.m1.2.2.1.1.1.3.1.cmml"><mrow id="S2.E1.m1.2.2.1.1.1.3.1.2" xref="S2.E1.m1.2.2.1.1.1.3.1.2.cmml"><mi id="S2.E1.m1.2.2.1.1.1.3.1.2.2" xref="S2.E1.m1.2.2.1.1.1.3.1.2.2.cmml">arg</mi><mo lspace="0.170em" rspace="0em" id="S2.E1.m1.2.2.1.1.1.3.1.2.1" xref="S2.E1.m1.2.2.1.1.1.3.1.2.1.cmml">​</mo><mi id="S2.E1.m1.2.2.1.1.1.3.1.2.3" xref="S2.E1.m1.2.2.1.1.1.3.1.2.3.cmml">max</mi></mrow><mrow id="S2.E1.m1.2.2.1.1.1.3.1.3" xref="S2.E1.m1.2.2.1.1.1.3.1.3.cmml"><mi id="S2.E1.m1.2.2.1.1.1.3.1.3.2" xref="S2.E1.m1.2.2.1.1.1.3.1.3.2.cmml">v</mi><mo id="S2.E1.m1.2.2.1.1.1.3.1.3.1" xref="S2.E1.m1.2.2.1.1.1.3.1.3.1.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S2.E1.m1.2.2.1.1.1.3.1.3.3" xref="S2.E1.m1.2.2.1.1.1.3.1.3.3.cmml">𝒱</mi></mrow></munder><mo id="S2.E1.m1.2.2.1.1.1.3a" xref="S2.E1.m1.2.2.1.1.1.3.cmml">⁡</mo><mi id="S2.E1.m1.2.2.1.1.1.3.2" xref="S2.E1.m1.2.2.1.1.1.3.2.cmml">P</mi></mrow><mo lspace="0em" rspace="0em" id="S2.E1.m1.2.2.1.1.1.2" xref="S2.E1.m1.2.2.1.1.1.2.cmml">​</mo><mrow id="S2.E1.m1.2.2.1.1.1.1.1" xref="S2.E1.m1.2.2.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.E1.m1.2.2.1.1.1.1.1.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.cmml">(</mo><mrow id="S2.E1.m1.2.2.1.1.1.1.1.1" xref="S2.E1.m1.2.2.1.1.1.1.1.1.cmml"><msub id="S2.E1.m1.2.2.1.1.1.1.1.1.5" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5.cmml"><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.5.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5.2.cmml">x</mi><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.5.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5.3.cmml">t</mi></msub><mo id="S2.E1.m1.2.2.1.1.1.1.1.1.4" xref="S2.E1.m1.2.2.1.1.1.1.1.1.4.cmml">=</mo><mrow id="S2.E1.m1.2.2.1.1.1.1.1.1.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.cmml"><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.3.5" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.5.cmml">v</mi><mo fence="false" id="S2.E1.m1.2.2.1.1.1.1.1.1.3.4" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.4.cmml">|</mo><mrow id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.4.cmml"><msub id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.4" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.cmml"><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.2.cmml">x</mi><mn id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.3.cmml">2</mn></msub><mo id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.5" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.E1.m1.1.1" xref="S2.E1.m1.1.1.cmml">…</mi><mo id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.6" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.cmml"><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.2.cmml">x</mi><mrow id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.cmml"><mi id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2.cmml">t</mi><mo id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1.cmml">−</mo><mn id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.E1.m1.2.2.1.1.1.1.1.3" xref="S2.E1.m1.2.2.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="S2.E1.m1.2.2.1.2" xref="S2.E1.m1.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.E1.m1.2b"><apply id="S2.E1.m1.2.2.1.1.cmml" xref="S2.E1.m1.2.2.1"><eq id="S2.E1.m1.2.2.1.1.2.cmml" xref="S2.E1.m1.2.2.1.1.2"></eq><apply id="S2.E1.m1.2.2.1.1.3.cmml" xref="S2.E1.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S2.E1.m1.2.2.1.1.3.1.cmml" xref="S2.E1.m1.2.2.1.1.3">subscript</csymbol><apply id="S2.E1.m1.2.2.1.1.3.2.cmml" xref="S2.E1.m1.2.2.1.1.3.2"><ci id="S2.E1.m1.2.2.1.1.3.2.1.cmml" xref="S2.E1.m1.2.2.1.1.3.2.1">^</ci><ci id="S2.E1.m1.2.2.1.1.3.2.2.cmml" xref="S2.E1.m1.2.2.1.1.3.2.2">𝑥</ci></apply><ci id="S2.E1.m1.2.2.1.1.3.3.cmml" xref="S2.E1.m1.2.2.1.1.3.3">𝑡</ci></apply><apply id="S2.E1.m1.2.2.1.1.1.cmml" xref="S2.E1.m1.2.2.1.1.1"><times id="S2.E1.m1.2.2.1.1.1.2.cmml" xref="S2.E1.m1.2.2.1.1.1.2"></times><apply id="S2.E1.m1.2.2.1.1.1.3.cmml" xref="S2.E1.m1.2.2.1.1.1.3"><apply id="S2.E1.m1.2.2.1.1.1.3.1.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1"><csymbol cd="ambiguous" id="S2.E1.m1.2.2.1.1.1.3.1.1.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1">subscript</csymbol><apply id="S2.E1.m1.2.2.1.1.1.3.1.2.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.2"><times id="S2.E1.m1.2.2.1.1.1.3.1.2.1.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.2.1"></times><ci id="S2.E1.m1.2.2.1.1.1.3.1.2.2.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.2.2">arg</ci><ci id="S2.E1.m1.2.2.1.1.1.3.1.2.3.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.2.3">max</ci></apply><apply id="S2.E1.m1.2.2.1.1.1.3.1.3.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.3"><in id="S2.E1.m1.2.2.1.1.1.3.1.3.1.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.3.1"></in><ci id="S2.E1.m1.2.2.1.1.1.3.1.3.2.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.3.2">𝑣</ci><ci id="S2.E1.m1.2.2.1.1.1.3.1.3.3.cmml" xref="S2.E1.m1.2.2.1.1.1.3.1.3.3">𝒱</ci></apply></apply><ci id="S2.E1.m1.2.2.1.1.1.3.2.cmml" xref="S2.E1.m1.2.2.1.1.1.3.2">𝑃</ci></apply><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1"><eq id="S2.E1.m1.2.2.1.1.1.1.1.1.4.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.4"></eq><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.5.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5"><csymbol cd="ambiguous" id="S2.E1.m1.2.2.1.1.1.1.1.1.5.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5">subscript</csymbol><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.5.2.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5.2">𝑥</ci><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.5.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.5.3">𝑡</ci></apply><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3"><csymbol cd="latexml" id="S2.E1.m1.2.2.1.1.1.1.1.1.3.4.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.4">conditional</csymbol><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.3.5.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.5">𝑣</ci><list id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.4.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3"><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.2.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.2">𝑥</ci><cn type="integer" id="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.2.2.2.2.3">2</cn></apply><ci id="S2.E1.m1.1.1.cmml" xref="S2.E1.m1.1.1">…</ci><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3"><csymbol cd="ambiguous" id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3">subscript</csymbol><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.2.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.2">𝑥</ci><apply id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3"><minus id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1"></minus><ci id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3.cmml" xref="S2.E1.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E1.m1.2c">\hat{x}_{t}=\operatorname*{arg\,max}_{v\in\mathcal{V}}P(x_{t}=v|x_{1},x_{2},\dots,x_{t-1}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(1)</span></td>
</tr></tbody>
</table>
<p id="S2.SS1.SSS0.Px1.p1.6" class="ltx_p">By repeatedly selecting the most probable token at each step, we can generate a sequence of tokens <math id="S2.SS1.SSS0.Px1.p1.5.m1.4" class="ltx_Math" alttext="(\hat{x}_{1},\hat{x}_{2},\dots,\hat{x}_{m})" display="inline"><semantics id="S2.SS1.SSS0.Px1.p1.5.m1.4a"><mrow id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml"><mo stretchy="false" id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.4" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml">(</mo><msub id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.cmml"><mover accent="true" id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.cmml"><mi id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.2.cmml">x</mi><mo id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.1" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.1.cmml">^</mo></mover><mn id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.3" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.5" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml">,</mo><msub id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.cmml"><mover accent="true" id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.cmml"><mi id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.2.cmml">x</mi><mo id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.1" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.1.cmml">^</mo></mover><mn id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.3" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.6" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.SSS0.Px1.p1.5.m1.1.1" xref="S2.SS1.SSS0.Px1.p1.5.m1.1.1.cmml">…</mi><mo id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.7" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml">,</mo><msub id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.cmml"><mover accent="true" id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.cmml"><mi id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.2" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.2.cmml">x</mi><mo id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.1" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.1.cmml">^</mo></mover><mi id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.3" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.3.cmml">m</mi></msub><mo stretchy="false" id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.8" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml">)</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px1.p1.5.m1.4b"><vector id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.4.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3"><apply id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1">subscript</csymbol><apply id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2"><ci id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.1">^</ci><ci id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.2.2">𝑥</ci></apply><cn type="integer" id="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.3.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.2.2.1.1.3">1</cn></apply><apply id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2">subscript</csymbol><apply id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2"><ci id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.1">^</ci><ci id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.2.2">𝑥</ci></apply><cn type="integer" id="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.3.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.3.3.2.2.3">2</cn></apply><ci id="S2.SS1.SSS0.Px1.p1.5.m1.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.1.1">…</ci><apply id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3">subscript</csymbol><apply id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2"><ci id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.1.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.1">^</ci><ci id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.2.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.2.2">𝑥</ci></apply><ci id="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.3.cmml" xref="S2.SS1.SSS0.Px1.p1.5.m1.4.4.3.3.3">𝑚</ci></apply></vector></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px1.p1.5.m1.4c">(\hat{x}_{1},\hat{x}_{2},\dots,\hat{x}_{m})</annotation></semantics></math> of arbitrary length <math id="S2.SS1.SSS0.Px1.p1.6.m2.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S2.SS1.SSS0.Px1.p1.6.m2.1a"><mi id="S2.SS1.SSS0.Px1.p1.6.m2.1.1" xref="S2.SS1.SSS0.Px1.p1.6.m2.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px1.p1.6.m2.1b"><ci id="S2.SS1.SSS0.Px1.p1.6.m2.1.1.cmml" xref="S2.SS1.SSS0.Px1.p1.6.m2.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px1.p1.6.m2.1c">m</annotation></semantics></math>. However, this greedy approach often leads to degenerate output, as it always chooses the most likely continuation without considering other plausible alternatives. This deterministic sampling has found to often result in repetitive or generic generated text that lacks diversity and creativity.</p>
</div>
</section>
<section id="S2.SS1.SSS0.Px2" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Stochastic Sampling.</h4>

<div id="S2.SS1.SSS0.Px2.p1" class="ltx_para">
<p id="S2.SS1.SSS0.Px2.p1.1" class="ltx_p">The stochastic alternative to greedy decoding is sampling of tokens according to the probability distribution <math id="S2.SS1.SSS0.Px2.p1.1.m1.2" class="ltx_Math" alttext="P(x_{t}|x_{1},x_{2},\dots,x_{t-1})" display="inline"><semantics id="S2.SS1.SSS0.Px2.p1.1.m1.2a"><mrow id="S2.SS1.SSS0.Px2.p1.1.m1.2.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.cmml"><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.2.cmml">​</mo><mrow id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.cmml">(</mo><mrow id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.cmml"><msub id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.cmml"><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.2.cmml">x</mi><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.3.cmml">t</mi></msub><mo fence="false" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.4" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.4.cmml">|</mo><mrow id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.4.cmml"><msub id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.cmml"><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.4" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.cmml"><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.2.cmml">x</mi><mn id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.5" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS1.SSS0.Px2.p1.1.m1.1.1" xref="S2.SS1.SSS0.Px2.p1.1.m1.1.1.cmml">…</mi><mo id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.6" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.cmml"><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.2.cmml">x</mi><mrow id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.cmml"><mi id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.2" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.2.cmml">t</mi><mo id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.1" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.1.cmml">−</mo><mn id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.3" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS1.SSS0.Px2.p1.1.m1.2b"><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2"><times id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.2"></times><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.3">𝑃</ci><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1"><csymbol cd="latexml" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.4.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.4">conditional</csymbol><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5">subscript</csymbol><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.2">𝑥</ci><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.5.3">𝑡</ci></apply><list id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.4.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3"><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2">subscript</csymbol><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.2.2.2.3">2</cn></apply><ci id="S2.SS1.SSS0.Px2.p1.1.m1.1.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.1.1">…</ci><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3">subscript</csymbol><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.2">𝑥</ci><apply id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3"><minus id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.1.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.1"></minus><ci id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.2.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.3.cmml" xref="S2.SS1.SSS0.Px2.p1.1.m1.2.2.1.1.1.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS1.SSS0.Px2.p1.1.m1.2c">P(x_{t}|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math>:</p>
<table id="S2.E2" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E2.m1.2" class="ltx_Math" alttext="\hat{x}_{t}\sim P(x_{t}=\hat{x}_{t}|x_{1},x_{2},\dots,x_{t-1})." display="block"><semantics id="S2.E2.m1.2a"><mrow id="S2.E2.m1.2.2.1" xref="S2.E2.m1.2.2.1.1.cmml"><mrow id="S2.E2.m1.2.2.1.1" xref="S2.E2.m1.2.2.1.1.cmml"><msub id="S2.E2.m1.2.2.1.1.3" xref="S2.E2.m1.2.2.1.1.3.cmml"><mover accent="true" id="S2.E2.m1.2.2.1.1.3.2" xref="S2.E2.m1.2.2.1.1.3.2.cmml"><mi id="S2.E2.m1.2.2.1.1.3.2.2" xref="S2.E2.m1.2.2.1.1.3.2.2.cmml">x</mi><mo id="S2.E2.m1.2.2.1.1.3.2.1" xref="S2.E2.m1.2.2.1.1.3.2.1.cmml">^</mo></mover><mi id="S2.E2.m1.2.2.1.1.3.3" xref="S2.E2.m1.2.2.1.1.3.3.cmml">t</mi></msub><mo id="S2.E2.m1.2.2.1.1.2" xref="S2.E2.m1.2.2.1.1.2.cmml">∼</mo><mrow id="S2.E2.m1.2.2.1.1.1" xref="S2.E2.m1.2.2.1.1.1.cmml"><mi id="S2.E2.m1.2.2.1.1.1.3" xref="S2.E2.m1.2.2.1.1.1.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.E2.m1.2.2.1.1.1.2" xref="S2.E2.m1.2.2.1.1.1.2.cmml">​</mo><mrow id="S2.E2.m1.2.2.1.1.1.1.1" xref="S2.E2.m1.2.2.1.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.E2.m1.2.2.1.1.1.1.1.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.cmml">(</mo><mrow id="S2.E2.m1.2.2.1.1.1.1.1.1" xref="S2.E2.m1.2.2.1.1.1.1.1.1.cmml"><msub id="S2.E2.m1.2.2.1.1.1.1.1.1.5" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5.cmml"><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.5.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5.2.cmml">x</mi><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.5.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5.3.cmml">t</mi></msub><mo id="S2.E2.m1.2.2.1.1.1.1.1.1.4" xref="S2.E2.m1.2.2.1.1.1.1.1.1.4.cmml">=</mo><mrow id="S2.E2.m1.2.2.1.1.1.1.1.1.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.cmml"><msub id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.cmml"><mover accent="true" id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.cmml"><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.2.cmml">x</mi><mo id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.1" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.1.cmml">^</mo></mover><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.3.cmml">t</mi></msub><mo fence="false" id="S2.E2.m1.2.2.1.1.1.1.1.1.3.4" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.4.cmml">|</mo><mrow id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.4.cmml"><msub id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.cmml"><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.4" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.cmml"><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.2.cmml">x</mi><mn id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.3.cmml">2</mn></msub><mo id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.5" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.E2.m1.1.1" xref="S2.E2.m1.1.1.cmml">…</mi><mo id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.6" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.cmml"><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.2.cmml">x</mi><mrow id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.cmml"><mi id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2.cmml">t</mi><mo id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1.cmml">−</mo><mn id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.E2.m1.2.2.1.1.1.1.1.3" xref="S2.E2.m1.2.2.1.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo lspace="0em" id="S2.E2.m1.2.2.1.2" xref="S2.E2.m1.2.2.1.1.cmml">.</mo></mrow><annotation-xml encoding="MathML-Content" id="S2.E2.m1.2b"><apply id="S2.E2.m1.2.2.1.1.cmml" xref="S2.E2.m1.2.2.1"><csymbol cd="latexml" id="S2.E2.m1.2.2.1.1.2.cmml" xref="S2.E2.m1.2.2.1.1.2">similar-to</csymbol><apply id="S2.E2.m1.2.2.1.1.3.cmml" xref="S2.E2.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S2.E2.m1.2.2.1.1.3.1.cmml" xref="S2.E2.m1.2.2.1.1.3">subscript</csymbol><apply id="S2.E2.m1.2.2.1.1.3.2.cmml" xref="S2.E2.m1.2.2.1.1.3.2"><ci id="S2.E2.m1.2.2.1.1.3.2.1.cmml" xref="S2.E2.m1.2.2.1.1.3.2.1">^</ci><ci id="S2.E2.m1.2.2.1.1.3.2.2.cmml" xref="S2.E2.m1.2.2.1.1.3.2.2">𝑥</ci></apply><ci id="S2.E2.m1.2.2.1.1.3.3.cmml" xref="S2.E2.m1.2.2.1.1.3.3">𝑡</ci></apply><apply id="S2.E2.m1.2.2.1.1.1.cmml" xref="S2.E2.m1.2.2.1.1.1"><times id="S2.E2.m1.2.2.1.1.1.2.cmml" xref="S2.E2.m1.2.2.1.1.1.2"></times><ci id="S2.E2.m1.2.2.1.1.1.3.cmml" xref="S2.E2.m1.2.2.1.1.1.3">𝑃</ci><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1"><eq id="S2.E2.m1.2.2.1.1.1.1.1.1.4.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.4"></eq><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.5.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5"><csymbol cd="ambiguous" id="S2.E2.m1.2.2.1.1.1.1.1.1.5.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5">subscript</csymbol><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.5.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5.2">𝑥</ci><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.5.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.5.3">𝑡</ci></apply><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3"><csymbol cd="latexml" id="S2.E2.m1.2.2.1.1.1.1.1.1.3.4.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.4">conditional</csymbol><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5"><csymbol cd="ambiguous" id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5">subscript</csymbol><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2"><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.1">^</ci><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.2.2">𝑥</ci></apply><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.5.3">𝑡</ci></apply><list id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.4.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3"><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.2">𝑥</ci><cn type="integer" id="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.2.2.2.2.3">2</cn></apply><ci id="S2.E2.m1.1.1.cmml" xref="S2.E2.m1.1.1">…</ci><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3"><csymbol cd="ambiguous" id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3">subscript</csymbol><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.2">𝑥</ci><apply id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3"><minus id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.1"></minus><ci id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3.cmml" xref="S2.E2.m1.2.2.1.1.1.1.1.1.3.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E2.m1.2c">\hat{x}_{t}\sim P(x_{t}=\hat{x}_{t}|x_{1},x_{2},\dots,x_{t-1}).</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(2)</span></td>
</tr></tbody>
</table>
<p id="S2.SS1.SSS0.Px2.p1.2" class="ltx_p">In this case, there are several ways to modify and constrain the sampling distribution to improve the quality of the generated text. We will look at the most commonly used approaches next.</p>
</div>
</section>
</section>
<section id="S2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.2 </span>Temperature Scaling</h3>

<div id="S2.SS2.p1" class="ltx_para">
<p id="S2.SS2.p1.1" class="ltx_p">Temperature scaling modifies the sharpness of the token distribution. The term temperature is inspired by thermodynamics and the Boltzmann distribution: higher temperatures correspond to more uniform mixing that more uniform sampling while lower temperatures converge to the argmax <cite class="ltx_cite ltx_citemacro_cite">Ackley et al. (<a href="#bib.bib1" title="" class="ltx_ref">1985</a>)</cite>.</p>
</div>
<div id="S2.SS2.p2" class="ltx_para">
<p id="S2.SS2.p2.5" class="ltx_p">Formally, given a probability distribution <math id="S2.SS2.p2.1.m1.2" class="ltx_Math" alttext="P(x_{t}|x_{1},x_{2},\dots,x_{t-1})" display="inline"><semantics id="S2.SS2.p2.1.m1.2a"><mrow id="S2.SS2.p2.1.m1.2.2" xref="S2.SS2.p2.1.m1.2.2.cmml"><mi id="S2.SS2.p2.1.m1.2.2.3" xref="S2.SS2.p2.1.m1.2.2.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.SS2.p2.1.m1.2.2.2" xref="S2.SS2.p2.1.m1.2.2.2.cmml">​</mo><mrow id="S2.SS2.p2.1.m1.2.2.1.1" xref="S2.SS2.p2.1.m1.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS2.p2.1.m1.2.2.1.1.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.cmml">(</mo><mrow id="S2.SS2.p2.1.m1.2.2.1.1.1" xref="S2.SS2.p2.1.m1.2.2.1.1.1.cmml"><msub id="S2.SS2.p2.1.m1.2.2.1.1.1.5" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5.cmml"><mi id="S2.SS2.p2.1.m1.2.2.1.1.1.5.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5.2.cmml">x</mi><mi id="S2.SS2.p2.1.m1.2.2.1.1.1.5.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5.3.cmml">t</mi></msub><mo fence="false" id="S2.SS2.p2.1.m1.2.2.1.1.1.4" xref="S2.SS2.p2.1.m1.2.2.1.1.1.4.cmml">|</mo><mrow id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.4.cmml"><msub id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.cmml"><mi id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.4" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.cmml"><mi id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.2.cmml">x</mi><mn id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.5" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS2.p2.1.m1.1.1" xref="S2.SS2.p2.1.m1.1.1.cmml">…</mi><mo id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.6" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.4.cmml">,</mo><msub id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.cmml"><mi id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.2.cmml">x</mi><mrow id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.cmml"><mi id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.2" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.2.cmml">t</mi><mo id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.1" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.1.cmml">−</mo><mn id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S2.SS2.p2.1.m1.2.2.1.1.3" xref="S2.SS2.p2.1.m1.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.1.m1.2b"><apply id="S2.SS2.p2.1.m1.2.2.cmml" xref="S2.SS2.p2.1.m1.2.2"><times id="S2.SS2.p2.1.m1.2.2.2.cmml" xref="S2.SS2.p2.1.m1.2.2.2"></times><ci id="S2.SS2.p2.1.m1.2.2.3.cmml" xref="S2.SS2.p2.1.m1.2.2.3">𝑃</ci><apply id="S2.SS2.p2.1.m1.2.2.1.1.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1"><csymbol cd="latexml" id="S2.SS2.p2.1.m1.2.2.1.1.1.4.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.4">conditional</csymbol><apply id="S2.SS2.p2.1.m1.2.2.1.1.1.5.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5"><csymbol cd="ambiguous" id="S2.SS2.p2.1.m1.2.2.1.1.1.5.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5">subscript</csymbol><ci id="S2.SS2.p2.1.m1.2.2.1.1.1.5.2.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5.2">𝑥</ci><ci id="S2.SS2.p2.1.m1.2.2.1.1.1.5.3.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.5.3">𝑡</ci></apply><list id="S2.SS2.p2.1.m1.2.2.1.1.1.3.4.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3"><apply id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.2.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.3.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2">subscript</csymbol><ci id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.2.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.3.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.2.2.2.3">2</cn></apply><ci id="S2.SS2.p2.1.m1.1.1.cmml" xref="S2.SS2.p2.1.m1.1.1">…</ci><apply id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3">subscript</csymbol><ci id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.2.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.2">𝑥</ci><apply id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3"><minus id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.1.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.1"></minus><ci id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.2.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.3.cmml" xref="S2.SS2.p2.1.m1.2.2.1.1.1.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.1.m1.2c">P(x_{t}|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math> over the vocabulary <math id="S2.SS2.p2.2.m2.1" class="ltx_Math" alttext="\mathcal{V}" display="inline"><semantics id="S2.SS2.p2.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S2.SS2.p2.2.m2.1.1" xref="S2.SS2.p2.2.m2.1.1.cmml">𝒱</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.2.m2.1b"><ci id="S2.SS2.p2.2.m2.1.1.cmml" xref="S2.SS2.p2.2.m2.1.1">𝒱</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.2.m2.1c">\mathcal{V}</annotation></semantics></math> at position <math id="S2.SS2.p2.3.m3.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS2.p2.3.m3.1a"><mi id="S2.SS2.p2.3.m3.1.1" xref="S2.SS2.p2.3.m3.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.3.m3.1b"><ci id="S2.SS2.p2.3.m3.1.1.cmml" xref="S2.SS2.p2.3.m3.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.3.m3.1c">t</annotation></semantics></math>, temperature sampling first divides the logits (unnormalized log probabilities) by a temperature parameter <math id="S2.SS2.p2.4.m4.1" class="ltx_Math" alttext="\tau&gt;0" display="inline"><semantics id="S2.SS2.p2.4.m4.1a"><mrow id="S2.SS2.p2.4.m4.1.1" xref="S2.SS2.p2.4.m4.1.1.cmml"><mi id="S2.SS2.p2.4.m4.1.1.2" xref="S2.SS2.p2.4.m4.1.1.2.cmml">τ</mi><mo id="S2.SS2.p2.4.m4.1.1.1" xref="S2.SS2.p2.4.m4.1.1.1.cmml">&gt;</mo><mn id="S2.SS2.p2.4.m4.1.1.3" xref="S2.SS2.p2.4.m4.1.1.3.cmml">0</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.4.m4.1b"><apply id="S2.SS2.p2.4.m4.1.1.cmml" xref="S2.SS2.p2.4.m4.1.1"><gt id="S2.SS2.p2.4.m4.1.1.1.cmml" xref="S2.SS2.p2.4.m4.1.1.1"></gt><ci id="S2.SS2.p2.4.m4.1.1.2.cmml" xref="S2.SS2.p2.4.m4.1.1.2">𝜏</ci><cn type="integer" id="S2.SS2.p2.4.m4.1.1.3.cmml" xref="S2.SS2.p2.4.m4.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.4.m4.1c">\tau&gt;0</annotation></semantics></math> before passing them through the softmax function to obtain a modified probability distribution <math id="S2.SS2.p2.5.m5.2" class="ltx_Math" alttext="P_{\tau}(x_{t}=v|x_{1},\dots,x_{t-1})" display="inline"><semantics id="S2.SS2.p2.5.m5.2a"><mrow id="S2.SS2.p2.5.m5.2.2" xref="S2.SS2.p2.5.m5.2.2.cmml"><msub id="S2.SS2.p2.5.m5.2.2.3" xref="S2.SS2.p2.5.m5.2.2.3.cmml"><mi id="S2.SS2.p2.5.m5.2.2.3.2" xref="S2.SS2.p2.5.m5.2.2.3.2.cmml">P</mi><mi id="S2.SS2.p2.5.m5.2.2.3.3" xref="S2.SS2.p2.5.m5.2.2.3.3.cmml">τ</mi></msub><mo lspace="0em" rspace="0em" id="S2.SS2.p2.5.m5.2.2.2" xref="S2.SS2.p2.5.m5.2.2.2.cmml">​</mo><mrow id="S2.SS2.p2.5.m5.2.2.1.1" xref="S2.SS2.p2.5.m5.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS2.p2.5.m5.2.2.1.1.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.cmml">(</mo><mrow id="S2.SS2.p2.5.m5.2.2.1.1.1" xref="S2.SS2.p2.5.m5.2.2.1.1.1.cmml"><msub id="S2.SS2.p2.5.m5.2.2.1.1.1.4" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4.cmml"><mi id="S2.SS2.p2.5.m5.2.2.1.1.1.4.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4.2.cmml">x</mi><mi id="S2.SS2.p2.5.m5.2.2.1.1.1.4.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4.3.cmml">t</mi></msub><mo id="S2.SS2.p2.5.m5.2.2.1.1.1.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.3.cmml">=</mo><mrow id="S2.SS2.p2.5.m5.2.2.1.1.1.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.cmml"><mi id="S2.SS2.p2.5.m5.2.2.1.1.1.2.4" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.4.cmml">v</mi><mo fence="false" id="S2.SS2.p2.5.m5.2.2.1.1.1.2.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.3.cmml">|</mo><mrow id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.3.cmml"><msub id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.cmml"><mi id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.SS2.p2.5.m5.1.1" xref="S2.SS2.p2.5.m5.1.1.cmml">…</mi><mo id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.4" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.3.cmml">,</mo><msub id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.cmml"><mi id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.2.cmml">x</mi><mrow id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.cmml"><mi id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.2" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.2.cmml">t</mi><mo id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.1" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.1.cmml">−</mo><mn id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.SS2.p2.5.m5.2.2.1.1.3" xref="S2.SS2.p2.5.m5.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.5.m5.2b"><apply id="S2.SS2.p2.5.m5.2.2.cmml" xref="S2.SS2.p2.5.m5.2.2"><times id="S2.SS2.p2.5.m5.2.2.2.cmml" xref="S2.SS2.p2.5.m5.2.2.2"></times><apply id="S2.SS2.p2.5.m5.2.2.3.cmml" xref="S2.SS2.p2.5.m5.2.2.3"><csymbol cd="ambiguous" id="S2.SS2.p2.5.m5.2.2.3.1.cmml" xref="S2.SS2.p2.5.m5.2.2.3">subscript</csymbol><ci id="S2.SS2.p2.5.m5.2.2.3.2.cmml" xref="S2.SS2.p2.5.m5.2.2.3.2">𝑃</ci><ci id="S2.SS2.p2.5.m5.2.2.3.3.cmml" xref="S2.SS2.p2.5.m5.2.2.3.3">𝜏</ci></apply><apply id="S2.SS2.p2.5.m5.2.2.1.1.1.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1"><eq id="S2.SS2.p2.5.m5.2.2.1.1.1.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.3"></eq><apply id="S2.SS2.p2.5.m5.2.2.1.1.1.4.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4"><csymbol cd="ambiguous" id="S2.SS2.p2.5.m5.2.2.1.1.1.4.1.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4">subscript</csymbol><ci id="S2.SS2.p2.5.m5.2.2.1.1.1.4.2.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4.2">𝑥</ci><ci id="S2.SS2.p2.5.m5.2.2.1.1.1.4.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.4.3">𝑡</ci></apply><apply id="S2.SS2.p2.5.m5.2.2.1.1.1.2.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2"><csymbol cd="latexml" id="S2.SS2.p2.5.m5.2.2.1.1.1.2.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.3">conditional</csymbol><ci id="S2.SS2.p2.5.m5.2.2.1.1.1.2.4.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.4">𝑣</ci><list id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2"><apply id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.1.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.2.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.1.1.1.1.3">1</cn></apply><ci id="S2.SS2.p2.5.m5.1.1.cmml" xref="S2.SS2.p2.5.m5.1.1">…</ci><apply id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.1.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.2.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.2">𝑥</ci><apply id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3"><minus id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.1.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.1"></minus><ci id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.2.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.2">𝑡</ci><cn type="integer" id="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.3.cmml" xref="S2.SS2.p2.5.m5.2.2.1.1.1.2.2.2.2.3.3">1</cn></apply></apply></list></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.5.m5.2c">P_{\tau}(x_{t}=v|x_{1},\dots,x_{t-1})</annotation></semantics></math>:</p>
<table id="S2.E3" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E3.m1.6" class="ltx_Math" alttext="P_{\tau}(x_{t}=v|x_{1},\dots,x_{t-1})=\frac{\exp(z_{v}/\tau)}{\sum_{v^{\prime}\in\mathcal{V}}\exp(z_{v^{\prime}}/\tau)}" display="block"><semantics id="S2.E3.m1.6a"><mrow id="S2.E3.m1.6.6" xref="S2.E3.m1.6.6.cmml"><mrow id="S2.E3.m1.6.6.1" xref="S2.E3.m1.6.6.1.cmml"><msub id="S2.E3.m1.6.6.1.3" xref="S2.E3.m1.6.6.1.3.cmml"><mi id="S2.E3.m1.6.6.1.3.2" xref="S2.E3.m1.6.6.1.3.2.cmml">P</mi><mi id="S2.E3.m1.6.6.1.3.3" xref="S2.E3.m1.6.6.1.3.3.cmml">τ</mi></msub><mo lspace="0em" rspace="0em" id="S2.E3.m1.6.6.1.2" xref="S2.E3.m1.6.6.1.2.cmml">​</mo><mrow id="S2.E3.m1.6.6.1.1.1" xref="S2.E3.m1.6.6.1.1.1.1.cmml"><mo stretchy="false" id="S2.E3.m1.6.6.1.1.1.2" xref="S2.E3.m1.6.6.1.1.1.1.cmml">(</mo><mrow id="S2.E3.m1.6.6.1.1.1.1" xref="S2.E3.m1.6.6.1.1.1.1.cmml"><msub id="S2.E3.m1.6.6.1.1.1.1.4" xref="S2.E3.m1.6.6.1.1.1.1.4.cmml"><mi id="S2.E3.m1.6.6.1.1.1.1.4.2" xref="S2.E3.m1.6.6.1.1.1.1.4.2.cmml">x</mi><mi id="S2.E3.m1.6.6.1.1.1.1.4.3" xref="S2.E3.m1.6.6.1.1.1.1.4.3.cmml">t</mi></msub><mo id="S2.E3.m1.6.6.1.1.1.1.3" xref="S2.E3.m1.6.6.1.1.1.1.3.cmml">=</mo><mrow id="S2.E3.m1.6.6.1.1.1.1.2" xref="S2.E3.m1.6.6.1.1.1.1.2.cmml"><mi id="S2.E3.m1.6.6.1.1.1.1.2.4" xref="S2.E3.m1.6.6.1.1.1.1.2.4.cmml">v</mi><mo fence="false" id="S2.E3.m1.6.6.1.1.1.1.2.3" xref="S2.E3.m1.6.6.1.1.1.1.2.3.cmml">|</mo><mrow id="S2.E3.m1.6.6.1.1.1.1.2.2.2" xref="S2.E3.m1.6.6.1.1.1.1.2.2.3.cmml"><msub id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.cmml"><mi id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.2" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.3" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.E3.m1.6.6.1.1.1.1.2.2.2.3" xref="S2.E3.m1.6.6.1.1.1.1.2.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.E3.m1.5.5" xref="S2.E3.m1.5.5.cmml">…</mi><mo id="S2.E3.m1.6.6.1.1.1.1.2.2.2.4" xref="S2.E3.m1.6.6.1.1.1.1.2.2.3.cmml">,</mo><msub id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.cmml"><mi id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.2" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.2.cmml">x</mi><mrow id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.cmml"><mi id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.2" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.2.cmml">t</mi><mo id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.1" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.1.cmml">−</mo><mn id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.3" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.E3.m1.6.6.1.1.1.3" xref="S2.E3.m1.6.6.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S2.E3.m1.6.6.2" xref="S2.E3.m1.6.6.2.cmml">=</mo><mfrac id="S2.E3.m1.4.4" xref="S2.E3.m1.4.4.cmml"><mrow id="S2.E3.m1.2.2.2.2" xref="S2.E3.m1.2.2.2.3.cmml"><mi id="S2.E3.m1.1.1.1.1" xref="S2.E3.m1.1.1.1.1.cmml">exp</mi><mo id="S2.E3.m1.2.2.2.2a" xref="S2.E3.m1.2.2.2.3.cmml">⁡</mo><mrow id="S2.E3.m1.2.2.2.2.1" xref="S2.E3.m1.2.2.2.3.cmml"><mo stretchy="false" id="S2.E3.m1.2.2.2.2.1.2" xref="S2.E3.m1.2.2.2.3.cmml">(</mo><mrow id="S2.E3.m1.2.2.2.2.1.1" xref="S2.E3.m1.2.2.2.2.1.1.cmml"><msub id="S2.E3.m1.2.2.2.2.1.1.2" xref="S2.E3.m1.2.2.2.2.1.1.2.cmml"><mi id="S2.E3.m1.2.2.2.2.1.1.2.2" xref="S2.E3.m1.2.2.2.2.1.1.2.2.cmml">z</mi><mi id="S2.E3.m1.2.2.2.2.1.1.2.3" xref="S2.E3.m1.2.2.2.2.1.1.2.3.cmml">v</mi></msub><mo id="S2.E3.m1.2.2.2.2.1.1.1" xref="S2.E3.m1.2.2.2.2.1.1.1.cmml">/</mo><mi id="S2.E3.m1.2.2.2.2.1.1.3" xref="S2.E3.m1.2.2.2.2.1.1.3.cmml">τ</mi></mrow><mo stretchy="false" id="S2.E3.m1.2.2.2.2.1.3" xref="S2.E3.m1.2.2.2.3.cmml">)</mo></mrow></mrow><mrow id="S2.E3.m1.4.4.4" xref="S2.E3.m1.4.4.4.cmml"><msub id="S2.E3.m1.4.4.4.3" xref="S2.E3.m1.4.4.4.3.cmml"><mo id="S2.E3.m1.4.4.4.3.2" xref="S2.E3.m1.4.4.4.3.2.cmml">∑</mo><mrow id="S2.E3.m1.4.4.4.3.3" xref="S2.E3.m1.4.4.4.3.3.cmml"><msup id="S2.E3.m1.4.4.4.3.3.2" xref="S2.E3.m1.4.4.4.3.3.2.cmml"><mi id="S2.E3.m1.4.4.4.3.3.2.2" xref="S2.E3.m1.4.4.4.3.3.2.2.cmml">v</mi><mo id="S2.E3.m1.4.4.4.3.3.2.3" xref="S2.E3.m1.4.4.4.3.3.2.3.cmml">′</mo></msup><mo id="S2.E3.m1.4.4.4.3.3.1" xref="S2.E3.m1.4.4.4.3.3.1.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S2.E3.m1.4.4.4.3.3.3" xref="S2.E3.m1.4.4.4.3.3.3.cmml">𝒱</mi></mrow></msub><mrow id="S2.E3.m1.4.4.4.2.1" xref="S2.E3.m1.4.4.4.2.2.cmml"><mi id="S2.E3.m1.3.3.3.1" xref="S2.E3.m1.3.3.3.1.cmml">exp</mi><mo id="S2.E3.m1.4.4.4.2.1a" xref="S2.E3.m1.4.4.4.2.2.cmml">⁡</mo><mrow id="S2.E3.m1.4.4.4.2.1.1" xref="S2.E3.m1.4.4.4.2.2.cmml"><mo stretchy="false" id="S2.E3.m1.4.4.4.2.1.1.2" xref="S2.E3.m1.4.4.4.2.2.cmml">(</mo><mrow id="S2.E3.m1.4.4.4.2.1.1.1" xref="S2.E3.m1.4.4.4.2.1.1.1.cmml"><msub id="S2.E3.m1.4.4.4.2.1.1.1.2" xref="S2.E3.m1.4.4.4.2.1.1.1.2.cmml"><mi id="S2.E3.m1.4.4.4.2.1.1.1.2.2" xref="S2.E3.m1.4.4.4.2.1.1.1.2.2.cmml">z</mi><msup id="S2.E3.m1.4.4.4.2.1.1.1.2.3" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3.cmml"><mi id="S2.E3.m1.4.4.4.2.1.1.1.2.3.2" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3.2.cmml">v</mi><mo id="S2.E3.m1.4.4.4.2.1.1.1.2.3.3" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3.3.cmml">′</mo></msup></msub><mo id="S2.E3.m1.4.4.4.2.1.1.1.1" xref="S2.E3.m1.4.4.4.2.1.1.1.1.cmml">/</mo><mi id="S2.E3.m1.4.4.4.2.1.1.1.3" xref="S2.E3.m1.4.4.4.2.1.1.1.3.cmml">τ</mi></mrow><mo stretchy="false" id="S2.E3.m1.4.4.4.2.1.1.3" xref="S2.E3.m1.4.4.4.2.2.cmml">)</mo></mrow></mrow></mrow></mfrac></mrow><annotation-xml encoding="MathML-Content" id="S2.E3.m1.6b"><apply id="S2.E3.m1.6.6.cmml" xref="S2.E3.m1.6.6"><eq id="S2.E3.m1.6.6.2.cmml" xref="S2.E3.m1.6.6.2"></eq><apply id="S2.E3.m1.6.6.1.cmml" xref="S2.E3.m1.6.6.1"><times id="S2.E3.m1.6.6.1.2.cmml" xref="S2.E3.m1.6.6.1.2"></times><apply id="S2.E3.m1.6.6.1.3.cmml" xref="S2.E3.m1.6.6.1.3"><csymbol cd="ambiguous" id="S2.E3.m1.6.6.1.3.1.cmml" xref="S2.E3.m1.6.6.1.3">subscript</csymbol><ci id="S2.E3.m1.6.6.1.3.2.cmml" xref="S2.E3.m1.6.6.1.3.2">𝑃</ci><ci id="S2.E3.m1.6.6.1.3.3.cmml" xref="S2.E3.m1.6.6.1.3.3">𝜏</ci></apply><apply id="S2.E3.m1.6.6.1.1.1.1.cmml" xref="S2.E3.m1.6.6.1.1.1"><eq id="S2.E3.m1.6.6.1.1.1.1.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.3"></eq><apply id="S2.E3.m1.6.6.1.1.1.1.4.cmml" xref="S2.E3.m1.6.6.1.1.1.1.4"><csymbol cd="ambiguous" id="S2.E3.m1.6.6.1.1.1.1.4.1.cmml" xref="S2.E3.m1.6.6.1.1.1.1.4">subscript</csymbol><ci id="S2.E3.m1.6.6.1.1.1.1.4.2.cmml" xref="S2.E3.m1.6.6.1.1.1.1.4.2">𝑥</ci><ci id="S2.E3.m1.6.6.1.1.1.1.4.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.4.3">𝑡</ci></apply><apply id="S2.E3.m1.6.6.1.1.1.1.2.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2"><csymbol cd="latexml" id="S2.E3.m1.6.6.1.1.1.1.2.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.3">conditional</csymbol><ci id="S2.E3.m1.6.6.1.1.1.1.2.4.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.4">𝑣</ci><list id="S2.E3.m1.6.6.1.1.1.1.2.2.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2"><apply id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.cmml" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.1.1.1.1.3">1</cn></apply><ci id="S2.E3.m1.5.5.cmml" xref="S2.E3.m1.5.5">…</ci><apply id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.1.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.2.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.2">𝑥</ci><apply id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3"><minus id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.1.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.1"></minus><ci id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.2.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.2">𝑡</ci><cn type="integer" id="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.3.cmml" xref="S2.E3.m1.6.6.1.1.1.1.2.2.2.2.3.3">1</cn></apply></apply></list></apply></apply></apply><apply id="S2.E3.m1.4.4.cmml" xref="S2.E3.m1.4.4"><divide id="S2.E3.m1.4.4.5.cmml" xref="S2.E3.m1.4.4"></divide><apply id="S2.E3.m1.2.2.2.3.cmml" xref="S2.E3.m1.2.2.2.2"><exp id="S2.E3.m1.1.1.1.1.cmml" xref="S2.E3.m1.1.1.1.1"></exp><apply id="S2.E3.m1.2.2.2.2.1.1.cmml" xref="S2.E3.m1.2.2.2.2.1.1"><divide id="S2.E3.m1.2.2.2.2.1.1.1.cmml" xref="S2.E3.m1.2.2.2.2.1.1.1"></divide><apply id="S2.E3.m1.2.2.2.2.1.1.2.cmml" xref="S2.E3.m1.2.2.2.2.1.1.2"><csymbol cd="ambiguous" id="S2.E3.m1.2.2.2.2.1.1.2.1.cmml" xref="S2.E3.m1.2.2.2.2.1.1.2">subscript</csymbol><ci id="S2.E3.m1.2.2.2.2.1.1.2.2.cmml" xref="S2.E3.m1.2.2.2.2.1.1.2.2">𝑧</ci><ci id="S2.E3.m1.2.2.2.2.1.1.2.3.cmml" xref="S2.E3.m1.2.2.2.2.1.1.2.3">𝑣</ci></apply><ci id="S2.E3.m1.2.2.2.2.1.1.3.cmml" xref="S2.E3.m1.2.2.2.2.1.1.3">𝜏</ci></apply></apply><apply id="S2.E3.m1.4.4.4.cmml" xref="S2.E3.m1.4.4.4"><apply id="S2.E3.m1.4.4.4.3.cmml" xref="S2.E3.m1.4.4.4.3"><csymbol cd="ambiguous" id="S2.E3.m1.4.4.4.3.1.cmml" xref="S2.E3.m1.4.4.4.3">subscript</csymbol><sum id="S2.E3.m1.4.4.4.3.2.cmml" xref="S2.E3.m1.4.4.4.3.2"></sum><apply id="S2.E3.m1.4.4.4.3.3.cmml" xref="S2.E3.m1.4.4.4.3.3"><in id="S2.E3.m1.4.4.4.3.3.1.cmml" xref="S2.E3.m1.4.4.4.3.3.1"></in><apply id="S2.E3.m1.4.4.4.3.3.2.cmml" xref="S2.E3.m1.4.4.4.3.3.2"><csymbol cd="ambiguous" id="S2.E3.m1.4.4.4.3.3.2.1.cmml" xref="S2.E3.m1.4.4.4.3.3.2">superscript</csymbol><ci id="S2.E3.m1.4.4.4.3.3.2.2.cmml" xref="S2.E3.m1.4.4.4.3.3.2.2">𝑣</ci><ci id="S2.E3.m1.4.4.4.3.3.2.3.cmml" xref="S2.E3.m1.4.4.4.3.3.2.3">′</ci></apply><ci id="S2.E3.m1.4.4.4.3.3.3.cmml" xref="S2.E3.m1.4.4.4.3.3.3">𝒱</ci></apply></apply><apply id="S2.E3.m1.4.4.4.2.2.cmml" xref="S2.E3.m1.4.4.4.2.1"><exp id="S2.E3.m1.3.3.3.1.cmml" xref="S2.E3.m1.3.3.3.1"></exp><apply id="S2.E3.m1.4.4.4.2.1.1.1.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1"><divide id="S2.E3.m1.4.4.4.2.1.1.1.1.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.1"></divide><apply id="S2.E3.m1.4.4.4.2.1.1.1.2.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2"><csymbol cd="ambiguous" id="S2.E3.m1.4.4.4.2.1.1.1.2.1.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2">subscript</csymbol><ci id="S2.E3.m1.4.4.4.2.1.1.1.2.2.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2.2">𝑧</ci><apply id="S2.E3.m1.4.4.4.2.1.1.1.2.3.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3"><csymbol cd="ambiguous" id="S2.E3.m1.4.4.4.2.1.1.1.2.3.1.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3">superscript</csymbol><ci id="S2.E3.m1.4.4.4.2.1.1.1.2.3.2.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3.2">𝑣</ci><ci id="S2.E3.m1.4.4.4.2.1.1.1.2.3.3.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.2.3.3">′</ci></apply></apply><ci id="S2.E3.m1.4.4.4.2.1.1.1.3.cmml" xref="S2.E3.m1.4.4.4.2.1.1.1.3">𝜏</ci></apply></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E3.m1.6c">P_{\tau}(x_{t}=v|x_{1},\dots,x_{t-1})=\frac{\exp(z_{v}/\tau)}{\sum_{v^{\prime}\in\mathcal{V}}\exp(z_{v^{\prime}}/\tau)}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(3)</span></td>
</tr></tbody>
</table>
<p id="S2.SS2.p2.8" class="ltx_p">where <math id="S2.SS2.p2.6.m1.2" class="ltx_Math" alttext="z_{v}=\log P(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})" display="inline"><semantics id="S2.SS2.p2.6.m1.2a"><mrow id="S2.SS2.p2.6.m1.2.2" xref="S2.SS2.p2.6.m1.2.2.cmml"><msub id="S2.SS2.p2.6.m1.2.2.3" xref="S2.SS2.p2.6.m1.2.2.3.cmml"><mi id="S2.SS2.p2.6.m1.2.2.3.2" xref="S2.SS2.p2.6.m1.2.2.3.2.cmml">z</mi><mi id="S2.SS2.p2.6.m1.2.2.3.3" xref="S2.SS2.p2.6.m1.2.2.3.3.cmml">v</mi></msub><mo id="S2.SS2.p2.6.m1.2.2.2" xref="S2.SS2.p2.6.m1.2.2.2.cmml">=</mo><mrow id="S2.SS2.p2.6.m1.2.2.1" xref="S2.SS2.p2.6.m1.2.2.1.cmml"><mrow id="S2.SS2.p2.6.m1.2.2.1.3" xref="S2.SS2.p2.6.m1.2.2.1.3.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.3.1" xref="S2.SS2.p2.6.m1.2.2.1.3.1.cmml">log</mi><mo lspace="0.167em" id="S2.SS2.p2.6.m1.2.2.1.3a" xref="S2.SS2.p2.6.m1.2.2.1.3.cmml">⁡</mo><mi id="S2.SS2.p2.6.m1.2.2.1.3.2" xref="S2.SS2.p2.6.m1.2.2.1.3.2.cmml">P</mi></mrow><mo lspace="0em" rspace="0em" id="S2.SS2.p2.6.m1.2.2.1.2" xref="S2.SS2.p2.6.m1.2.2.1.2.cmml">​</mo><mrow id="S2.SS2.p2.6.m1.2.2.1.1.1" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.cmml"><mo stretchy="false" id="S2.SS2.p2.6.m1.2.2.1.1.1.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.cmml">(</mo><mrow id="S2.SS2.p2.6.m1.2.2.1.1.1.1" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.cmml"><msub id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.2.cmml">x</mi><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.3.cmml">t</mi></msub><mo id="S2.SS2.p2.6.m1.2.2.1.1.1.1.4" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.4.cmml">=</mo><mrow id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.5" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.5.cmml">v</mi><mo fence="false" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.4" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.4.cmml">|</mo><mrow id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.4.cmml"><msub id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.4" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.2.cmml">x</mi><mn id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.5" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS2.p2.6.m1.1.1" xref="S2.SS2.p2.6.m1.1.1.cmml">…</mi><mo id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.6" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.2.cmml">x</mi><mrow id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.cmml"><mi id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.2" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.2.cmml">t</mi><mo id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.1" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.1.cmml">−</mo><mn id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.SS2.p2.6.m1.2.2.1.1.1.3" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.6.m1.2b"><apply id="S2.SS2.p2.6.m1.2.2.cmml" xref="S2.SS2.p2.6.m1.2.2"><eq id="S2.SS2.p2.6.m1.2.2.2.cmml" xref="S2.SS2.p2.6.m1.2.2.2"></eq><apply id="S2.SS2.p2.6.m1.2.2.3.cmml" xref="S2.SS2.p2.6.m1.2.2.3"><csymbol cd="ambiguous" id="S2.SS2.p2.6.m1.2.2.3.1.cmml" xref="S2.SS2.p2.6.m1.2.2.3">subscript</csymbol><ci id="S2.SS2.p2.6.m1.2.2.3.2.cmml" xref="S2.SS2.p2.6.m1.2.2.3.2">𝑧</ci><ci id="S2.SS2.p2.6.m1.2.2.3.3.cmml" xref="S2.SS2.p2.6.m1.2.2.3.3">𝑣</ci></apply><apply id="S2.SS2.p2.6.m1.2.2.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1"><times id="S2.SS2.p2.6.m1.2.2.1.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.2"></times><apply id="S2.SS2.p2.6.m1.2.2.1.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.3"><log id="S2.SS2.p2.6.m1.2.2.1.3.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.3.1"></log><ci id="S2.SS2.p2.6.m1.2.2.1.3.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.3.2">𝑃</ci></apply><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1"><eq id="S2.SS2.p2.6.m1.2.2.1.1.1.1.4.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.4"></eq><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5"><csymbol cd="ambiguous" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5">subscript</csymbol><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.2">𝑥</ci><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.5.3">𝑡</ci></apply><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3"><csymbol cd="latexml" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.4.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.4">conditional</csymbol><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.5.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.5">𝑣</ci><list id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.4.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3"><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.2.2.2.2.3">2</cn></apply><ci id="S2.SS2.p2.6.m1.1.1.cmml" xref="S2.SS2.p2.6.m1.1.1">…</ci><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3"><csymbol cd="ambiguous" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3">subscript</csymbol><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.2">𝑥</ci><apply id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3"><minus id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.1.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.1"></minus><ci id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.2.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.3.cmml" xref="S2.SS2.p2.6.m1.2.2.1.1.1.1.3.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.6.m1.2c">z_{v}=\log P(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math> is the (unnormalized) log probability of token <math id="S2.SS2.p2.7.m2.1" class="ltx_Math" alttext="v" display="inline"><semantics id="S2.SS2.p2.7.m2.1a"><mi id="S2.SS2.p2.7.m2.1.1" xref="S2.SS2.p2.7.m2.1.1.cmml">v</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.7.m2.1b"><ci id="S2.SS2.p2.7.m2.1.1.cmml" xref="S2.SS2.p2.7.m2.1.1">𝑣</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.7.m2.1c">v</annotation></semantics></math>.
The next token <math id="S2.SS2.p2.8.m3.1" class="ltx_Math" alttext="\hat{x}_{t}" display="inline"><semantics id="S2.SS2.p2.8.m3.1a"><msub id="S2.SS2.p2.8.m3.1.1" xref="S2.SS2.p2.8.m3.1.1.cmml"><mover accent="true" id="S2.SS2.p2.8.m3.1.1.2" xref="S2.SS2.p2.8.m3.1.1.2.cmml"><mi id="S2.SS2.p2.8.m3.1.1.2.2" xref="S2.SS2.p2.8.m3.1.1.2.2.cmml">x</mi><mo id="S2.SS2.p2.8.m3.1.1.2.1" xref="S2.SS2.p2.8.m3.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS2.p2.8.m3.1.1.3" xref="S2.SS2.p2.8.m3.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.8.m3.1b"><apply id="S2.SS2.p2.8.m3.1.1.cmml" xref="S2.SS2.p2.8.m3.1.1"><csymbol cd="ambiguous" id="S2.SS2.p2.8.m3.1.1.1.cmml" xref="S2.SS2.p2.8.m3.1.1">subscript</csymbol><apply id="S2.SS2.p2.8.m3.1.1.2.cmml" xref="S2.SS2.p2.8.m3.1.1.2"><ci id="S2.SS2.p2.8.m3.1.1.2.1.cmml" xref="S2.SS2.p2.8.m3.1.1.2.1">^</ci><ci id="S2.SS2.p2.8.m3.1.1.2.2.cmml" xref="S2.SS2.p2.8.m3.1.1.2.2">𝑥</ci></apply><ci id="S2.SS2.p2.8.m3.1.1.3.cmml" xref="S2.SS2.p2.8.m3.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.8.m3.1c">\hat{x}_{t}</annotation></semantics></math> is then randomly sampled from the modified probability distribution:</p>
<table id="S2.E4" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E4.m1.2" class="ltx_Math" alttext="\hat{x}_{t}\sim P\tau(v|x_{1},x_{2},\dots,x_{t-1})" display="block"><semantics id="S2.E4.m1.2a"><mrow id="S2.E4.m1.2.2" xref="S2.E4.m1.2.2.cmml"><msub id="S2.E4.m1.2.2.3" xref="S2.E4.m1.2.2.3.cmml"><mover accent="true" id="S2.E4.m1.2.2.3.2" xref="S2.E4.m1.2.2.3.2.cmml"><mi id="S2.E4.m1.2.2.3.2.2" xref="S2.E4.m1.2.2.3.2.2.cmml">x</mi><mo id="S2.E4.m1.2.2.3.2.1" xref="S2.E4.m1.2.2.3.2.1.cmml">^</mo></mover><mi id="S2.E4.m1.2.2.3.3" xref="S2.E4.m1.2.2.3.3.cmml">t</mi></msub><mo id="S2.E4.m1.2.2.2" xref="S2.E4.m1.2.2.2.cmml">∼</mo><mrow id="S2.E4.m1.2.2.1" xref="S2.E4.m1.2.2.1.cmml"><mi id="S2.E4.m1.2.2.1.3" xref="S2.E4.m1.2.2.1.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S2.E4.m1.2.2.1.2" xref="S2.E4.m1.2.2.1.2.cmml">​</mo><mi id="S2.E4.m1.2.2.1.4" xref="S2.E4.m1.2.2.1.4.cmml">τ</mi><mo lspace="0em" rspace="0em" id="S2.E4.m1.2.2.1.2a" xref="S2.E4.m1.2.2.1.2.cmml">​</mo><mrow id="S2.E4.m1.2.2.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.cmml"><mo stretchy="false" id="S2.E4.m1.2.2.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.cmml">(</mo><mrow id="S2.E4.m1.2.2.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.5" xref="S2.E4.m1.2.2.1.1.1.1.5.cmml">v</mi><mo fence="false" id="S2.E4.m1.2.2.1.1.1.1.4" xref="S2.E4.m1.2.2.1.1.1.1.4.cmml">|</mo><mrow id="S2.E4.m1.2.2.1.1.1.1.3.3" xref="S2.E4.m1.2.2.1.1.1.1.3.4.cmml"><msub id="S2.E4.m1.2.2.1.1.1.1.1.1.1" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.E4.m1.2.2.1.1.1.1.1.1.1.3" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.E4.m1.2.2.1.1.1.1.3.3.4" xref="S2.E4.m1.2.2.1.1.1.1.3.4.cmml">,</mo><msub id="S2.E4.m1.2.2.1.1.1.1.2.2.2" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.2.2.2.2" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2.2.cmml">x</mi><mn id="S2.E4.m1.2.2.1.1.1.1.2.2.2.3" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2.3.cmml">2</mn></msub><mo id="S2.E4.m1.2.2.1.1.1.1.3.3.5" xref="S2.E4.m1.2.2.1.1.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.E4.m1.1.1" xref="S2.E4.m1.1.1.cmml">…</mi><mo id="S2.E4.m1.2.2.1.1.1.1.3.3.6" xref="S2.E4.m1.2.2.1.1.1.1.3.4.cmml">,</mo><msub id="S2.E4.m1.2.2.1.1.1.1.3.3.3" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.3.3.3.2" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.2.cmml">x</mi><mrow id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.cmml"><mi id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.2" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.2.cmml">t</mi><mo id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.1" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.1.cmml">−</mo><mn id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.3" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S2.E4.m1.2.2.1.1.1.3" xref="S2.E4.m1.2.2.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.E4.m1.2b"><apply id="S2.E4.m1.2.2.cmml" xref="S2.E4.m1.2.2"><csymbol cd="latexml" id="S2.E4.m1.2.2.2.cmml" xref="S2.E4.m1.2.2.2">similar-to</csymbol><apply id="S2.E4.m1.2.2.3.cmml" xref="S2.E4.m1.2.2.3"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.3.1.cmml" xref="S2.E4.m1.2.2.3">subscript</csymbol><apply id="S2.E4.m1.2.2.3.2.cmml" xref="S2.E4.m1.2.2.3.2"><ci id="S2.E4.m1.2.2.3.2.1.cmml" xref="S2.E4.m1.2.2.3.2.1">^</ci><ci id="S2.E4.m1.2.2.3.2.2.cmml" xref="S2.E4.m1.2.2.3.2.2">𝑥</ci></apply><ci id="S2.E4.m1.2.2.3.3.cmml" xref="S2.E4.m1.2.2.3.3">𝑡</ci></apply><apply id="S2.E4.m1.2.2.1.cmml" xref="S2.E4.m1.2.2.1"><times id="S2.E4.m1.2.2.1.2.cmml" xref="S2.E4.m1.2.2.1.2"></times><ci id="S2.E4.m1.2.2.1.3.cmml" xref="S2.E4.m1.2.2.1.3">𝑃</ci><ci id="S2.E4.m1.2.2.1.4.cmml" xref="S2.E4.m1.2.2.1.4">𝜏</ci><apply id="S2.E4.m1.2.2.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1"><csymbol cd="latexml" id="S2.E4.m1.2.2.1.1.1.1.4.cmml" xref="S2.E4.m1.2.2.1.1.1.1.4">conditional</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.5.cmml" xref="S2.E4.m1.2.2.1.1.1.1.5">𝑣</ci><list id="S2.E4.m1.2.2.1.1.1.1.3.4.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3"><apply id="S2.E4.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.E4.m1.2.2.1.1.1.1.1.1.1.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.E4.m1.2.2.1.1.1.1.2.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.2.2.2.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.2.2.2.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2.2">𝑥</ci><cn type="integer" id="S2.E4.m1.2.2.1.1.1.1.2.2.2.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.2.2.2.3">2</cn></apply><ci id="S2.E4.m1.1.1.cmml" xref="S2.E4.m1.1.1">…</ci><apply id="S2.E4.m1.2.2.1.1.1.1.3.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S2.E4.m1.2.2.1.1.1.1.3.3.3.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3">subscript</csymbol><ci id="S2.E4.m1.2.2.1.1.1.1.3.3.3.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.2">𝑥</ci><apply id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3"><minus id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.1.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.1"></minus><ci id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.2.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.3.cmml" xref="S2.E4.m1.2.2.1.1.1.1.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E4.m1.2c">\hat{x}_{t}\sim P\tau(v|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(4)</span></td>
</tr></tbody>
</table>
<p id="S2.SS2.p2.14" class="ltx_p">The temperature parameter <math id="S2.SS2.p2.9.m1.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="S2.SS2.p2.9.m1.1a"><mi id="S2.SS2.p2.9.m1.1.1" xref="S2.SS2.p2.9.m1.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.9.m1.1b"><ci id="S2.SS2.p2.9.m1.1.1.cmml" xref="S2.SS2.p2.9.m1.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.9.m1.1c">\tau</annotation></semantics></math> controls the randomness of the sampling process. Higher values of <math id="S2.SS2.p2.10.m2.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="S2.SS2.p2.10.m2.1a"><mi id="S2.SS2.p2.10.m2.1.1" xref="S2.SS2.p2.10.m2.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.10.m2.1b"><ci id="S2.SS2.p2.10.m2.1.1.cmml" xref="S2.SS2.p2.10.m2.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.10.m2.1c">\tau</annotation></semantics></math> (e.g., <math id="S2.SS2.p2.11.m3.1" class="ltx_Math" alttext="\tau&gt;1" display="inline"><semantics id="S2.SS2.p2.11.m3.1a"><mrow id="S2.SS2.p2.11.m3.1.1" xref="S2.SS2.p2.11.m3.1.1.cmml"><mi id="S2.SS2.p2.11.m3.1.1.2" xref="S2.SS2.p2.11.m3.1.1.2.cmml">τ</mi><mo id="S2.SS2.p2.11.m3.1.1.1" xref="S2.SS2.p2.11.m3.1.1.1.cmml">&gt;</mo><mn id="S2.SS2.p2.11.m3.1.1.3" xref="S2.SS2.p2.11.m3.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.11.m3.1b"><apply id="S2.SS2.p2.11.m3.1.1.cmml" xref="S2.SS2.p2.11.m3.1.1"><gt id="S2.SS2.p2.11.m3.1.1.1.cmml" xref="S2.SS2.p2.11.m3.1.1.1"></gt><ci id="S2.SS2.p2.11.m3.1.1.2.cmml" xref="S2.SS2.p2.11.m3.1.1.2">𝜏</ci><cn type="integer" id="S2.SS2.p2.11.m3.1.1.3.cmml" xref="S2.SS2.p2.11.m3.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.11.m3.1c">\tau&gt;1</annotation></semantics></math>) result in a more uniform probability distribution, increasing the chances of sampling lower-probability tokens and generating more diverse output. Lower values of <math id="S2.SS2.p2.12.m4.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="S2.SS2.p2.12.m4.1a"><mi id="S2.SS2.p2.12.m4.1.1" xref="S2.SS2.p2.12.m4.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.12.m4.1b"><ci id="S2.SS2.p2.12.m4.1.1.cmml" xref="S2.SS2.p2.12.m4.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.12.m4.1c">\tau</annotation></semantics></math> (e.g., <math id="S2.SS2.p2.13.m5.1" class="ltx_Math" alttext="\tau&lt;1" display="inline"><semantics id="S2.SS2.p2.13.m5.1a"><mrow id="S2.SS2.p2.13.m5.1.1" xref="S2.SS2.p2.13.m5.1.1.cmml"><mi id="S2.SS2.p2.13.m5.1.1.2" xref="S2.SS2.p2.13.m5.1.1.2.cmml">τ</mi><mo id="S2.SS2.p2.13.m5.1.1.1" xref="S2.SS2.p2.13.m5.1.1.1.cmml">&lt;</mo><mn id="S2.SS2.p2.13.m5.1.1.3" xref="S2.SS2.p2.13.m5.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.13.m5.1b"><apply id="S2.SS2.p2.13.m5.1.1.cmml" xref="S2.SS2.p2.13.m5.1.1"><lt id="S2.SS2.p2.13.m5.1.1.1.cmml" xref="S2.SS2.p2.13.m5.1.1.1"></lt><ci id="S2.SS2.p2.13.m5.1.1.2.cmml" xref="S2.SS2.p2.13.m5.1.1.2">𝜏</ci><cn type="integer" id="S2.SS2.p2.13.m5.1.1.3.cmml" xref="S2.SS2.p2.13.m5.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.13.m5.1c">\tau&lt;1</annotation></semantics></math>) make the distribution sharper, favoring higher-probability tokens andmore conservative and deterministic outputs. Setting <math id="S2.SS2.p2.14.m6.1" class="ltx_Math" alttext="\tau=1" display="inline"><semantics id="S2.SS2.p2.14.m6.1a"><mrow id="S2.SS2.p2.14.m6.1.1" xref="S2.SS2.p2.14.m6.1.1.cmml"><mi id="S2.SS2.p2.14.m6.1.1.2" xref="S2.SS2.p2.14.m6.1.1.2.cmml">τ</mi><mo id="S2.SS2.p2.14.m6.1.1.1" xref="S2.SS2.p2.14.m6.1.1.1.cmml">=</mo><mn id="S2.SS2.p2.14.m6.1.1.3" xref="S2.SS2.p2.14.m6.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S2.SS2.p2.14.m6.1b"><apply id="S2.SS2.p2.14.m6.1.1.cmml" xref="S2.SS2.p2.14.m6.1.1"><eq id="S2.SS2.p2.14.m6.1.1.1.cmml" xref="S2.SS2.p2.14.m6.1.1.1"></eq><ci id="S2.SS2.p2.14.m6.1.1.2.cmml" xref="S2.SS2.p2.14.m6.1.1.2">𝜏</ci><cn type="integer" id="S2.SS2.p2.14.m6.1.1.3.cmml" xref="S2.SS2.p2.14.m6.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS2.p2.14.m6.1c">\tau=1</annotation></semantics></math> leaves the original probability distribution unchanged.</p>
</div>
</section>
<section id="S2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.3 </span>Top-<math id="S2.SS3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.1.m1.1b"><mi id="S2.SS3.1.m1.1.1" xref="S2.SS3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.1.m1.1c"><ci id="S2.SS3.1.m1.1.1.cmml" xref="S2.SS3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.1.m1.1d">p</annotation></semantics></math> Sampling</h3>

<div id="S2.SS3.p1" class="ltx_para">
<p id="S2.SS3.p1.2" class="ltx_p">Top-<math id="S2.SS3.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p1.1.m1.1a"><mi id="S2.SS3.p1.1.m1.1.1" xref="S2.SS3.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.1.m1.1b"><ci id="S2.SS3.p1.1.m1.1.1.cmml" xref="S2.SS3.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.1.m1.1c">p</annotation></semantics></math> sampling, also known as nucleus sampling, is a stochastic truncation decoding method that aims to address the limitations of greedy decoding <cite class="ltx_cite ltx_citemacro_citep">(Holtzman et al., <a href="#bib.bib8" title="" class="ltx_ref">2020</a>)</cite>. Instead of always selecting the most probable token like in greedy decoding, top-<math id="S2.SS3.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p1.2.m2.1a"><mi id="S2.SS3.p1.2.m2.1.1" xref="S2.SS3.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p1.2.m2.1b"><ci id="S2.SS3.p1.2.m2.1.1.cmml" xref="S2.SS3.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p1.2.m2.1c">p</annotation></semantics></math> sampling introduces stochasticity while still focusing on high-probability tokens.</p>
</div>
<div id="S2.SS3.p2" class="ltx_para">
<p id="S2.SS3.p2.7" class="ltx_p">Given <math id="S2.SS3.p2.1.m1.2" class="ltx_Math" alttext="P_{\tau}(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})" display="inline"><semantics id="S2.SS3.p2.1.m1.2a"><mrow id="S2.SS3.p2.1.m1.2.2" xref="S2.SS3.p2.1.m1.2.2.cmml"><msub id="S2.SS3.p2.1.m1.2.2.3" xref="S2.SS3.p2.1.m1.2.2.3.cmml"><mi id="S2.SS3.p2.1.m1.2.2.3.2" xref="S2.SS3.p2.1.m1.2.2.3.2.cmml">P</mi><mi id="S2.SS3.p2.1.m1.2.2.3.3" xref="S2.SS3.p2.1.m1.2.2.3.3.cmml">τ</mi></msub><mo lspace="0em" rspace="0em" id="S2.SS3.p2.1.m1.2.2.2" xref="S2.SS3.p2.1.m1.2.2.2.cmml">​</mo><mrow id="S2.SS3.p2.1.m1.2.2.1.1" xref="S2.SS3.p2.1.m1.2.2.1.1.1.cmml"><mo stretchy="false" id="S2.SS3.p2.1.m1.2.2.1.1.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.cmml">(</mo><mrow id="S2.SS3.p2.1.m1.2.2.1.1.1" xref="S2.SS3.p2.1.m1.2.2.1.1.1.cmml"><msub id="S2.SS3.p2.1.m1.2.2.1.1.1.5" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5.cmml"><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.5.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5.2.cmml">x</mi><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.5.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5.3.cmml">t</mi></msub><mo id="S2.SS3.p2.1.m1.2.2.1.1.1.4" xref="S2.SS3.p2.1.m1.2.2.1.1.1.4.cmml">=</mo><mrow id="S2.SS3.p2.1.m1.2.2.1.1.1.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.cmml"><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.3.5" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.5.cmml">v</mi><mo fence="false" id="S2.SS3.p2.1.m1.2.2.1.1.1.3.4" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.4.cmml">|</mo><mrow id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.4.cmml"><msub id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.cmml"><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.4" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.4.cmml">,</mo><msub id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.cmml"><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.2.cmml">x</mi><mn id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.3.cmml">2</mn></msub><mo id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.5" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.SS3.p2.1.m1.1.1" xref="S2.SS3.p2.1.m1.1.1.cmml">…</mi><mo id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.6" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.4.cmml">,</mo><msub id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.cmml"><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.2.cmml">x</mi><mrow id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.cmml"><mi id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.2" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.2.cmml">t</mi><mo id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.1" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.1.cmml">−</mo><mn id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.SS3.p2.1.m1.2.2.1.1.3" xref="S2.SS3.p2.1.m1.2.2.1.1.1.cmml">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.1.m1.2b"><apply id="S2.SS3.p2.1.m1.2.2.cmml" xref="S2.SS3.p2.1.m1.2.2"><times id="S2.SS3.p2.1.m1.2.2.2.cmml" xref="S2.SS3.p2.1.m1.2.2.2"></times><apply id="S2.SS3.p2.1.m1.2.2.3.cmml" xref="S2.SS3.p2.1.m1.2.2.3"><csymbol cd="ambiguous" id="S2.SS3.p2.1.m1.2.2.3.1.cmml" xref="S2.SS3.p2.1.m1.2.2.3">subscript</csymbol><ci id="S2.SS3.p2.1.m1.2.2.3.2.cmml" xref="S2.SS3.p2.1.m1.2.2.3.2">𝑃</ci><ci id="S2.SS3.p2.1.m1.2.2.3.3.cmml" xref="S2.SS3.p2.1.m1.2.2.3.3">𝜏</ci></apply><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1"><eq id="S2.SS3.p2.1.m1.2.2.1.1.1.4.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.4"></eq><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.5.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5"><csymbol cd="ambiguous" id="S2.SS3.p2.1.m1.2.2.1.1.1.5.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5">subscript</csymbol><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.5.2.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5.2">𝑥</ci><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.5.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.5.3">𝑡</ci></apply><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3"><csymbol cd="latexml" id="S2.SS3.p2.1.m1.2.2.1.1.1.3.4.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.4">conditional</csymbol><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.3.5.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.5">𝑣</ci><list id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.4.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3"><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.2.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.2">𝑥</ci><cn type="integer" id="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.2.2.2.2.3">2</cn></apply><ci id="S2.SS3.p2.1.m1.1.1.cmml" xref="S2.SS3.p2.1.m1.1.1">…</ci><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3"><csymbol cd="ambiguous" id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3">subscript</csymbol><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.2.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.2">𝑥</ci><apply id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3"><minus id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.1.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.1"></minus><ci id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.2.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.3.cmml" xref="S2.SS3.p2.1.m1.2.2.1.1.1.3.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.1.m1.2c">P_{\tau}(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})</annotation></semantics></math> over the vocabulary <math id="S2.SS3.p2.2.m2.1" class="ltx_Math" alttext="\mathcal{V}" display="inline"><semantics id="S2.SS3.p2.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S2.SS3.p2.2.m2.1.1" xref="S2.SS3.p2.2.m2.1.1.cmml">𝒱</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.2.m2.1b"><ci id="S2.SS3.p2.2.m2.1.1.cmml" xref="S2.SS3.p2.2.m2.1.1">𝒱</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.2.m2.1c">\mathcal{V}</annotation></semantics></math> at position <math id="S2.SS3.p2.3.m3.1" class="ltx_Math" alttext="t" display="inline"><semantics id="S2.SS3.p2.3.m3.1a"><mi id="S2.SS3.p2.3.m3.1.1" xref="S2.SS3.p2.3.m3.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.3.m3.1b"><ci id="S2.SS3.p2.3.m3.1.1.cmml" xref="S2.SS3.p2.3.m3.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.3.m3.1c">t</annotation></semantics></math>, top-<math id="S2.SS3.p2.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p2.4.m4.1a"><mi id="S2.SS3.p2.4.m4.1.1" xref="S2.SS3.p2.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.4.m4.1b"><ci id="S2.SS3.p2.4.m4.1.1.cmml" xref="S2.SS3.p2.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.4.m4.1c">p</annotation></semantics></math> sampling first sorts the tokens in descending order of their probabilities. It then selects the smallest set of tokens whose cumulative probability exceeds a predefined threshold <math id="S2.SS3.p2.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p2.5.m5.1a"><mi id="S2.SS3.p2.5.m5.1.1" xref="S2.SS3.p2.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.5.m5.1b"><ci id="S2.SS3.p2.5.m5.1.1.cmml" xref="S2.SS3.p2.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.5.m5.1c">p</annotation></semantics></math>, where <math id="S2.SS3.p2.6.m6.2" class="ltx_Math" alttext="p\in(0,1]" display="inline"><semantics id="S2.SS3.p2.6.m6.2a"><mrow id="S2.SS3.p2.6.m6.2.3" xref="S2.SS3.p2.6.m6.2.3.cmml"><mi id="S2.SS3.p2.6.m6.2.3.2" xref="S2.SS3.p2.6.m6.2.3.2.cmml">p</mi><mo id="S2.SS3.p2.6.m6.2.3.1" xref="S2.SS3.p2.6.m6.2.3.1.cmml">∈</mo><mrow id="S2.SS3.p2.6.m6.2.3.3.2" xref="S2.SS3.p2.6.m6.2.3.3.1.cmml"><mo stretchy="false" id="S2.SS3.p2.6.m6.2.3.3.2.1" xref="S2.SS3.p2.6.m6.2.3.3.1.cmml">(</mo><mn id="S2.SS3.p2.6.m6.1.1" xref="S2.SS3.p2.6.m6.1.1.cmml">0</mn><mo id="S2.SS3.p2.6.m6.2.3.3.2.2" xref="S2.SS3.p2.6.m6.2.3.3.1.cmml">,</mo><mn id="S2.SS3.p2.6.m6.2.2" xref="S2.SS3.p2.6.m6.2.2.cmml">1</mn><mo stretchy="false" id="S2.SS3.p2.6.m6.2.3.3.2.3" xref="S2.SS3.p2.6.m6.2.3.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.6.m6.2b"><apply id="S2.SS3.p2.6.m6.2.3.cmml" xref="S2.SS3.p2.6.m6.2.3"><in id="S2.SS3.p2.6.m6.2.3.1.cmml" xref="S2.SS3.p2.6.m6.2.3.1"></in><ci id="S2.SS3.p2.6.m6.2.3.2.cmml" xref="S2.SS3.p2.6.m6.2.3.2">𝑝</ci><interval closure="open-closed" id="S2.SS3.p2.6.m6.2.3.3.1.cmml" xref="S2.SS3.p2.6.m6.2.3.3.2"><cn type="integer" id="S2.SS3.p2.6.m6.1.1.cmml" xref="S2.SS3.p2.6.m6.1.1">0</cn><cn type="integer" id="S2.SS3.p2.6.m6.2.2.cmml" xref="S2.SS3.p2.6.m6.2.2">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.6.m6.2c">p\in(0,1]</annotation></semantics></math>. Formally, let <math id="S2.SS3.p2.7.m7.1" class="ltx_Math" alttext="\mathcal{V}p\subseteq\mathcal{V}" display="inline"><semantics id="S2.SS3.p2.7.m7.1a"><mrow id="S2.SS3.p2.7.m7.1.1" xref="S2.SS3.p2.7.m7.1.1.cmml"><mrow id="S2.SS3.p2.7.m7.1.1.2" xref="S2.SS3.p2.7.m7.1.1.2.cmml"><mi class="ltx_font_mathcaligraphic" id="S2.SS3.p2.7.m7.1.1.2.2" xref="S2.SS3.p2.7.m7.1.1.2.2.cmml">𝒱</mi><mo lspace="0em" rspace="0em" id="S2.SS3.p2.7.m7.1.1.2.1" xref="S2.SS3.p2.7.m7.1.1.2.1.cmml">​</mo><mi id="S2.SS3.p2.7.m7.1.1.2.3" xref="S2.SS3.p2.7.m7.1.1.2.3.cmml">p</mi></mrow><mo id="S2.SS3.p2.7.m7.1.1.1" xref="S2.SS3.p2.7.m7.1.1.1.cmml">⊆</mo><mi class="ltx_font_mathcaligraphic" id="S2.SS3.p2.7.m7.1.1.3" xref="S2.SS3.p2.7.m7.1.1.3.cmml">𝒱</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.7.m7.1b"><apply id="S2.SS3.p2.7.m7.1.1.cmml" xref="S2.SS3.p2.7.m7.1.1"><subset id="S2.SS3.p2.7.m7.1.1.1.cmml" xref="S2.SS3.p2.7.m7.1.1.1"></subset><apply id="S2.SS3.p2.7.m7.1.1.2.cmml" xref="S2.SS3.p2.7.m7.1.1.2"><times id="S2.SS3.p2.7.m7.1.1.2.1.cmml" xref="S2.SS3.p2.7.m7.1.1.2.1"></times><ci id="S2.SS3.p2.7.m7.1.1.2.2.cmml" xref="S2.SS3.p2.7.m7.1.1.2.2">𝒱</ci><ci id="S2.SS3.p2.7.m7.1.1.2.3.cmml" xref="S2.SS3.p2.7.m7.1.1.2.3">𝑝</ci></apply><ci id="S2.SS3.p2.7.m7.1.1.3.cmml" xref="S2.SS3.p2.7.m7.1.1.3">𝒱</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.7.m7.1c">\mathcal{V}p\subseteq\mathcal{V}</annotation></semantics></math> be the smallest set such that:</p>
<table id="S2.E5" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.E5.m1.2" class="ltx_Math" alttext="\sum_{v\in\mathcal{V}p}P_{\tau}(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})\geq p" display="block"><semantics id="S2.E5.m1.2a"><mrow id="S2.E5.m1.2.2" xref="S2.E5.m1.2.2.cmml"><mrow id="S2.E5.m1.2.2.1" xref="S2.E5.m1.2.2.1.cmml"><munder id="S2.E5.m1.2.2.1.2" xref="S2.E5.m1.2.2.1.2.cmml"><mo movablelimits="false" id="S2.E5.m1.2.2.1.2.2" xref="S2.E5.m1.2.2.1.2.2.cmml">∑</mo><mrow id="S2.E5.m1.2.2.1.2.3" xref="S2.E5.m1.2.2.1.2.3.cmml"><mi id="S2.E5.m1.2.2.1.2.3.2" xref="S2.E5.m1.2.2.1.2.3.2.cmml">v</mi><mo id="S2.E5.m1.2.2.1.2.3.1" xref="S2.E5.m1.2.2.1.2.3.1.cmml">∈</mo><mrow id="S2.E5.m1.2.2.1.2.3.3" xref="S2.E5.m1.2.2.1.2.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S2.E5.m1.2.2.1.2.3.3.2" xref="S2.E5.m1.2.2.1.2.3.3.2.cmml">𝒱</mi><mo lspace="0em" rspace="0em" id="S2.E5.m1.2.2.1.2.3.3.1" xref="S2.E5.m1.2.2.1.2.3.3.1.cmml">​</mo><mi id="S2.E5.m1.2.2.1.2.3.3.3" xref="S2.E5.m1.2.2.1.2.3.3.3.cmml">p</mi></mrow></mrow></munder><mrow id="S2.E5.m1.2.2.1.1" xref="S2.E5.m1.2.2.1.1.cmml"><msub id="S2.E5.m1.2.2.1.1.3" xref="S2.E5.m1.2.2.1.1.3.cmml"><mi id="S2.E5.m1.2.2.1.1.3.2" xref="S2.E5.m1.2.2.1.1.3.2.cmml">P</mi><mi id="S2.E5.m1.2.2.1.1.3.3" xref="S2.E5.m1.2.2.1.1.3.3.cmml">τ</mi></msub><mo lspace="0em" rspace="0em" id="S2.E5.m1.2.2.1.1.2" xref="S2.E5.m1.2.2.1.1.2.cmml">​</mo><mrow id="S2.E5.m1.2.2.1.1.1.1" xref="S2.E5.m1.2.2.1.1.1.1.1.cmml"><mo stretchy="false" id="S2.E5.m1.2.2.1.1.1.1.2" xref="S2.E5.m1.2.2.1.1.1.1.1.cmml">(</mo><mrow id="S2.E5.m1.2.2.1.1.1.1.1" xref="S2.E5.m1.2.2.1.1.1.1.1.cmml"><msub id="S2.E5.m1.2.2.1.1.1.1.1.5" xref="S2.E5.m1.2.2.1.1.1.1.1.5.cmml"><mi id="S2.E5.m1.2.2.1.1.1.1.1.5.2" xref="S2.E5.m1.2.2.1.1.1.1.1.5.2.cmml">x</mi><mi id="S2.E5.m1.2.2.1.1.1.1.1.5.3" xref="S2.E5.m1.2.2.1.1.1.1.1.5.3.cmml">t</mi></msub><mo id="S2.E5.m1.2.2.1.1.1.1.1.4" xref="S2.E5.m1.2.2.1.1.1.1.1.4.cmml">=</mo><mrow id="S2.E5.m1.2.2.1.1.1.1.1.3" xref="S2.E5.m1.2.2.1.1.1.1.1.3.cmml"><mi id="S2.E5.m1.2.2.1.1.1.1.1.3.5" xref="S2.E5.m1.2.2.1.1.1.1.1.3.5.cmml">v</mi><mo fence="false" id="S2.E5.m1.2.2.1.1.1.1.1.3.4" xref="S2.E5.m1.2.2.1.1.1.1.1.3.4.cmml">|</mo><mrow id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.4.cmml"><msub id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.cmml"><mi id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.2" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.3" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.4" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.cmml"><mi id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.2" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.2.cmml">x</mi><mn id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.3" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.3.cmml">2</mn></msub><mo id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.5" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.4.cmml">,</mo><mi mathvariant="normal" id="S2.E5.m1.1.1" xref="S2.E5.m1.1.1.cmml">…</mi><mo id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.6" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.4.cmml">,</mo><msub id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.cmml"><mi id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.2" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.2.cmml">x</mi><mrow id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.cmml"><mi id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.2" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.2.cmml">t</mi><mo id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.1" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.1.cmml">−</mo><mn id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.3" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.E5.m1.2.2.1.1.1.1.3" xref="S2.E5.m1.2.2.1.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><mo id="S2.E5.m1.2.2.2" xref="S2.E5.m1.2.2.2.cmml">≥</mo><mi id="S2.E5.m1.2.2.3" xref="S2.E5.m1.2.2.3.cmml">p</mi></mrow><annotation-xml encoding="MathML-Content" id="S2.E5.m1.2b"><apply id="S2.E5.m1.2.2.cmml" xref="S2.E5.m1.2.2"><geq id="S2.E5.m1.2.2.2.cmml" xref="S2.E5.m1.2.2.2"></geq><apply id="S2.E5.m1.2.2.1.cmml" xref="S2.E5.m1.2.2.1"><apply id="S2.E5.m1.2.2.1.2.cmml" xref="S2.E5.m1.2.2.1.2"><csymbol cd="ambiguous" id="S2.E5.m1.2.2.1.2.1.cmml" xref="S2.E5.m1.2.2.1.2">subscript</csymbol><sum id="S2.E5.m1.2.2.1.2.2.cmml" xref="S2.E5.m1.2.2.1.2.2"></sum><apply id="S2.E5.m1.2.2.1.2.3.cmml" xref="S2.E5.m1.2.2.1.2.3"><in id="S2.E5.m1.2.2.1.2.3.1.cmml" xref="S2.E5.m1.2.2.1.2.3.1"></in><ci id="S2.E5.m1.2.2.1.2.3.2.cmml" xref="S2.E5.m1.2.2.1.2.3.2">𝑣</ci><apply id="S2.E5.m1.2.2.1.2.3.3.cmml" xref="S2.E5.m1.2.2.1.2.3.3"><times id="S2.E5.m1.2.2.1.2.3.3.1.cmml" xref="S2.E5.m1.2.2.1.2.3.3.1"></times><ci id="S2.E5.m1.2.2.1.2.3.3.2.cmml" xref="S2.E5.m1.2.2.1.2.3.3.2">𝒱</ci><ci id="S2.E5.m1.2.2.1.2.3.3.3.cmml" xref="S2.E5.m1.2.2.1.2.3.3.3">𝑝</ci></apply></apply></apply><apply id="S2.E5.m1.2.2.1.1.cmml" xref="S2.E5.m1.2.2.1.1"><times id="S2.E5.m1.2.2.1.1.2.cmml" xref="S2.E5.m1.2.2.1.1.2"></times><apply id="S2.E5.m1.2.2.1.1.3.cmml" xref="S2.E5.m1.2.2.1.1.3"><csymbol cd="ambiguous" id="S2.E5.m1.2.2.1.1.3.1.cmml" xref="S2.E5.m1.2.2.1.1.3">subscript</csymbol><ci id="S2.E5.m1.2.2.1.1.3.2.cmml" xref="S2.E5.m1.2.2.1.1.3.2">𝑃</ci><ci id="S2.E5.m1.2.2.1.1.3.3.cmml" xref="S2.E5.m1.2.2.1.1.3.3">𝜏</ci></apply><apply id="S2.E5.m1.2.2.1.1.1.1.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1"><eq id="S2.E5.m1.2.2.1.1.1.1.1.4.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.4"></eq><apply id="S2.E5.m1.2.2.1.1.1.1.1.5.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.5"><csymbol cd="ambiguous" id="S2.E5.m1.2.2.1.1.1.1.1.5.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.5">subscript</csymbol><ci id="S2.E5.m1.2.2.1.1.1.1.1.5.2.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.5.2">𝑥</ci><ci id="S2.E5.m1.2.2.1.1.1.1.1.5.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.5.3">𝑡</ci></apply><apply id="S2.E5.m1.2.2.1.1.1.1.1.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3"><csymbol cd="latexml" id="S2.E5.m1.2.2.1.1.1.1.1.3.4.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.4">conditional</csymbol><ci id="S2.E5.m1.2.2.1.1.1.1.1.3.5.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.5">𝑣</ci><list id="S2.E5.m1.2.2.1.1.1.1.1.3.3.4.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3"><apply id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.2.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.2.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.2">𝑥</ci><cn type="integer" id="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.2.2.2.2.3">2</cn></apply><ci id="S2.E5.m1.1.1.cmml" xref="S2.E5.m1.1.1">…</ci><apply id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3"><csymbol cd="ambiguous" id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3">subscript</csymbol><ci id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.2.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.2">𝑥</ci><apply id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3"><minus id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.1.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.1"></minus><ci id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.2.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.2">𝑡</ci><cn type="integer" id="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.3.cmml" xref="S2.E5.m1.2.2.1.1.1.1.1.3.3.3.3.3.3">1</cn></apply></apply></list></apply></apply></apply></apply><ci id="S2.E5.m1.2.2.3.cmml" xref="S2.E5.m1.2.2.3">𝑝</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.E5.m1.2c">\sum_{v\in\mathcal{V}p}P_{\tau}(x_{t}=v|x_{1},x_{2},\dots,x_{t-1})\geq p</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(5)</span></td>
</tr></tbody>
</table>
<p id="S2.SS3.p2.9" class="ltx_p">The next token <math id="S2.SS3.p2.8.m1.1" class="ltx_Math" alttext="\hat{x}_{t}" display="inline"><semantics id="S2.SS3.p2.8.m1.1a"><msub id="S2.SS3.p2.8.m1.1.1" xref="S2.SS3.p2.8.m1.1.1.cmml"><mover accent="true" id="S2.SS3.p2.8.m1.1.1.2" xref="S2.SS3.p2.8.m1.1.1.2.cmml"><mi id="S2.SS3.p2.8.m1.1.1.2.2" xref="S2.SS3.p2.8.m1.1.1.2.2.cmml">x</mi><mo id="S2.SS3.p2.8.m1.1.1.2.1" xref="S2.SS3.p2.8.m1.1.1.2.1.cmml">^</mo></mover><mi id="S2.SS3.p2.8.m1.1.1.3" xref="S2.SS3.p2.8.m1.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.8.m1.1b"><apply id="S2.SS3.p2.8.m1.1.1.cmml" xref="S2.SS3.p2.8.m1.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.8.m1.1.1.1.cmml" xref="S2.SS3.p2.8.m1.1.1">subscript</csymbol><apply id="S2.SS3.p2.8.m1.1.1.2.cmml" xref="S2.SS3.p2.8.m1.1.1.2"><ci id="S2.SS3.p2.8.m1.1.1.2.1.cmml" xref="S2.SS3.p2.8.m1.1.1.2.1">^</ci><ci id="S2.SS3.p2.8.m1.1.1.2.2.cmml" xref="S2.SS3.p2.8.m1.1.1.2.2">𝑥</ci></apply><ci id="S2.SS3.p2.8.m1.1.1.3.cmml" xref="S2.SS3.p2.8.m1.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.8.m1.1c">\hat{x}_{t}</annotation></semantics></math> is then randomly sampled from this reduced set <math id="S2.SS3.p2.9.m2.1" class="ltx_Math" alttext="\mathcal{V}_{p}" display="inline"><semantics id="S2.SS3.p2.9.m2.1a"><msub id="S2.SS3.p2.9.m2.1.1" xref="S2.SS3.p2.9.m2.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="S2.SS3.p2.9.m2.1.1.2" xref="S2.SS3.p2.9.m2.1.1.2.cmml">𝒱</mi><mi id="S2.SS3.p2.9.m2.1.1.3" xref="S2.SS3.p2.9.m2.1.1.3.cmml">p</mi></msub><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.9.m2.1b"><apply id="S2.SS3.p2.9.m2.1.1.cmml" xref="S2.SS3.p2.9.m2.1.1"><csymbol cd="ambiguous" id="S2.SS3.p2.9.m2.1.1.1.cmml" xref="S2.SS3.p2.9.m2.1.1">subscript</csymbol><ci id="S2.SS3.p2.9.m2.1.1.2.cmml" xref="S2.SS3.p2.9.m2.1.1.2">𝒱</ci><ci id="S2.SS3.p2.9.m2.1.1.3.cmml" xref="S2.SS3.p2.9.m2.1.1.3">𝑝</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.9.m2.1c">\mathcal{V}_{p}</annotation></semantics></math> according to the renormalized probabilities:</p>
<table id="S2.Ex1" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S2.Ex1.m1.4" class="ltx_Math" alttext="\hat{x}_{t}\sim\frac{P_{\tau}(x_{t}=v|x_{1},\dots,x_{t-1})}{\sum_{v^{\prime}\in\mathcal{V}_{p}}P_{\tau}(x_{t}=v^{\prime}|x_{1},\dots,x_{t-1})}\text{ for }v\in\mathcal{V}_{p}" display="block"><semantics id="S2.Ex1.m1.4a"><mrow id="S2.Ex1.m1.4.5" xref="S2.Ex1.m1.4.5.cmml"><msub id="S2.Ex1.m1.4.5.2" xref="S2.Ex1.m1.4.5.2.cmml"><mover accent="true" id="S2.Ex1.m1.4.5.2.2" xref="S2.Ex1.m1.4.5.2.2.cmml"><mi id="S2.Ex1.m1.4.5.2.2.2" xref="S2.Ex1.m1.4.5.2.2.2.cmml">x</mi><mo id="S2.Ex1.m1.4.5.2.2.1" xref="S2.Ex1.m1.4.5.2.2.1.cmml">^</mo></mover><mi id="S2.Ex1.m1.4.5.2.3" xref="S2.Ex1.m1.4.5.2.3.cmml">t</mi></msub><mo id="S2.Ex1.m1.4.5.3" xref="S2.Ex1.m1.4.5.3.cmml">∼</mo><mrow id="S2.Ex1.m1.4.5.4" xref="S2.Ex1.m1.4.5.4.cmml"><mfrac id="S2.Ex1.m1.4.4" xref="S2.Ex1.m1.4.4.cmml"><mrow id="S2.Ex1.m1.2.2.2" xref="S2.Ex1.m1.2.2.2.cmml"><msub id="S2.Ex1.m1.2.2.2.4" xref="S2.Ex1.m1.2.2.2.4.cmml"><mi id="S2.Ex1.m1.2.2.2.4.2" xref="S2.Ex1.m1.2.2.2.4.2.cmml">P</mi><mi id="S2.Ex1.m1.2.2.2.4.3" xref="S2.Ex1.m1.2.2.2.4.3.cmml">τ</mi></msub><mo lspace="0em" rspace="0em" id="S2.Ex1.m1.2.2.2.3" xref="S2.Ex1.m1.2.2.2.3.cmml">​</mo><mrow id="S2.Ex1.m1.2.2.2.2.1" xref="S2.Ex1.m1.2.2.2.2.1.1.cmml"><mo stretchy="false" id="S2.Ex1.m1.2.2.2.2.1.2" xref="S2.Ex1.m1.2.2.2.2.1.1.cmml">(</mo><mrow id="S2.Ex1.m1.2.2.2.2.1.1" xref="S2.Ex1.m1.2.2.2.2.1.1.cmml"><msub id="S2.Ex1.m1.2.2.2.2.1.1.4" xref="S2.Ex1.m1.2.2.2.2.1.1.4.cmml"><mi id="S2.Ex1.m1.2.2.2.2.1.1.4.2" xref="S2.Ex1.m1.2.2.2.2.1.1.4.2.cmml">x</mi><mi id="S2.Ex1.m1.2.2.2.2.1.1.4.3" xref="S2.Ex1.m1.2.2.2.2.1.1.4.3.cmml">t</mi></msub><mo id="S2.Ex1.m1.2.2.2.2.1.1.3" xref="S2.Ex1.m1.2.2.2.2.1.1.3.cmml">=</mo><mrow id="S2.Ex1.m1.2.2.2.2.1.1.2" xref="S2.Ex1.m1.2.2.2.2.1.1.2.cmml"><mi id="S2.Ex1.m1.2.2.2.2.1.1.2.4" xref="S2.Ex1.m1.2.2.2.2.1.1.2.4.cmml">v</mi><mo fence="false" id="S2.Ex1.m1.2.2.2.2.1.1.2.3" xref="S2.Ex1.m1.2.2.2.2.1.1.2.3.cmml">|</mo><mrow id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.3.cmml"><msub id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.cmml"><mi id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.2" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.3" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.3" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.Ex1.m1.1.1.1.1" xref="S2.Ex1.m1.1.1.1.1.cmml">…</mi><mo id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.4" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.3.cmml">,</mo><msub id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.cmml"><mi id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.2" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.2.cmml">x</mi><mrow id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.cmml"><mi id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.2" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.2.cmml">t</mi><mo id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.1" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.1.cmml">−</mo><mn id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.3" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.Ex1.m1.2.2.2.2.1.3" xref="S2.Ex1.m1.2.2.2.2.1.1.cmml">)</mo></mrow></mrow><mrow id="S2.Ex1.m1.4.4.4" xref="S2.Ex1.m1.4.4.4.cmml"><msub id="S2.Ex1.m1.4.4.4.3" xref="S2.Ex1.m1.4.4.4.3.cmml"><mo id="S2.Ex1.m1.4.4.4.3.2" xref="S2.Ex1.m1.4.4.4.3.2.cmml">∑</mo><mrow id="S2.Ex1.m1.4.4.4.3.3" xref="S2.Ex1.m1.4.4.4.3.3.cmml"><msup id="S2.Ex1.m1.4.4.4.3.3.2" xref="S2.Ex1.m1.4.4.4.3.3.2.cmml"><mi id="S2.Ex1.m1.4.4.4.3.3.2.2" xref="S2.Ex1.m1.4.4.4.3.3.2.2.cmml">v</mi><mo id="S2.Ex1.m1.4.4.4.3.3.2.3" xref="S2.Ex1.m1.4.4.4.3.3.2.3.cmml">′</mo></msup><mo id="S2.Ex1.m1.4.4.4.3.3.1" xref="S2.Ex1.m1.4.4.4.3.3.1.cmml">∈</mo><msub id="S2.Ex1.m1.4.4.4.3.3.3" xref="S2.Ex1.m1.4.4.4.3.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S2.Ex1.m1.4.4.4.3.3.3.2" xref="S2.Ex1.m1.4.4.4.3.3.3.2.cmml">𝒱</mi><mi id="S2.Ex1.m1.4.4.4.3.3.3.3" xref="S2.Ex1.m1.4.4.4.3.3.3.3.cmml">p</mi></msub></mrow></msub><mrow id="S2.Ex1.m1.4.4.4.2" xref="S2.Ex1.m1.4.4.4.2.cmml"><msub id="S2.Ex1.m1.4.4.4.2.3" xref="S2.Ex1.m1.4.4.4.2.3.cmml"><mi id="S2.Ex1.m1.4.4.4.2.3.2" xref="S2.Ex1.m1.4.4.4.2.3.2.cmml">P</mi><mi id="S2.Ex1.m1.4.4.4.2.3.3" xref="S2.Ex1.m1.4.4.4.2.3.3.cmml">τ</mi></msub><mo lspace="0em" rspace="0em" id="S2.Ex1.m1.4.4.4.2.2" xref="S2.Ex1.m1.4.4.4.2.2.cmml">​</mo><mrow id="S2.Ex1.m1.4.4.4.2.1.1" xref="S2.Ex1.m1.4.4.4.2.1.1.1.cmml"><mo stretchy="false" id="S2.Ex1.m1.4.4.4.2.1.1.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.cmml">(</mo><mrow id="S2.Ex1.m1.4.4.4.2.1.1.1" xref="S2.Ex1.m1.4.4.4.2.1.1.1.cmml"><msub id="S2.Ex1.m1.4.4.4.2.1.1.1.4" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4.cmml"><mi id="S2.Ex1.m1.4.4.4.2.1.1.1.4.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4.2.cmml">x</mi><mi id="S2.Ex1.m1.4.4.4.2.1.1.1.4.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4.3.cmml">t</mi></msub><mo id="S2.Ex1.m1.4.4.4.2.1.1.1.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.3.cmml">=</mo><mrow id="S2.Ex1.m1.4.4.4.2.1.1.1.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.cmml"><msup id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.cmml"><mi id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.2.cmml">v</mi><mo id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.3.cmml">′</mo></msup><mo fence="false" id="S2.Ex1.m1.4.4.4.2.1.1.1.2.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.3.cmml">|</mo><mrow id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.3.cmml"><msub id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.cmml"><mi id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.3.cmml">,</mo><mi mathvariant="normal" id="S2.Ex1.m1.3.3.3.1" xref="S2.Ex1.m1.3.3.3.1.cmml">…</mi><mo id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.4" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.3.cmml">,</mo><msub id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.cmml"><mi id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.2.cmml">x</mi><mrow id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.cmml"><mi id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.2" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.2.cmml">t</mi><mo id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.1" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.1.cmml">−</mo><mn id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow></mrow><mo stretchy="false" id="S2.Ex1.m1.4.4.4.2.1.1.3" xref="S2.Ex1.m1.4.4.4.2.1.1.1.cmml">)</mo></mrow></mrow></mrow></mfrac><mo lspace="0em" rspace="0em" id="S2.Ex1.m1.4.5.4.1" xref="S2.Ex1.m1.4.5.4.1.cmml">​</mo><mtext id="S2.Ex1.m1.4.5.4.2" xref="S2.Ex1.m1.4.5.4.2a.cmml"> for </mtext><mo lspace="0em" rspace="0em" id="S2.Ex1.m1.4.5.4.1a" xref="S2.Ex1.m1.4.5.4.1.cmml">​</mo><mi id="S2.Ex1.m1.4.5.4.3" xref="S2.Ex1.m1.4.5.4.3.cmml">v</mi></mrow><mo id="S2.Ex1.m1.4.5.5" xref="S2.Ex1.m1.4.5.5.cmml">∈</mo><msub id="S2.Ex1.m1.4.5.6" xref="S2.Ex1.m1.4.5.6.cmml"><mi class="ltx_font_mathcaligraphic" id="S2.Ex1.m1.4.5.6.2" xref="S2.Ex1.m1.4.5.6.2.cmml">𝒱</mi><mi id="S2.Ex1.m1.4.5.6.3" xref="S2.Ex1.m1.4.5.6.3.cmml">p</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S2.Ex1.m1.4b"><apply id="S2.Ex1.m1.4.5.cmml" xref="S2.Ex1.m1.4.5"><and id="S2.Ex1.m1.4.5a.cmml" xref="S2.Ex1.m1.4.5"></and><apply id="S2.Ex1.m1.4.5b.cmml" xref="S2.Ex1.m1.4.5"><csymbol cd="latexml" id="S2.Ex1.m1.4.5.3.cmml" xref="S2.Ex1.m1.4.5.3">similar-to</csymbol><apply id="S2.Ex1.m1.4.5.2.cmml" xref="S2.Ex1.m1.4.5.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.5.2.1.cmml" xref="S2.Ex1.m1.4.5.2">subscript</csymbol><apply id="S2.Ex1.m1.4.5.2.2.cmml" xref="S2.Ex1.m1.4.5.2.2"><ci id="S2.Ex1.m1.4.5.2.2.1.cmml" xref="S2.Ex1.m1.4.5.2.2.1">^</ci><ci id="S2.Ex1.m1.4.5.2.2.2.cmml" xref="S2.Ex1.m1.4.5.2.2.2">𝑥</ci></apply><ci id="S2.Ex1.m1.4.5.2.3.cmml" xref="S2.Ex1.m1.4.5.2.3">𝑡</ci></apply><apply id="S2.Ex1.m1.4.5.4.cmml" xref="S2.Ex1.m1.4.5.4"><times id="S2.Ex1.m1.4.5.4.1.cmml" xref="S2.Ex1.m1.4.5.4.1"></times><apply id="S2.Ex1.m1.4.4.cmml" xref="S2.Ex1.m1.4.4"><divide id="S2.Ex1.m1.4.4.5.cmml" xref="S2.Ex1.m1.4.4"></divide><apply id="S2.Ex1.m1.2.2.2.cmml" xref="S2.Ex1.m1.2.2.2"><times id="S2.Ex1.m1.2.2.2.3.cmml" xref="S2.Ex1.m1.2.2.2.3"></times><apply id="S2.Ex1.m1.2.2.2.4.cmml" xref="S2.Ex1.m1.2.2.2.4"><csymbol cd="ambiguous" id="S2.Ex1.m1.2.2.2.4.1.cmml" xref="S2.Ex1.m1.2.2.2.4">subscript</csymbol><ci id="S2.Ex1.m1.2.2.2.4.2.cmml" xref="S2.Ex1.m1.2.2.2.4.2">𝑃</ci><ci id="S2.Ex1.m1.2.2.2.4.3.cmml" xref="S2.Ex1.m1.2.2.2.4.3">𝜏</ci></apply><apply id="S2.Ex1.m1.2.2.2.2.1.1.cmml" xref="S2.Ex1.m1.2.2.2.2.1"><eq id="S2.Ex1.m1.2.2.2.2.1.1.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.3"></eq><apply id="S2.Ex1.m1.2.2.2.2.1.1.4.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.4"><csymbol cd="ambiguous" id="S2.Ex1.m1.2.2.2.2.1.1.4.1.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.4">subscript</csymbol><ci id="S2.Ex1.m1.2.2.2.2.1.1.4.2.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.4.2">𝑥</ci><ci id="S2.Ex1.m1.2.2.2.2.1.1.4.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.4.3">𝑡</ci></apply><apply id="S2.Ex1.m1.2.2.2.2.1.1.2.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2"><csymbol cd="latexml" id="S2.Ex1.m1.2.2.2.2.1.1.2.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.3">conditional</csymbol><ci id="S2.Ex1.m1.2.2.2.2.1.1.2.4.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.4">𝑣</ci><list id="S2.Ex1.m1.2.2.2.2.1.1.2.2.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2"><apply id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.1.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1">subscript</csymbol><ci id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.2.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.1.1.1.1.3">1</cn></apply><ci id="S2.Ex1.m1.1.1.1.1.cmml" xref="S2.Ex1.m1.1.1.1.1">…</ci><apply id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.1.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2">subscript</csymbol><ci id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.2.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.2">𝑥</ci><apply id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3"><minus id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.1.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.1"></minus><ci id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.2.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.2">𝑡</ci><cn type="integer" id="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.3.cmml" xref="S2.Ex1.m1.2.2.2.2.1.1.2.2.2.2.3.3">1</cn></apply></apply></list></apply></apply></apply><apply id="S2.Ex1.m1.4.4.4.cmml" xref="S2.Ex1.m1.4.4.4"><apply id="S2.Ex1.m1.4.4.4.3.cmml" xref="S2.Ex1.m1.4.4.4.3"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.3.1.cmml" xref="S2.Ex1.m1.4.4.4.3">subscript</csymbol><sum id="S2.Ex1.m1.4.4.4.3.2.cmml" xref="S2.Ex1.m1.4.4.4.3.2"></sum><apply id="S2.Ex1.m1.4.4.4.3.3.cmml" xref="S2.Ex1.m1.4.4.4.3.3"><in id="S2.Ex1.m1.4.4.4.3.3.1.cmml" xref="S2.Ex1.m1.4.4.4.3.3.1"></in><apply id="S2.Ex1.m1.4.4.4.3.3.2.cmml" xref="S2.Ex1.m1.4.4.4.3.3.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.3.3.2.1.cmml" xref="S2.Ex1.m1.4.4.4.3.3.2">superscript</csymbol><ci id="S2.Ex1.m1.4.4.4.3.3.2.2.cmml" xref="S2.Ex1.m1.4.4.4.3.3.2.2">𝑣</ci><ci id="S2.Ex1.m1.4.4.4.3.3.2.3.cmml" xref="S2.Ex1.m1.4.4.4.3.3.2.3">′</ci></apply><apply id="S2.Ex1.m1.4.4.4.3.3.3.cmml" xref="S2.Ex1.m1.4.4.4.3.3.3"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.3.3.3.1.cmml" xref="S2.Ex1.m1.4.4.4.3.3.3">subscript</csymbol><ci id="S2.Ex1.m1.4.4.4.3.3.3.2.cmml" xref="S2.Ex1.m1.4.4.4.3.3.3.2">𝒱</ci><ci id="S2.Ex1.m1.4.4.4.3.3.3.3.cmml" xref="S2.Ex1.m1.4.4.4.3.3.3.3">𝑝</ci></apply></apply></apply><apply id="S2.Ex1.m1.4.4.4.2.cmml" xref="S2.Ex1.m1.4.4.4.2"><times id="S2.Ex1.m1.4.4.4.2.2.cmml" xref="S2.Ex1.m1.4.4.4.2.2"></times><apply id="S2.Ex1.m1.4.4.4.2.3.cmml" xref="S2.Ex1.m1.4.4.4.2.3"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.2.3.1.cmml" xref="S2.Ex1.m1.4.4.4.2.3">subscript</csymbol><ci id="S2.Ex1.m1.4.4.4.2.3.2.cmml" xref="S2.Ex1.m1.4.4.4.2.3.2">𝑃</ci><ci id="S2.Ex1.m1.4.4.4.2.3.3.cmml" xref="S2.Ex1.m1.4.4.4.2.3.3">𝜏</ci></apply><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1"><eq id="S2.Ex1.m1.4.4.4.2.1.1.1.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.3"></eq><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.4.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.2.1.1.1.4.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4">subscript</csymbol><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.4.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4.2">𝑥</ci><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.4.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.4.3">𝑡</ci></apply><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2"><csymbol cd="latexml" id="S2.Ex1.m1.4.4.4.2.1.1.1.2.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.3">conditional</csymbol><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4">superscript</csymbol><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.2">𝑣</ci><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.4.3">′</ci></apply><list id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2"><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1">subscript</csymbol><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.1.1.1.1.3">1</cn></apply><ci id="S2.Ex1.m1.3.3.3.1.cmml" xref="S2.Ex1.m1.3.3.3.1">…</ci><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2">subscript</csymbol><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.2">𝑥</ci><apply id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3"><minus id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.1.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.1"></minus><ci id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.2.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.2">𝑡</ci><cn type="integer" id="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.3.cmml" xref="S2.Ex1.m1.4.4.4.2.1.1.1.2.2.2.2.3.3">1</cn></apply></apply></list></apply></apply></apply></apply></apply><ci id="S2.Ex1.m1.4.5.4.2a.cmml" xref="S2.Ex1.m1.4.5.4.2"><mtext id="S2.Ex1.m1.4.5.4.2.cmml" xref="S2.Ex1.m1.4.5.4.2"> for </mtext></ci><ci id="S2.Ex1.m1.4.5.4.3.cmml" xref="S2.Ex1.m1.4.5.4.3">𝑣</ci></apply></apply><apply id="S2.Ex1.m1.4.5c.cmml" xref="S2.Ex1.m1.4.5"><in id="S2.Ex1.m1.4.5.5.cmml" xref="S2.Ex1.m1.4.5.5"></in><share href="#S2.Ex1.m1.4.5.4.cmml" id="S2.Ex1.m1.4.5d.cmml" xref="S2.Ex1.m1.4.5"></share><apply id="S2.Ex1.m1.4.5.6.cmml" xref="S2.Ex1.m1.4.5.6"><csymbol cd="ambiguous" id="S2.Ex1.m1.4.5.6.1.cmml" xref="S2.Ex1.m1.4.5.6">subscript</csymbol><ci id="S2.Ex1.m1.4.5.6.2.cmml" xref="S2.Ex1.m1.4.5.6.2">𝒱</ci><ci id="S2.Ex1.m1.4.5.6.3.cmml" xref="S2.Ex1.m1.4.5.6.3">𝑝</ci></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S2.Ex1.m1.4c">\hat{x}_{t}\sim\frac{P_{\tau}(x_{t}=v|x_{1},\dots,x_{t-1})}{\sum_{v^{\prime}\in\mathcal{V}_{p}}P_{\tau}(x_{t}=v^{\prime}|x_{1},\dots,x_{t-1})}\text{ for }v\in\mathcal{V}_{p}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr></tbody>
</table>
<p id="S2.SS3.p2.10" class="ltx_p">By restricting the sampling to the nucleus of high-probability tokens, top-<math id="S2.SS3.p2.10.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p2.10.m1.1a"><mi id="S2.SS3.p2.10.m1.1.1" xref="S2.SS3.p2.10.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p2.10.m1.1b"><ci id="S2.SS3.p2.10.m1.1.1.cmml" xref="S2.SS3.p2.10.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p2.10.m1.1c">p</annotation></semantics></math> sampling allows for more diverse and coherent generated text compared to greedy decoding.</p>
</div>
<div id="S2.SS3.p3" class="ltx_para">
<p id="S2.SS3.p3.4" class="ltx_p">However, combining truncation methods like top-<math id="S2.SS3.p3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p3.1.m1.1a"><mi id="S2.SS3.p3.1.m1.1.1" xref="S2.SS3.p3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p3.1.m1.1b"><ci id="S2.SS3.p3.1.m1.1.1.cmml" xref="S2.SS3.p3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p3.1.m1.1c">p</annotation></semantics></math> sampling with temperature scaling can lead to suboptimal results. When using top-<math id="S2.SS3.p3.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p3.2.m2.1a"><mi id="S2.SS3.p3.2.m2.1.1" xref="S2.SS3.p3.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p3.2.m2.1b"><ci id="S2.SS3.p3.2.m2.1.1.cmml" xref="S2.SS3.p3.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p3.2.m2.1c">p</annotation></semantics></math> sampling, it is recommended to adjust either the value of <math id="S2.SS3.p3.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S2.SS3.p3.3.m3.1a"><mi id="S2.SS3.p3.3.m3.1.1" xref="S2.SS3.p3.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p3.3.m3.1b"><ci id="S2.SS3.p3.3.m3.1.1.cmml" xref="S2.SS3.p3.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p3.3.m3.1c">p</annotation></semantics></math> or the temperature <math id="S2.SS3.p3.4.m4.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="S2.SS3.p3.4.m4.1a"><mi id="S2.SS3.p3.4.m4.1.1" xref="S2.SS3.p3.4.m4.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="S2.SS3.p3.4.m4.1b"><ci id="S2.SS3.p3.4.m4.1.1.cmml" xref="S2.SS3.p3.4.m4.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="S2.SS3.p3.4.m4.1c">\tau</annotation></semantics></math>, but not both simultaneously <cite class="ltx_cite ltx_citemacro_citep">(OpenAI API, <a href="#bib.bib13" title="" class="ltx_ref">2024</a>)</cite> as they can lead to conflicting effects.</p>
</div>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Min-<math id="S3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S3.1.m1.1b"><mi id="S3.1.m1.1.1" xref="S3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S3.1.m1.1c"><ci id="S3.1.m1.1.1.cmml" xref="S3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.1.m1.1d">p</annotation></semantics></math> Sampling</h2>

<div id="S3.p1" class="ltx_para">
<p id="S3.p1.5" class="ltx_p">We introduce min-<math id="S3.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S3.p1.1.m1.1a"><mi id="S3.p1.1.m1.1.1" xref="S3.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S3.p1.1.m1.1b"><ci id="S3.p1.1.m1.1.1.cmml" xref="S3.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.1.m1.1c">p</annotation></semantics></math> sampling, a stochastic truncation sampling method that maintains a minimum level of token diversity while still focusing on high-probability tokens. The method uses a relative probability threshold <math id="S3.p1.2.m2.2" class="ltx_Math" alttext="p_{base}\in(0,1]" display="inline"><semantics id="S3.p1.2.m2.2a"><mrow id="S3.p1.2.m2.2.3" xref="S3.p1.2.m2.2.3.cmml"><msub id="S3.p1.2.m2.2.3.2" xref="S3.p1.2.m2.2.3.2.cmml"><mi id="S3.p1.2.m2.2.3.2.2" xref="S3.p1.2.m2.2.3.2.2.cmml">p</mi><mrow id="S3.p1.2.m2.2.3.2.3" xref="S3.p1.2.m2.2.3.2.3.cmml"><mi id="S3.p1.2.m2.2.3.2.3.2" xref="S3.p1.2.m2.2.3.2.3.2.cmml">b</mi><mo lspace="0em" rspace="0em" id="S3.p1.2.m2.2.3.2.3.1" xref="S3.p1.2.m2.2.3.2.3.1.cmml">​</mo><mi id="S3.p1.2.m2.2.3.2.3.3" xref="S3.p1.2.m2.2.3.2.3.3.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p1.2.m2.2.3.2.3.1a" xref="S3.p1.2.m2.2.3.2.3.1.cmml">​</mo><mi id="S3.p1.2.m2.2.3.2.3.4" xref="S3.p1.2.m2.2.3.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.p1.2.m2.2.3.2.3.1b" xref="S3.p1.2.m2.2.3.2.3.1.cmml">​</mo><mi id="S3.p1.2.m2.2.3.2.3.5" xref="S3.p1.2.m2.2.3.2.3.5.cmml">e</mi></mrow></msub><mo id="S3.p1.2.m2.2.3.1" xref="S3.p1.2.m2.2.3.1.cmml">∈</mo><mrow id="S3.p1.2.m2.2.3.3.2" xref="S3.p1.2.m2.2.3.3.1.cmml"><mo stretchy="false" id="S3.p1.2.m2.2.3.3.2.1" xref="S3.p1.2.m2.2.3.3.1.cmml">(</mo><mn id="S3.p1.2.m2.1.1" xref="S3.p1.2.m2.1.1.cmml">0</mn><mo id="S3.p1.2.m2.2.3.3.2.2" xref="S3.p1.2.m2.2.3.3.1.cmml">,</mo><mn id="S3.p1.2.m2.2.2" xref="S3.p1.2.m2.2.2.cmml">1</mn><mo stretchy="false" id="S3.p1.2.m2.2.3.3.2.3" xref="S3.p1.2.m2.2.3.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p1.2.m2.2b"><apply id="S3.p1.2.m2.2.3.cmml" xref="S3.p1.2.m2.2.3"><in id="S3.p1.2.m2.2.3.1.cmml" xref="S3.p1.2.m2.2.3.1"></in><apply id="S3.p1.2.m2.2.3.2.cmml" xref="S3.p1.2.m2.2.3.2"><csymbol cd="ambiguous" id="S3.p1.2.m2.2.3.2.1.cmml" xref="S3.p1.2.m2.2.3.2">subscript</csymbol><ci id="S3.p1.2.m2.2.3.2.2.cmml" xref="S3.p1.2.m2.2.3.2.2">𝑝</ci><apply id="S3.p1.2.m2.2.3.2.3.cmml" xref="S3.p1.2.m2.2.3.2.3"><times id="S3.p1.2.m2.2.3.2.3.1.cmml" xref="S3.p1.2.m2.2.3.2.3.1"></times><ci id="S3.p1.2.m2.2.3.2.3.2.cmml" xref="S3.p1.2.m2.2.3.2.3.2">𝑏</ci><ci id="S3.p1.2.m2.2.3.2.3.3.cmml" xref="S3.p1.2.m2.2.3.2.3.3">𝑎</ci><ci id="S3.p1.2.m2.2.3.2.3.4.cmml" xref="S3.p1.2.m2.2.3.2.3.4">𝑠</ci><ci id="S3.p1.2.m2.2.3.2.3.5.cmml" xref="S3.p1.2.m2.2.3.2.3.5">𝑒</ci></apply></apply><interval closure="open-closed" id="S3.p1.2.m2.2.3.3.1.cmml" xref="S3.p1.2.m2.2.3.3.2"><cn type="integer" id="S3.p1.2.m2.1.1.cmml" xref="S3.p1.2.m2.1.1">0</cn><cn type="integer" id="S3.p1.2.m2.2.2.cmml" xref="S3.p1.2.m2.2.2">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.2.m2.2c">p_{base}\in(0,1]</annotation></semantics></math> to scales the maximum token probability <math id="S3.p1.3.m3.1" class="ltx_Math" alttext="p_{max}" display="inline"><semantics id="S3.p1.3.m3.1a"><msub id="S3.p1.3.m3.1.1" xref="S3.p1.3.m3.1.1.cmml"><mi id="S3.p1.3.m3.1.1.2" xref="S3.p1.3.m3.1.1.2.cmml">p</mi><mrow id="S3.p1.3.m3.1.1.3" xref="S3.p1.3.m3.1.1.3.cmml"><mi id="S3.p1.3.m3.1.1.3.2" xref="S3.p1.3.m3.1.1.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.p1.3.m3.1.1.3.1" xref="S3.p1.3.m3.1.1.3.1.cmml">​</mo><mi id="S3.p1.3.m3.1.1.3.3" xref="S3.p1.3.m3.1.1.3.3.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p1.3.m3.1.1.3.1a" xref="S3.p1.3.m3.1.1.3.1.cmml">​</mo><mi id="S3.p1.3.m3.1.1.3.4" xref="S3.p1.3.m3.1.1.3.4.cmml">x</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p1.3.m3.1b"><apply id="S3.p1.3.m3.1.1.cmml" xref="S3.p1.3.m3.1.1"><csymbol cd="ambiguous" id="S3.p1.3.m3.1.1.1.cmml" xref="S3.p1.3.m3.1.1">subscript</csymbol><ci id="S3.p1.3.m3.1.1.2.cmml" xref="S3.p1.3.m3.1.1.2">𝑝</ci><apply id="S3.p1.3.m3.1.1.3.cmml" xref="S3.p1.3.m3.1.1.3"><times id="S3.p1.3.m3.1.1.3.1.cmml" xref="S3.p1.3.m3.1.1.3.1"></times><ci id="S3.p1.3.m3.1.1.3.2.cmml" xref="S3.p1.3.m3.1.1.3.2">𝑚</ci><ci id="S3.p1.3.m3.1.1.3.3.cmml" xref="S3.p1.3.m3.1.1.3.3">𝑎</ci><ci id="S3.p1.3.m3.1.1.3.4.cmml" xref="S3.p1.3.m3.1.1.3.4">𝑥</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.3.m3.1c">p_{max}</annotation></semantics></math> to determine the absolute probability threshold <math id="S3.p1.4.m4.1" class="ltx_Math" alttext="p_{scaled}" display="inline"><semantics id="S3.p1.4.m4.1a"><msub id="S3.p1.4.m4.1.1" xref="S3.p1.4.m4.1.1.cmml"><mi id="S3.p1.4.m4.1.1.2" xref="S3.p1.4.m4.1.1.2.cmml">p</mi><mrow id="S3.p1.4.m4.1.1.3" xref="S3.p1.4.m4.1.1.3.cmml"><mi id="S3.p1.4.m4.1.1.3.2" xref="S3.p1.4.m4.1.1.3.2.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.1.3.1" xref="S3.p1.4.m4.1.1.3.1.cmml">​</mo><mi id="S3.p1.4.m4.1.1.3.3" xref="S3.p1.4.m4.1.1.3.3.cmml">c</mi><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.1.3.1a" xref="S3.p1.4.m4.1.1.3.1.cmml">​</mo><mi id="S3.p1.4.m4.1.1.3.4" xref="S3.p1.4.m4.1.1.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.1.3.1b" xref="S3.p1.4.m4.1.1.3.1.cmml">​</mo><mi id="S3.p1.4.m4.1.1.3.5" xref="S3.p1.4.m4.1.1.3.5.cmml">l</mi><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.1.3.1c" xref="S3.p1.4.m4.1.1.3.1.cmml">​</mo><mi id="S3.p1.4.m4.1.1.3.6" xref="S3.p1.4.m4.1.1.3.6.cmml">e</mi><mo lspace="0em" rspace="0em" id="S3.p1.4.m4.1.1.3.1d" xref="S3.p1.4.m4.1.1.3.1.cmml">​</mo><mi id="S3.p1.4.m4.1.1.3.7" xref="S3.p1.4.m4.1.1.3.7.cmml">d</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p1.4.m4.1b"><apply id="S3.p1.4.m4.1.1.cmml" xref="S3.p1.4.m4.1.1"><csymbol cd="ambiguous" id="S3.p1.4.m4.1.1.1.cmml" xref="S3.p1.4.m4.1.1">subscript</csymbol><ci id="S3.p1.4.m4.1.1.2.cmml" xref="S3.p1.4.m4.1.1.2">𝑝</ci><apply id="S3.p1.4.m4.1.1.3.cmml" xref="S3.p1.4.m4.1.1.3"><times id="S3.p1.4.m4.1.1.3.1.cmml" xref="S3.p1.4.m4.1.1.3.1"></times><ci id="S3.p1.4.m4.1.1.3.2.cmml" xref="S3.p1.4.m4.1.1.3.2">𝑠</ci><ci id="S3.p1.4.m4.1.1.3.3.cmml" xref="S3.p1.4.m4.1.1.3.3">𝑐</ci><ci id="S3.p1.4.m4.1.1.3.4.cmml" xref="S3.p1.4.m4.1.1.3.4">𝑎</ci><ci id="S3.p1.4.m4.1.1.3.5.cmml" xref="S3.p1.4.m4.1.1.3.5">𝑙</ci><ci id="S3.p1.4.m4.1.1.3.6.cmml" xref="S3.p1.4.m4.1.1.3.6">𝑒</ci><ci id="S3.p1.4.m4.1.1.3.7.cmml" xref="S3.p1.4.m4.1.1.3.7">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.4.m4.1c">p_{scaled}</annotation></semantics></math>. Sampling is then performed on tokens with probability greater than or equal to <math id="S3.p1.5.m5.1" class="ltx_Math" alttext="p_{scaled}" display="inline"><semantics id="S3.p1.5.m5.1a"><msub id="S3.p1.5.m5.1.1" xref="S3.p1.5.m5.1.1.cmml"><mi id="S3.p1.5.m5.1.1.2" xref="S3.p1.5.m5.1.1.2.cmml">p</mi><mrow id="S3.p1.5.m5.1.1.3" xref="S3.p1.5.m5.1.1.3.cmml"><mi id="S3.p1.5.m5.1.1.3.2" xref="S3.p1.5.m5.1.1.3.2.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.p1.5.m5.1.1.3.1" xref="S3.p1.5.m5.1.1.3.1.cmml">​</mo><mi id="S3.p1.5.m5.1.1.3.3" xref="S3.p1.5.m5.1.1.3.3.cmml">c</mi><mo lspace="0em" rspace="0em" id="S3.p1.5.m5.1.1.3.1a" xref="S3.p1.5.m5.1.1.3.1.cmml">​</mo><mi id="S3.p1.5.m5.1.1.3.4" xref="S3.p1.5.m5.1.1.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p1.5.m5.1.1.3.1b" xref="S3.p1.5.m5.1.1.3.1.cmml">​</mo><mi id="S3.p1.5.m5.1.1.3.5" xref="S3.p1.5.m5.1.1.3.5.cmml">l</mi><mo lspace="0em" rspace="0em" id="S3.p1.5.m5.1.1.3.1c" xref="S3.p1.5.m5.1.1.3.1.cmml">​</mo><mi id="S3.p1.5.m5.1.1.3.6" xref="S3.p1.5.m5.1.1.3.6.cmml">e</mi><mo lspace="0em" rspace="0em" id="S3.p1.5.m5.1.1.3.1d" xref="S3.p1.5.m5.1.1.3.1.cmml">​</mo><mi id="S3.p1.5.m5.1.1.3.7" xref="S3.p1.5.m5.1.1.3.7.cmml">d</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p1.5.m5.1b"><apply id="S3.p1.5.m5.1.1.cmml" xref="S3.p1.5.m5.1.1"><csymbol cd="ambiguous" id="S3.p1.5.m5.1.1.1.cmml" xref="S3.p1.5.m5.1.1">subscript</csymbol><ci id="S3.p1.5.m5.1.1.2.cmml" xref="S3.p1.5.m5.1.1.2">𝑝</ci><apply id="S3.p1.5.m5.1.1.3.cmml" xref="S3.p1.5.m5.1.1.3"><times id="S3.p1.5.m5.1.1.3.1.cmml" xref="S3.p1.5.m5.1.1.3.1"></times><ci id="S3.p1.5.m5.1.1.3.2.cmml" xref="S3.p1.5.m5.1.1.3.2">𝑠</ci><ci id="S3.p1.5.m5.1.1.3.3.cmml" xref="S3.p1.5.m5.1.1.3.3">𝑐</ci><ci id="S3.p1.5.m5.1.1.3.4.cmml" xref="S3.p1.5.m5.1.1.3.4">𝑎</ci><ci id="S3.p1.5.m5.1.1.3.5.cmml" xref="S3.p1.5.m5.1.1.3.5">𝑙</ci><ci id="S3.p1.5.m5.1.1.3.6.cmml" xref="S3.p1.5.m5.1.1.3.6">𝑒</ci><ci id="S3.p1.5.m5.1.1.3.7.cmml" xref="S3.p1.5.m5.1.1.3.7">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p1.5.m5.1c">p_{scaled}</annotation></semantics></math>.</p>
</div>
<div id="S3.p2" class="ltx_para">
<p id="S3.p2.2" class="ltx_p">Formally, given the maximum probability over the token distribution <math id="S3.p2.1.m1.2" class="ltx_Math" alttext="p_{max}=\max_{v\in\mathcal{V}}P(v|x_{1},\dots,x_{t-1})" display="inline"><semantics id="S3.p2.1.m1.2a"><mrow id="S3.p2.1.m1.2.2" xref="S3.p2.1.m1.2.2.cmml"><msub id="S3.p2.1.m1.2.2.3" xref="S3.p2.1.m1.2.2.3.cmml"><mi id="S3.p2.1.m1.2.2.3.2" xref="S3.p2.1.m1.2.2.3.2.cmml">p</mi><mrow id="S3.p2.1.m1.2.2.3.3" xref="S3.p2.1.m1.2.2.3.3.cmml"><mi id="S3.p2.1.m1.2.2.3.3.2" xref="S3.p2.1.m1.2.2.3.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.p2.1.m1.2.2.3.3.1" xref="S3.p2.1.m1.2.2.3.3.1.cmml">​</mo><mi id="S3.p2.1.m1.2.2.3.3.3" xref="S3.p2.1.m1.2.2.3.3.3.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p2.1.m1.2.2.3.3.1a" xref="S3.p2.1.m1.2.2.3.3.1.cmml">​</mo><mi id="S3.p2.1.m1.2.2.3.3.4" xref="S3.p2.1.m1.2.2.3.3.4.cmml">x</mi></mrow></msub><mo id="S3.p2.1.m1.2.2.2" xref="S3.p2.1.m1.2.2.2.cmml">=</mo><mrow id="S3.p2.1.m1.2.2.1" xref="S3.p2.1.m1.2.2.1.cmml"><mrow id="S3.p2.1.m1.2.2.1.3" xref="S3.p2.1.m1.2.2.1.3.cmml"><msub id="S3.p2.1.m1.2.2.1.3.1" xref="S3.p2.1.m1.2.2.1.3.1.cmml"><mi id="S3.p2.1.m1.2.2.1.3.1.2" xref="S3.p2.1.m1.2.2.1.3.1.2.cmml">max</mi><mrow id="S3.p2.1.m1.2.2.1.3.1.3" xref="S3.p2.1.m1.2.2.1.3.1.3.cmml"><mi id="S3.p2.1.m1.2.2.1.3.1.3.2" xref="S3.p2.1.m1.2.2.1.3.1.3.2.cmml">v</mi><mo id="S3.p2.1.m1.2.2.1.3.1.3.1" xref="S3.p2.1.m1.2.2.1.3.1.3.1.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S3.p2.1.m1.2.2.1.3.1.3.3" xref="S3.p2.1.m1.2.2.1.3.1.3.3.cmml">𝒱</mi></mrow></msub><mo lspace="0.167em" id="S3.p2.1.m1.2.2.1.3a" xref="S3.p2.1.m1.2.2.1.3.cmml">⁡</mo><mi id="S3.p2.1.m1.2.2.1.3.2" xref="S3.p2.1.m1.2.2.1.3.2.cmml">P</mi></mrow><mo lspace="0em" rspace="0em" id="S3.p2.1.m1.2.2.1.2" xref="S3.p2.1.m1.2.2.1.2.cmml">​</mo><mrow id="S3.p2.1.m1.2.2.1.1.1" xref="S3.p2.1.m1.2.2.1.1.1.1.cmml"><mo stretchy="false" id="S3.p2.1.m1.2.2.1.1.1.2" xref="S3.p2.1.m1.2.2.1.1.1.1.cmml">(</mo><mrow id="S3.p2.1.m1.2.2.1.1.1.1" xref="S3.p2.1.m1.2.2.1.1.1.1.cmml"><mi id="S3.p2.1.m1.2.2.1.1.1.1.4" xref="S3.p2.1.m1.2.2.1.1.1.1.4.cmml">v</mi><mo fence="false" id="S3.p2.1.m1.2.2.1.1.1.1.3" xref="S3.p2.1.m1.2.2.1.1.1.1.3.cmml">|</mo><mrow id="S3.p2.1.m1.2.2.1.1.1.1.2.2" xref="S3.p2.1.m1.2.2.1.1.1.1.2.3.cmml"><msub id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.cmml"><mi id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.2" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.3" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S3.p2.1.m1.2.2.1.1.1.1.2.2.3" xref="S3.p2.1.m1.2.2.1.1.1.1.2.3.cmml">,</mo><mi mathvariant="normal" id="S3.p2.1.m1.1.1" xref="S3.p2.1.m1.1.1.cmml">…</mi><mo id="S3.p2.1.m1.2.2.1.1.1.1.2.2.4" xref="S3.p2.1.m1.2.2.1.1.1.1.2.3.cmml">,</mo><msub id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.cmml"><mi id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.2" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.2.cmml">x</mi><mrow id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.cmml"><mi id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.2" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.2.cmml">t</mi><mo id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.1" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.1.cmml">−</mo><mn id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.3" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S3.p2.1.m1.2.2.1.1.1.3" xref="S3.p2.1.m1.2.2.1.1.1.1.cmml">)</mo></mrow></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.p2.1.m1.2b"><apply id="S3.p2.1.m1.2.2.cmml" xref="S3.p2.1.m1.2.2"><eq id="S3.p2.1.m1.2.2.2.cmml" xref="S3.p2.1.m1.2.2.2"></eq><apply id="S3.p2.1.m1.2.2.3.cmml" xref="S3.p2.1.m1.2.2.3"><csymbol cd="ambiguous" id="S3.p2.1.m1.2.2.3.1.cmml" xref="S3.p2.1.m1.2.2.3">subscript</csymbol><ci id="S3.p2.1.m1.2.2.3.2.cmml" xref="S3.p2.1.m1.2.2.3.2">𝑝</ci><apply id="S3.p2.1.m1.2.2.3.3.cmml" xref="S3.p2.1.m1.2.2.3.3"><times id="S3.p2.1.m1.2.2.3.3.1.cmml" xref="S3.p2.1.m1.2.2.3.3.1"></times><ci id="S3.p2.1.m1.2.2.3.3.2.cmml" xref="S3.p2.1.m1.2.2.3.3.2">𝑚</ci><ci id="S3.p2.1.m1.2.2.3.3.3.cmml" xref="S3.p2.1.m1.2.2.3.3.3">𝑎</ci><ci id="S3.p2.1.m1.2.2.3.3.4.cmml" xref="S3.p2.1.m1.2.2.3.3.4">𝑥</ci></apply></apply><apply id="S3.p2.1.m1.2.2.1.cmml" xref="S3.p2.1.m1.2.2.1"><times id="S3.p2.1.m1.2.2.1.2.cmml" xref="S3.p2.1.m1.2.2.1.2"></times><apply id="S3.p2.1.m1.2.2.1.3.cmml" xref="S3.p2.1.m1.2.2.1.3"><apply id="S3.p2.1.m1.2.2.1.3.1.cmml" xref="S3.p2.1.m1.2.2.1.3.1"><csymbol cd="ambiguous" id="S3.p2.1.m1.2.2.1.3.1.1.cmml" xref="S3.p2.1.m1.2.2.1.3.1">subscript</csymbol><max id="S3.p2.1.m1.2.2.1.3.1.2.cmml" xref="S3.p2.1.m1.2.2.1.3.1.2"></max><apply id="S3.p2.1.m1.2.2.1.3.1.3.cmml" xref="S3.p2.1.m1.2.2.1.3.1.3"><in id="S3.p2.1.m1.2.2.1.3.1.3.1.cmml" xref="S3.p2.1.m1.2.2.1.3.1.3.1"></in><ci id="S3.p2.1.m1.2.2.1.3.1.3.2.cmml" xref="S3.p2.1.m1.2.2.1.3.1.3.2">𝑣</ci><ci id="S3.p2.1.m1.2.2.1.3.1.3.3.cmml" xref="S3.p2.1.m1.2.2.1.3.1.3.3">𝒱</ci></apply></apply><ci id="S3.p2.1.m1.2.2.1.3.2.cmml" xref="S3.p2.1.m1.2.2.1.3.2">𝑃</ci></apply><apply id="S3.p2.1.m1.2.2.1.1.1.1.cmml" xref="S3.p2.1.m1.2.2.1.1.1"><csymbol cd="latexml" id="S3.p2.1.m1.2.2.1.1.1.1.3.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.3">conditional</csymbol><ci id="S3.p2.1.m1.2.2.1.1.1.1.4.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.4">𝑣</ci><list id="S3.p2.1.m1.2.2.1.1.1.1.2.3.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2"><apply id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1">subscript</csymbol><ci id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.2.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.3.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.1.1.1.3">1</cn></apply><ci id="S3.p2.1.m1.1.1.cmml" xref="S3.p2.1.m1.1.1">…</ci><apply id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.1.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2">subscript</csymbol><ci id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.2.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.2">𝑥</ci><apply id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3"><minus id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.1.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.1"></minus><ci id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.2.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.2">𝑡</ci><cn type="integer" id="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.3.cmml" xref="S3.p2.1.m1.2.2.1.1.1.1.2.2.2.3.3">1</cn></apply></apply></list></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.1.m1.2c">p_{max}=\max_{v\in\mathcal{V}}P(v|x_{1},\dots,x_{t-1})</annotation></semantics></math>, the absolute probability threshold <math id="S3.p2.2.m2.1" class="ltx_Math" alttext="p_{scaled}" display="inline"><semantics id="S3.p2.2.m2.1a"><msub id="S3.p2.2.m2.1.1" xref="S3.p2.2.m2.1.1.cmml"><mi id="S3.p2.2.m2.1.1.2" xref="S3.p2.2.m2.1.1.2.cmml">p</mi><mrow id="S3.p2.2.m2.1.1.3" xref="S3.p2.2.m2.1.1.3.cmml"><mi id="S3.p2.2.m2.1.1.3.2" xref="S3.p2.2.m2.1.1.3.2.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.p2.2.m2.1.1.3.1" xref="S3.p2.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p2.2.m2.1.1.3.3" xref="S3.p2.2.m2.1.1.3.3.cmml">c</mi><mo lspace="0em" rspace="0em" id="S3.p2.2.m2.1.1.3.1a" xref="S3.p2.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p2.2.m2.1.1.3.4" xref="S3.p2.2.m2.1.1.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p2.2.m2.1.1.3.1b" xref="S3.p2.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p2.2.m2.1.1.3.5" xref="S3.p2.2.m2.1.1.3.5.cmml">l</mi><mo lspace="0em" rspace="0em" id="S3.p2.2.m2.1.1.3.1c" xref="S3.p2.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p2.2.m2.1.1.3.6" xref="S3.p2.2.m2.1.1.3.6.cmml">e</mi><mo lspace="0em" rspace="0em" id="S3.p2.2.m2.1.1.3.1d" xref="S3.p2.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p2.2.m2.1.1.3.7" xref="S3.p2.2.m2.1.1.3.7.cmml">d</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p2.2.m2.1b"><apply id="S3.p2.2.m2.1.1.cmml" xref="S3.p2.2.m2.1.1"><csymbol cd="ambiguous" id="S3.p2.2.m2.1.1.1.cmml" xref="S3.p2.2.m2.1.1">subscript</csymbol><ci id="S3.p2.2.m2.1.1.2.cmml" xref="S3.p2.2.m2.1.1.2">𝑝</ci><apply id="S3.p2.2.m2.1.1.3.cmml" xref="S3.p2.2.m2.1.1.3"><times id="S3.p2.2.m2.1.1.3.1.cmml" xref="S3.p2.2.m2.1.1.3.1"></times><ci id="S3.p2.2.m2.1.1.3.2.cmml" xref="S3.p2.2.m2.1.1.3.2">𝑠</ci><ci id="S3.p2.2.m2.1.1.3.3.cmml" xref="S3.p2.2.m2.1.1.3.3">𝑐</ci><ci id="S3.p2.2.m2.1.1.3.4.cmml" xref="S3.p2.2.m2.1.1.3.4">𝑎</ci><ci id="S3.p2.2.m2.1.1.3.5.cmml" xref="S3.p2.2.m2.1.1.3.5">𝑙</ci><ci id="S3.p2.2.m2.1.1.3.6.cmml" xref="S3.p2.2.m2.1.1.3.6">𝑒</ci><ci id="S3.p2.2.m2.1.1.3.7.cmml" xref="S3.p2.2.m2.1.1.3.7">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.2.m2.1c">p_{scaled}</annotation></semantics></math> is calculated as:</p>
<table id="S3.E6" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E6.m1.1" class="ltx_Math" alttext="p_{scaled}=p_{base}\times p_{max}" display="block"><semantics id="S3.E6.m1.1a"><mrow id="S3.E6.m1.1.1" xref="S3.E6.m1.1.1.cmml"><msub id="S3.E6.m1.1.1.2" xref="S3.E6.m1.1.1.2.cmml"><mi id="S3.E6.m1.1.1.2.2" xref="S3.E6.m1.1.1.2.2.cmml">p</mi><mrow id="S3.E6.m1.1.1.2.3" xref="S3.E6.m1.1.1.2.3.cmml"><mi id="S3.E6.m1.1.1.2.3.2" xref="S3.E6.m1.1.1.2.3.2.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.2.3.1" xref="S3.E6.m1.1.1.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.2.3.3" xref="S3.E6.m1.1.1.2.3.3.cmml">c</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.2.3.1a" xref="S3.E6.m1.1.1.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.2.3.4" xref="S3.E6.m1.1.1.2.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.2.3.1b" xref="S3.E6.m1.1.1.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.2.3.5" xref="S3.E6.m1.1.1.2.3.5.cmml">l</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.2.3.1c" xref="S3.E6.m1.1.1.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.2.3.6" xref="S3.E6.m1.1.1.2.3.6.cmml">e</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.2.3.1d" xref="S3.E6.m1.1.1.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.2.3.7" xref="S3.E6.m1.1.1.2.3.7.cmml">d</mi></mrow></msub><mo id="S3.E6.m1.1.1.1" xref="S3.E6.m1.1.1.1.cmml">=</mo><mrow id="S3.E6.m1.1.1.3" xref="S3.E6.m1.1.1.3.cmml"><msub id="S3.E6.m1.1.1.3.2" xref="S3.E6.m1.1.1.3.2.cmml"><mi id="S3.E6.m1.1.1.3.2.2" xref="S3.E6.m1.1.1.3.2.2.cmml">p</mi><mrow id="S3.E6.m1.1.1.3.2.3" xref="S3.E6.m1.1.1.3.2.3.cmml"><mi id="S3.E6.m1.1.1.3.2.3.2" xref="S3.E6.m1.1.1.3.2.3.2.cmml">b</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.3.2.3.1" xref="S3.E6.m1.1.1.3.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.3.2.3.3" xref="S3.E6.m1.1.1.3.2.3.3.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.3.2.3.1a" xref="S3.E6.m1.1.1.3.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.3.2.3.4" xref="S3.E6.m1.1.1.3.2.3.4.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.3.2.3.1b" xref="S3.E6.m1.1.1.3.2.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.3.2.3.5" xref="S3.E6.m1.1.1.3.2.3.5.cmml">e</mi></mrow></msub><mo lspace="0.222em" rspace="0.222em" id="S3.E6.m1.1.1.3.1" xref="S3.E6.m1.1.1.3.1.cmml">×</mo><msub id="S3.E6.m1.1.1.3.3" xref="S3.E6.m1.1.1.3.3.cmml"><mi id="S3.E6.m1.1.1.3.3.2" xref="S3.E6.m1.1.1.3.3.2.cmml">p</mi><mrow id="S3.E6.m1.1.1.3.3.3" xref="S3.E6.m1.1.1.3.3.3.cmml"><mi id="S3.E6.m1.1.1.3.3.3.2" xref="S3.E6.m1.1.1.3.3.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.3.3.3.1" xref="S3.E6.m1.1.1.3.3.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.3.3.3.3" xref="S3.E6.m1.1.1.3.3.3.3.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.E6.m1.1.1.3.3.3.1a" xref="S3.E6.m1.1.1.3.3.3.1.cmml">​</mo><mi id="S3.E6.m1.1.1.3.3.3.4" xref="S3.E6.m1.1.1.3.3.3.4.cmml">x</mi></mrow></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.E6.m1.1b"><apply id="S3.E6.m1.1.1.cmml" xref="S3.E6.m1.1.1"><eq id="S3.E6.m1.1.1.1.cmml" xref="S3.E6.m1.1.1.1"></eq><apply id="S3.E6.m1.1.1.2.cmml" xref="S3.E6.m1.1.1.2"><csymbol cd="ambiguous" id="S3.E6.m1.1.1.2.1.cmml" xref="S3.E6.m1.1.1.2">subscript</csymbol><ci id="S3.E6.m1.1.1.2.2.cmml" xref="S3.E6.m1.1.1.2.2">𝑝</ci><apply id="S3.E6.m1.1.1.2.3.cmml" xref="S3.E6.m1.1.1.2.3"><times id="S3.E6.m1.1.1.2.3.1.cmml" xref="S3.E6.m1.1.1.2.3.1"></times><ci id="S3.E6.m1.1.1.2.3.2.cmml" xref="S3.E6.m1.1.1.2.3.2">𝑠</ci><ci id="S3.E6.m1.1.1.2.3.3.cmml" xref="S3.E6.m1.1.1.2.3.3">𝑐</ci><ci id="S3.E6.m1.1.1.2.3.4.cmml" xref="S3.E6.m1.1.1.2.3.4">𝑎</ci><ci id="S3.E6.m1.1.1.2.3.5.cmml" xref="S3.E6.m1.1.1.2.3.5">𝑙</ci><ci id="S3.E6.m1.1.1.2.3.6.cmml" xref="S3.E6.m1.1.1.2.3.6">𝑒</ci><ci id="S3.E6.m1.1.1.2.3.7.cmml" xref="S3.E6.m1.1.1.2.3.7">𝑑</ci></apply></apply><apply id="S3.E6.m1.1.1.3.cmml" xref="S3.E6.m1.1.1.3"><times id="S3.E6.m1.1.1.3.1.cmml" xref="S3.E6.m1.1.1.3.1"></times><apply id="S3.E6.m1.1.1.3.2.cmml" xref="S3.E6.m1.1.1.3.2"><csymbol cd="ambiguous" id="S3.E6.m1.1.1.3.2.1.cmml" xref="S3.E6.m1.1.1.3.2">subscript</csymbol><ci id="S3.E6.m1.1.1.3.2.2.cmml" xref="S3.E6.m1.1.1.3.2.2">𝑝</ci><apply id="S3.E6.m1.1.1.3.2.3.cmml" xref="S3.E6.m1.1.1.3.2.3"><times id="S3.E6.m1.1.1.3.2.3.1.cmml" xref="S3.E6.m1.1.1.3.2.3.1"></times><ci id="S3.E6.m1.1.1.3.2.3.2.cmml" xref="S3.E6.m1.1.1.3.2.3.2">𝑏</ci><ci id="S3.E6.m1.1.1.3.2.3.3.cmml" xref="S3.E6.m1.1.1.3.2.3.3">𝑎</ci><ci id="S3.E6.m1.1.1.3.2.3.4.cmml" xref="S3.E6.m1.1.1.3.2.3.4">𝑠</ci><ci id="S3.E6.m1.1.1.3.2.3.5.cmml" xref="S3.E6.m1.1.1.3.2.3.5">𝑒</ci></apply></apply><apply id="S3.E6.m1.1.1.3.3.cmml" xref="S3.E6.m1.1.1.3.3"><csymbol cd="ambiguous" id="S3.E6.m1.1.1.3.3.1.cmml" xref="S3.E6.m1.1.1.3.3">subscript</csymbol><ci id="S3.E6.m1.1.1.3.3.2.cmml" xref="S3.E6.m1.1.1.3.3.2">𝑝</ci><apply id="S3.E6.m1.1.1.3.3.3.cmml" xref="S3.E6.m1.1.1.3.3.3"><times id="S3.E6.m1.1.1.3.3.3.1.cmml" xref="S3.E6.m1.1.1.3.3.3.1"></times><ci id="S3.E6.m1.1.1.3.3.3.2.cmml" xref="S3.E6.m1.1.1.3.3.3.2">𝑚</ci><ci id="S3.E6.m1.1.1.3.3.3.3.cmml" xref="S3.E6.m1.1.1.3.3.3.3">𝑎</ci><ci id="S3.E6.m1.1.1.3.3.3.4.cmml" xref="S3.E6.m1.1.1.3.3.3.4">𝑥</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E6.m1.1c">p_{scaled}=p_{base}\times p_{max}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(6)</span></td>
</tr></tbody>
</table>
</div>
<div id="S3.p3" class="ltx_para">
<p id="S3.p3.2" class="ltx_p">The sampling pool <math id="S3.p3.1.m1.1" class="ltx_Math" alttext="\mathcal{V}_{{min}}" display="inline"><semantics id="S3.p3.1.m1.1a"><msub id="S3.p3.1.m1.1.1" xref="S3.p3.1.m1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.p3.1.m1.1.1.2" xref="S3.p3.1.m1.1.1.2.cmml">𝒱</mi><mrow id="S3.p3.1.m1.1.1.3" xref="S3.p3.1.m1.1.1.3.cmml"><mi id="S3.p3.1.m1.1.1.3.2" xref="S3.p3.1.m1.1.1.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.p3.1.m1.1.1.3.1" xref="S3.p3.1.m1.1.1.3.1.cmml">​</mo><mi id="S3.p3.1.m1.1.1.3.3" xref="S3.p3.1.m1.1.1.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="S3.p3.1.m1.1.1.3.1a" xref="S3.p3.1.m1.1.1.3.1.cmml">​</mo><mi id="S3.p3.1.m1.1.1.3.4" xref="S3.p3.1.m1.1.1.3.4.cmml">n</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p3.1.m1.1b"><apply id="S3.p3.1.m1.1.1.cmml" xref="S3.p3.1.m1.1.1"><csymbol cd="ambiguous" id="S3.p3.1.m1.1.1.1.cmml" xref="S3.p3.1.m1.1.1">subscript</csymbol><ci id="S3.p3.1.m1.1.1.2.cmml" xref="S3.p3.1.m1.1.1.2">𝒱</ci><apply id="S3.p3.1.m1.1.1.3.cmml" xref="S3.p3.1.m1.1.1.3"><times id="S3.p3.1.m1.1.1.3.1.cmml" xref="S3.p3.1.m1.1.1.3.1"></times><ci id="S3.p3.1.m1.1.1.3.2.cmml" xref="S3.p3.1.m1.1.1.3.2">𝑚</ci><ci id="S3.p3.1.m1.1.1.3.3.cmml" xref="S3.p3.1.m1.1.1.3.3">𝑖</ci><ci id="S3.p3.1.m1.1.1.3.4.cmml" xref="S3.p3.1.m1.1.1.3.4">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.1.m1.1c">\mathcal{V}_{{min}}</annotation></semantics></math> is then defined as the set of tokens whose probability is greater than or equal to <math id="S3.p3.2.m2.1" class="ltx_Math" alttext="p_{scaled}" display="inline"><semantics id="S3.p3.2.m2.1a"><msub id="S3.p3.2.m2.1.1" xref="S3.p3.2.m2.1.1.cmml"><mi id="S3.p3.2.m2.1.1.2" xref="S3.p3.2.m2.1.1.2.cmml">p</mi><mrow id="S3.p3.2.m2.1.1.3" xref="S3.p3.2.m2.1.1.3.cmml"><mi id="S3.p3.2.m2.1.1.3.2" xref="S3.p3.2.m2.1.1.3.2.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.p3.2.m2.1.1.3.1" xref="S3.p3.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p3.2.m2.1.1.3.3" xref="S3.p3.2.m2.1.1.3.3.cmml">c</mi><mo lspace="0em" rspace="0em" id="S3.p3.2.m2.1.1.3.1a" xref="S3.p3.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p3.2.m2.1.1.3.4" xref="S3.p3.2.m2.1.1.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.p3.2.m2.1.1.3.1b" xref="S3.p3.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p3.2.m2.1.1.3.5" xref="S3.p3.2.m2.1.1.3.5.cmml">l</mi><mo lspace="0em" rspace="0em" id="S3.p3.2.m2.1.1.3.1c" xref="S3.p3.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p3.2.m2.1.1.3.6" xref="S3.p3.2.m2.1.1.3.6.cmml">e</mi><mo lspace="0em" rspace="0em" id="S3.p3.2.m2.1.1.3.1d" xref="S3.p3.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p3.2.m2.1.1.3.7" xref="S3.p3.2.m2.1.1.3.7.cmml">d</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p3.2.m2.1b"><apply id="S3.p3.2.m2.1.1.cmml" xref="S3.p3.2.m2.1.1"><csymbol cd="ambiguous" id="S3.p3.2.m2.1.1.1.cmml" xref="S3.p3.2.m2.1.1">subscript</csymbol><ci id="S3.p3.2.m2.1.1.2.cmml" xref="S3.p3.2.m2.1.1.2">𝑝</ci><apply id="S3.p3.2.m2.1.1.3.cmml" xref="S3.p3.2.m2.1.1.3"><times id="S3.p3.2.m2.1.1.3.1.cmml" xref="S3.p3.2.m2.1.1.3.1"></times><ci id="S3.p3.2.m2.1.1.3.2.cmml" xref="S3.p3.2.m2.1.1.3.2">𝑠</ci><ci id="S3.p3.2.m2.1.1.3.3.cmml" xref="S3.p3.2.m2.1.1.3.3">𝑐</ci><ci id="S3.p3.2.m2.1.1.3.4.cmml" xref="S3.p3.2.m2.1.1.3.4">𝑎</ci><ci id="S3.p3.2.m2.1.1.3.5.cmml" xref="S3.p3.2.m2.1.1.3.5">𝑙</ci><ci id="S3.p3.2.m2.1.1.3.6.cmml" xref="S3.p3.2.m2.1.1.3.6">𝑒</ci><ci id="S3.p3.2.m2.1.1.3.7.cmml" xref="S3.p3.2.m2.1.1.3.7">𝑑</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.2.m2.1c">p_{scaled}</annotation></semantics></math>:</p>
<table id="S3.E7" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E7.m1.2" class="ltx_Math" alttext="\mathcal{V}_{min}={v\in\mathcal{V}:P(v|x_{1},x_{2},\dots,x_{t-1})\geq p_{scaled}}" display="block"><semantics id="S3.E7.m1.2a"><mrow id="S3.E7.m1.2.2" xref="S3.E7.m1.2.2.cmml"><mrow id="S3.E7.m1.2.2.3" xref="S3.E7.m1.2.2.3.cmml"><msub id="S3.E7.m1.2.2.3.2" xref="S3.E7.m1.2.2.3.2.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.E7.m1.2.2.3.2.2" xref="S3.E7.m1.2.2.3.2.2.cmml">𝒱</mi><mrow id="S3.E7.m1.2.2.3.2.3" xref="S3.E7.m1.2.2.3.2.3.cmml"><mi id="S3.E7.m1.2.2.3.2.3.2" xref="S3.E7.m1.2.2.3.2.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.3.2.3.1" xref="S3.E7.m1.2.2.3.2.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.3.2.3.3" xref="S3.E7.m1.2.2.3.2.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.3.2.3.1a" xref="S3.E7.m1.2.2.3.2.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.3.2.3.4" xref="S3.E7.m1.2.2.3.2.3.4.cmml">n</mi></mrow></msub><mo id="S3.E7.m1.2.2.3.3" xref="S3.E7.m1.2.2.3.3.cmml">=</mo><mi id="S3.E7.m1.2.2.3.4" xref="S3.E7.m1.2.2.3.4.cmml">v</mi><mo id="S3.E7.m1.2.2.3.5" xref="S3.E7.m1.2.2.3.5.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S3.E7.m1.2.2.3.6" xref="S3.E7.m1.2.2.3.6.cmml">𝒱</mi></mrow><mo lspace="0.278em" rspace="0.278em" id="S3.E7.m1.2.2.2" xref="S3.E7.m1.2.2.2.cmml">:</mo><mrow id="S3.E7.m1.2.2.1" xref="S3.E7.m1.2.2.1.cmml"><mrow id="S3.E7.m1.2.2.1.1" xref="S3.E7.m1.2.2.1.1.cmml"><mi id="S3.E7.m1.2.2.1.1.3" xref="S3.E7.m1.2.2.1.1.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.1.1.2" xref="S3.E7.m1.2.2.1.1.2.cmml">​</mo><mrow id="S3.E7.m1.2.2.1.1.1.1" xref="S3.E7.m1.2.2.1.1.1.1.1.cmml"><mo stretchy="false" id="S3.E7.m1.2.2.1.1.1.1.2" xref="S3.E7.m1.2.2.1.1.1.1.1.cmml">(</mo><mrow id="S3.E7.m1.2.2.1.1.1.1.1" xref="S3.E7.m1.2.2.1.1.1.1.1.cmml"><mi id="S3.E7.m1.2.2.1.1.1.1.1.5" xref="S3.E7.m1.2.2.1.1.1.1.1.5.cmml">v</mi><mo fence="false" id="S3.E7.m1.2.2.1.1.1.1.1.4" xref="S3.E7.m1.2.2.1.1.1.1.1.4.cmml">|</mo><mrow id="S3.E7.m1.2.2.1.1.1.1.1.3.3" xref="S3.E7.m1.2.2.1.1.1.1.1.3.4.cmml"><msub id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.cmml"><mi id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.2" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.2.cmml">x</mi><mn id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.3" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S3.E7.m1.2.2.1.1.1.1.1.3.3.4" xref="S3.E7.m1.2.2.1.1.1.1.1.3.4.cmml">,</mo><msub id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.cmml"><mi id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.2" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.2.cmml">x</mi><mn id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.3" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.3.cmml">2</mn></msub><mo id="S3.E7.m1.2.2.1.1.1.1.1.3.3.5" xref="S3.E7.m1.2.2.1.1.1.1.1.3.4.cmml">,</mo><mi mathvariant="normal" id="S3.E7.m1.1.1" xref="S3.E7.m1.1.1.cmml">…</mi><mo id="S3.E7.m1.2.2.1.1.1.1.1.3.3.6" xref="S3.E7.m1.2.2.1.1.1.1.1.3.4.cmml">,</mo><msub id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.cmml"><mi id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.2" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.2.cmml">x</mi><mrow id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.cmml"><mi id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.2" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.2.cmml">t</mi><mo id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.1" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.1.cmml">−</mo><mn id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.3" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S3.E7.m1.2.2.1.1.1.1.3" xref="S3.E7.m1.2.2.1.1.1.1.1.cmml">)</mo></mrow></mrow><mo id="S3.E7.m1.2.2.1.2" xref="S3.E7.m1.2.2.1.2.cmml">≥</mo><msub id="S3.E7.m1.2.2.1.3" xref="S3.E7.m1.2.2.1.3.cmml"><mi id="S3.E7.m1.2.2.1.3.2" xref="S3.E7.m1.2.2.1.3.2.cmml">p</mi><mrow id="S3.E7.m1.2.2.1.3.3" xref="S3.E7.m1.2.2.1.3.3.cmml"><mi id="S3.E7.m1.2.2.1.3.3.2" xref="S3.E7.m1.2.2.1.3.3.2.cmml">s</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.1.3.3.1" xref="S3.E7.m1.2.2.1.3.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.1.3.3.3" xref="S3.E7.m1.2.2.1.3.3.3.cmml">c</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.1.3.3.1a" xref="S3.E7.m1.2.2.1.3.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.1.3.3.4" xref="S3.E7.m1.2.2.1.3.3.4.cmml">a</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.1.3.3.1b" xref="S3.E7.m1.2.2.1.3.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.1.3.3.5" xref="S3.E7.m1.2.2.1.3.3.5.cmml">l</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.1.3.3.1c" xref="S3.E7.m1.2.2.1.3.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.1.3.3.6" xref="S3.E7.m1.2.2.1.3.3.6.cmml">e</mi><mo lspace="0em" rspace="0em" id="S3.E7.m1.2.2.1.3.3.1d" xref="S3.E7.m1.2.2.1.3.3.1.cmml">​</mo><mi id="S3.E7.m1.2.2.1.3.3.7" xref="S3.E7.m1.2.2.1.3.3.7.cmml">d</mi></mrow></msub></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.E7.m1.2b"><apply id="S3.E7.m1.2.2.cmml" xref="S3.E7.m1.2.2"><ci id="S3.E7.m1.2.2.2.cmml" xref="S3.E7.m1.2.2.2">:</ci><apply id="S3.E7.m1.2.2.3.cmml" xref="S3.E7.m1.2.2.3"><and id="S3.E7.m1.2.2.3a.cmml" xref="S3.E7.m1.2.2.3"></and><apply id="S3.E7.m1.2.2.3b.cmml" xref="S3.E7.m1.2.2.3"><eq id="S3.E7.m1.2.2.3.3.cmml" xref="S3.E7.m1.2.2.3.3"></eq><apply id="S3.E7.m1.2.2.3.2.cmml" xref="S3.E7.m1.2.2.3.2"><csymbol cd="ambiguous" id="S3.E7.m1.2.2.3.2.1.cmml" xref="S3.E7.m1.2.2.3.2">subscript</csymbol><ci id="S3.E7.m1.2.2.3.2.2.cmml" xref="S3.E7.m1.2.2.3.2.2">𝒱</ci><apply id="S3.E7.m1.2.2.3.2.3.cmml" xref="S3.E7.m1.2.2.3.2.3"><times id="S3.E7.m1.2.2.3.2.3.1.cmml" xref="S3.E7.m1.2.2.3.2.3.1"></times><ci id="S3.E7.m1.2.2.3.2.3.2.cmml" xref="S3.E7.m1.2.2.3.2.3.2">𝑚</ci><ci id="S3.E7.m1.2.2.3.2.3.3.cmml" xref="S3.E7.m1.2.2.3.2.3.3">𝑖</ci><ci id="S3.E7.m1.2.2.3.2.3.4.cmml" xref="S3.E7.m1.2.2.3.2.3.4">𝑛</ci></apply></apply><ci id="S3.E7.m1.2.2.3.4.cmml" xref="S3.E7.m1.2.2.3.4">𝑣</ci></apply><apply id="S3.E7.m1.2.2.3c.cmml" xref="S3.E7.m1.2.2.3"><in id="S3.E7.m1.2.2.3.5.cmml" xref="S3.E7.m1.2.2.3.5"></in><share href="#S3.E7.m1.2.2.3.4.cmml" id="S3.E7.m1.2.2.3d.cmml" xref="S3.E7.m1.2.2.3"></share><ci id="S3.E7.m1.2.2.3.6.cmml" xref="S3.E7.m1.2.2.3.6">𝒱</ci></apply></apply><apply id="S3.E7.m1.2.2.1.cmml" xref="S3.E7.m1.2.2.1"><geq id="S3.E7.m1.2.2.1.2.cmml" xref="S3.E7.m1.2.2.1.2"></geq><apply id="S3.E7.m1.2.2.1.1.cmml" xref="S3.E7.m1.2.2.1.1"><times id="S3.E7.m1.2.2.1.1.2.cmml" xref="S3.E7.m1.2.2.1.1.2"></times><ci id="S3.E7.m1.2.2.1.1.3.cmml" xref="S3.E7.m1.2.2.1.1.3">𝑃</ci><apply id="S3.E7.m1.2.2.1.1.1.1.1.cmml" xref="S3.E7.m1.2.2.1.1.1.1"><csymbol cd="latexml" id="S3.E7.m1.2.2.1.1.1.1.1.4.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.4">conditional</csymbol><ci id="S3.E7.m1.2.2.1.1.1.1.1.5.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.5">𝑣</ci><list id="S3.E7.m1.2.2.1.1.1.1.1.3.4.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3"><apply id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.1.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1">subscript</csymbol><ci id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.2.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.3.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.1.1.1.3">1</cn></apply><apply id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.1.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2">subscript</csymbol><ci id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.2.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.2">𝑥</ci><cn type="integer" id="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.3.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.2.2.2.3">2</cn></apply><ci id="S3.E7.m1.1.1.cmml" xref="S3.E7.m1.1.1">…</ci><apply id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3"><csymbol cd="ambiguous" id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.1.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3">subscript</csymbol><ci id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.2.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.2">𝑥</ci><apply id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3"><minus id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.1.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.1"></minus><ci id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.2.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.2">𝑡</ci><cn type="integer" id="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.3.cmml" xref="S3.E7.m1.2.2.1.1.1.1.1.3.3.3.3.3">1</cn></apply></apply></list></apply></apply><apply id="S3.E7.m1.2.2.1.3.cmml" xref="S3.E7.m1.2.2.1.3"><csymbol cd="ambiguous" id="S3.E7.m1.2.2.1.3.1.cmml" xref="S3.E7.m1.2.2.1.3">subscript</csymbol><ci id="S3.E7.m1.2.2.1.3.2.cmml" xref="S3.E7.m1.2.2.1.3.2">𝑝</ci><apply id="S3.E7.m1.2.2.1.3.3.cmml" xref="S3.E7.m1.2.2.1.3.3"><times id="S3.E7.m1.2.2.1.3.3.1.cmml" xref="S3.E7.m1.2.2.1.3.3.1"></times><ci id="S3.E7.m1.2.2.1.3.3.2.cmml" xref="S3.E7.m1.2.2.1.3.3.2">𝑠</ci><ci id="S3.E7.m1.2.2.1.3.3.3.cmml" xref="S3.E7.m1.2.2.1.3.3.3">𝑐</ci><ci id="S3.E7.m1.2.2.1.3.3.4.cmml" xref="S3.E7.m1.2.2.1.3.3.4">𝑎</ci><ci id="S3.E7.m1.2.2.1.3.3.5.cmml" xref="S3.E7.m1.2.2.1.3.3.5">𝑙</ci><ci id="S3.E7.m1.2.2.1.3.3.6.cmml" xref="S3.E7.m1.2.2.1.3.3.6">𝑒</ci><ci id="S3.E7.m1.2.2.1.3.3.7.cmml" xref="S3.E7.m1.2.2.1.3.3.7">𝑑</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E7.m1.2c">\mathcal{V}_{min}={v\in\mathcal{V}:P(v|x_{1},x_{2},\dots,x_{t-1})\geq p_{scaled}}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(7)</span></td>
</tr></tbody>
</table>
</div>
<div id="S3.p4" class="ltx_para">
<p id="S3.p4.2" class="ltx_p">Finally, the next token <math id="S3.p4.1.m1.1" class="ltx_Math" alttext="\hat{x}_{t}" display="inline"><semantics id="S3.p4.1.m1.1a"><msub id="S3.p4.1.m1.1.1" xref="S3.p4.1.m1.1.1.cmml"><mover accent="true" id="S3.p4.1.m1.1.1.2" xref="S3.p4.1.m1.1.1.2.cmml"><mi id="S3.p4.1.m1.1.1.2.2" xref="S3.p4.1.m1.1.1.2.2.cmml">x</mi><mo id="S3.p4.1.m1.1.1.2.1" xref="S3.p4.1.m1.1.1.2.1.cmml">^</mo></mover><mi id="S3.p4.1.m1.1.1.3" xref="S3.p4.1.m1.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S3.p4.1.m1.1b"><apply id="S3.p4.1.m1.1.1.cmml" xref="S3.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.p4.1.m1.1.1.1.cmml" xref="S3.p4.1.m1.1.1">subscript</csymbol><apply id="S3.p4.1.m1.1.1.2.cmml" xref="S3.p4.1.m1.1.1.2"><ci id="S3.p4.1.m1.1.1.2.1.cmml" xref="S3.p4.1.m1.1.1.2.1">^</ci><ci id="S3.p4.1.m1.1.1.2.2.cmml" xref="S3.p4.1.m1.1.1.2.2">𝑥</ci></apply><ci id="S3.p4.1.m1.1.1.3.cmml" xref="S3.p4.1.m1.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.1.m1.1c">\hat{x}_{t}</annotation></semantics></math> is randomly sampled from the set <math id="S3.p4.2.m2.1" class="ltx_Math" alttext="\mathcal{V}_{min}" display="inline"><semantics id="S3.p4.2.m2.1a"><msub id="S3.p4.2.m2.1.1" xref="S3.p4.2.m2.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.p4.2.m2.1.1.2" xref="S3.p4.2.m2.1.1.2.cmml">𝒱</mi><mrow id="S3.p4.2.m2.1.1.3" xref="S3.p4.2.m2.1.1.3.cmml"><mi id="S3.p4.2.m2.1.1.3.2" xref="S3.p4.2.m2.1.1.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.p4.2.m2.1.1.3.1" xref="S3.p4.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p4.2.m2.1.1.3.3" xref="S3.p4.2.m2.1.1.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="S3.p4.2.m2.1.1.3.1a" xref="S3.p4.2.m2.1.1.3.1.cmml">​</mo><mi id="S3.p4.2.m2.1.1.3.4" xref="S3.p4.2.m2.1.1.3.4.cmml">n</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.p4.2.m2.1b"><apply id="S3.p4.2.m2.1.1.cmml" xref="S3.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.p4.2.m2.1.1.1.cmml" xref="S3.p4.2.m2.1.1">subscript</csymbol><ci id="S3.p4.2.m2.1.1.2.cmml" xref="S3.p4.2.m2.1.1.2">𝒱</ci><apply id="S3.p4.2.m2.1.1.3.cmml" xref="S3.p4.2.m2.1.1.3"><times id="S3.p4.2.m2.1.1.3.1.cmml" xref="S3.p4.2.m2.1.1.3.1"></times><ci id="S3.p4.2.m2.1.1.3.2.cmml" xref="S3.p4.2.m2.1.1.3.2">𝑚</ci><ci id="S3.p4.2.m2.1.1.3.3.cmml" xref="S3.p4.2.m2.1.1.3.3">𝑖</ci><ci id="S3.p4.2.m2.1.1.3.4.cmml" xref="S3.p4.2.m2.1.1.3.4">𝑛</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p4.2.m2.1c">\mathcal{V}_{min}</annotation></semantics></math> according to the normalized probabilities:</p>
<table id="S3.E8" class="ltx_equation ltx_eqn_table">

<tbody><tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S3.E8.m1.4" class="ltx_Math" alttext="\hat{x}_{t}\sim\frac{P(v|x_{1},\dots,x_{t-1})}{\sum_{v^{\prime}\in\mathcal{V}_{min}}P(v^{\prime}|x_{1},\dots,x_{t-1})}\text{ for }v\in\mathcal{V}_{min}" display="block"><semantics id="S3.E8.m1.4a"><mrow id="S3.E8.m1.4.5" xref="S3.E8.m1.4.5.cmml"><msub id="S3.E8.m1.4.5.2" xref="S3.E8.m1.4.5.2.cmml"><mover accent="true" id="S3.E8.m1.4.5.2.2" xref="S3.E8.m1.4.5.2.2.cmml"><mi id="S3.E8.m1.4.5.2.2.2" xref="S3.E8.m1.4.5.2.2.2.cmml">x</mi><mo id="S3.E8.m1.4.5.2.2.1" xref="S3.E8.m1.4.5.2.2.1.cmml">^</mo></mover><mi id="S3.E8.m1.4.5.2.3" xref="S3.E8.m1.4.5.2.3.cmml">t</mi></msub><mo id="S3.E8.m1.4.5.3" xref="S3.E8.m1.4.5.3.cmml">∼</mo><mrow id="S3.E8.m1.4.5.4" xref="S3.E8.m1.4.5.4.cmml"><mfrac id="S3.E8.m1.4.4" xref="S3.E8.m1.4.4.cmml"><mrow id="S3.E8.m1.2.2.2" xref="S3.E8.m1.2.2.2.cmml"><mi id="S3.E8.m1.2.2.2.4" xref="S3.E8.m1.2.2.2.4.cmml">P</mi><mo lspace="0em" rspace="0em" id="S3.E8.m1.2.2.2.3" xref="S3.E8.m1.2.2.2.3.cmml">​</mo><mrow id="S3.E8.m1.2.2.2.2.1" xref="S3.E8.m1.2.2.2.2.1.1.cmml"><mo stretchy="false" id="S3.E8.m1.2.2.2.2.1.2" xref="S3.E8.m1.2.2.2.2.1.1.cmml">(</mo><mrow id="S3.E8.m1.2.2.2.2.1.1" xref="S3.E8.m1.2.2.2.2.1.1.cmml"><mi id="S3.E8.m1.2.2.2.2.1.1.4" xref="S3.E8.m1.2.2.2.2.1.1.4.cmml">v</mi><mo fence="false" id="S3.E8.m1.2.2.2.2.1.1.3" xref="S3.E8.m1.2.2.2.2.1.1.3.cmml">|</mo><mrow id="S3.E8.m1.2.2.2.2.1.1.2.2" xref="S3.E8.m1.2.2.2.2.1.1.2.3.cmml"><msub id="S3.E8.m1.2.2.2.2.1.1.1.1.1" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1.cmml"><mi id="S3.E8.m1.2.2.2.2.1.1.1.1.1.2" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1.2.cmml">x</mi><mn id="S3.E8.m1.2.2.2.2.1.1.1.1.1.3" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S3.E8.m1.2.2.2.2.1.1.2.2.3" xref="S3.E8.m1.2.2.2.2.1.1.2.3.cmml">,</mo><mi mathvariant="normal" id="S3.E8.m1.1.1.1.1" xref="S3.E8.m1.1.1.1.1.cmml">…</mi><mo id="S3.E8.m1.2.2.2.2.1.1.2.2.4" xref="S3.E8.m1.2.2.2.2.1.1.2.3.cmml">,</mo><msub id="S3.E8.m1.2.2.2.2.1.1.2.2.2" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.cmml"><mi id="S3.E8.m1.2.2.2.2.1.1.2.2.2.2" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.2.cmml">x</mi><mrow id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.cmml"><mi id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.2" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.2.cmml">t</mi><mo id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.1" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.1.cmml">−</mo><mn id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.3" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S3.E8.m1.2.2.2.2.1.3" xref="S3.E8.m1.2.2.2.2.1.1.cmml">)</mo></mrow></mrow><mrow id="S3.E8.m1.4.4.4" xref="S3.E8.m1.4.4.4.cmml"><msub id="S3.E8.m1.4.4.4.3" xref="S3.E8.m1.4.4.4.3.cmml"><mo id="S3.E8.m1.4.4.4.3.2" xref="S3.E8.m1.4.4.4.3.2.cmml">∑</mo><mrow id="S3.E8.m1.4.4.4.3.3" xref="S3.E8.m1.4.4.4.3.3.cmml"><msup id="S3.E8.m1.4.4.4.3.3.2" xref="S3.E8.m1.4.4.4.3.3.2.cmml"><mi id="S3.E8.m1.4.4.4.3.3.2.2" xref="S3.E8.m1.4.4.4.3.3.2.2.cmml">v</mi><mo id="S3.E8.m1.4.4.4.3.3.2.3" xref="S3.E8.m1.4.4.4.3.3.2.3.cmml">′</mo></msup><mo id="S3.E8.m1.4.4.4.3.3.1" xref="S3.E8.m1.4.4.4.3.3.1.cmml">∈</mo><msub id="S3.E8.m1.4.4.4.3.3.3" xref="S3.E8.m1.4.4.4.3.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.E8.m1.4.4.4.3.3.3.2" xref="S3.E8.m1.4.4.4.3.3.3.2.cmml">𝒱</mi><mrow id="S3.E8.m1.4.4.4.3.3.3.3" xref="S3.E8.m1.4.4.4.3.3.3.3.cmml"><mi id="S3.E8.m1.4.4.4.3.3.3.3.2" xref="S3.E8.m1.4.4.4.3.3.3.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.4.4.3.3.3.3.1" xref="S3.E8.m1.4.4.4.3.3.3.3.1.cmml">​</mo><mi id="S3.E8.m1.4.4.4.3.3.3.3.3" xref="S3.E8.m1.4.4.4.3.3.3.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.4.4.3.3.3.3.1a" xref="S3.E8.m1.4.4.4.3.3.3.3.1.cmml">​</mo><mi id="S3.E8.m1.4.4.4.3.3.3.3.4" xref="S3.E8.m1.4.4.4.3.3.3.3.4.cmml">n</mi></mrow></msub></mrow></msub><mrow id="S3.E8.m1.4.4.4.2" xref="S3.E8.m1.4.4.4.2.cmml"><mi id="S3.E8.m1.4.4.4.2.3" xref="S3.E8.m1.4.4.4.2.3.cmml">P</mi><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.4.4.2.2" xref="S3.E8.m1.4.4.4.2.2.cmml">​</mo><mrow id="S3.E8.m1.4.4.4.2.1.1" xref="S3.E8.m1.4.4.4.2.1.1.1.cmml"><mo stretchy="false" id="S3.E8.m1.4.4.4.2.1.1.2" xref="S3.E8.m1.4.4.4.2.1.1.1.cmml">(</mo><mrow id="S3.E8.m1.4.4.4.2.1.1.1" xref="S3.E8.m1.4.4.4.2.1.1.1.cmml"><msup id="S3.E8.m1.4.4.4.2.1.1.1.4" xref="S3.E8.m1.4.4.4.2.1.1.1.4.cmml"><mi id="S3.E8.m1.4.4.4.2.1.1.1.4.2" xref="S3.E8.m1.4.4.4.2.1.1.1.4.2.cmml">v</mi><mo id="S3.E8.m1.4.4.4.2.1.1.1.4.3" xref="S3.E8.m1.4.4.4.2.1.1.1.4.3.cmml">′</mo></msup><mo fence="false" id="S3.E8.m1.4.4.4.2.1.1.1.3" xref="S3.E8.m1.4.4.4.2.1.1.1.3.cmml">|</mo><mrow id="S3.E8.m1.4.4.4.2.1.1.1.2.2" xref="S3.E8.m1.4.4.4.2.1.1.1.2.3.cmml"><msub id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.cmml"><mi id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.2" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.2.cmml">x</mi><mn id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.3" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.3.cmml">1</mn></msub><mo id="S3.E8.m1.4.4.4.2.1.1.1.2.2.3" xref="S3.E8.m1.4.4.4.2.1.1.1.2.3.cmml">,</mo><mi mathvariant="normal" id="S3.E8.m1.3.3.3.1" xref="S3.E8.m1.3.3.3.1.cmml">…</mi><mo id="S3.E8.m1.4.4.4.2.1.1.1.2.2.4" xref="S3.E8.m1.4.4.4.2.1.1.1.2.3.cmml">,</mo><msub id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.cmml"><mi id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.2" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.2.cmml">x</mi><mrow id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.cmml"><mi id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.2" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.2.cmml">t</mi><mo id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.1" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.1.cmml">−</mo><mn id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.3" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.3.cmml">1</mn></mrow></msub></mrow></mrow><mo stretchy="false" id="S3.E8.m1.4.4.4.2.1.1.3" xref="S3.E8.m1.4.4.4.2.1.1.1.cmml">)</mo></mrow></mrow></mrow></mfrac><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.5.4.1" xref="S3.E8.m1.4.5.4.1.cmml">​</mo><mtext id="S3.E8.m1.4.5.4.2" xref="S3.E8.m1.4.5.4.2a.cmml"> for </mtext><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.5.4.1a" xref="S3.E8.m1.4.5.4.1.cmml">​</mo><mi id="S3.E8.m1.4.5.4.3" xref="S3.E8.m1.4.5.4.3.cmml">v</mi></mrow><mo id="S3.E8.m1.4.5.5" xref="S3.E8.m1.4.5.5.cmml">∈</mo><msub id="S3.E8.m1.4.5.6" xref="S3.E8.m1.4.5.6.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.E8.m1.4.5.6.2" xref="S3.E8.m1.4.5.6.2.cmml">𝒱</mi><mrow id="S3.E8.m1.4.5.6.3" xref="S3.E8.m1.4.5.6.3.cmml"><mi id="S3.E8.m1.4.5.6.3.2" xref="S3.E8.m1.4.5.6.3.2.cmml">m</mi><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.5.6.3.1" xref="S3.E8.m1.4.5.6.3.1.cmml">​</mo><mi id="S3.E8.m1.4.5.6.3.3" xref="S3.E8.m1.4.5.6.3.3.cmml">i</mi><mo lspace="0em" rspace="0em" id="S3.E8.m1.4.5.6.3.1a" xref="S3.E8.m1.4.5.6.3.1.cmml">​</mo><mi id="S3.E8.m1.4.5.6.3.4" xref="S3.E8.m1.4.5.6.3.4.cmml">n</mi></mrow></msub></mrow><annotation-xml encoding="MathML-Content" id="S3.E8.m1.4b"><apply id="S3.E8.m1.4.5.cmml" xref="S3.E8.m1.4.5"><and id="S3.E8.m1.4.5a.cmml" xref="S3.E8.m1.4.5"></and><apply id="S3.E8.m1.4.5b.cmml" xref="S3.E8.m1.4.5"><csymbol cd="latexml" id="S3.E8.m1.4.5.3.cmml" xref="S3.E8.m1.4.5.3">similar-to</csymbol><apply id="S3.E8.m1.4.5.2.cmml" xref="S3.E8.m1.4.5.2"><csymbol cd="ambiguous" id="S3.E8.m1.4.5.2.1.cmml" xref="S3.E8.m1.4.5.2">subscript</csymbol><apply id="S3.E8.m1.4.5.2.2.cmml" xref="S3.E8.m1.4.5.2.2"><ci id="S3.E8.m1.4.5.2.2.1.cmml" xref="S3.E8.m1.4.5.2.2.1">^</ci><ci id="S3.E8.m1.4.5.2.2.2.cmml" xref="S3.E8.m1.4.5.2.2.2">𝑥</ci></apply><ci id="S3.E8.m1.4.5.2.3.cmml" xref="S3.E8.m1.4.5.2.3">𝑡</ci></apply><apply id="S3.E8.m1.4.5.4.cmml" xref="S3.E8.m1.4.5.4"><times id="S3.E8.m1.4.5.4.1.cmml" xref="S3.E8.m1.4.5.4.1"></times><apply id="S3.E8.m1.4.4.cmml" xref="S3.E8.m1.4.4"><divide id="S3.E8.m1.4.4.5.cmml" xref="S3.E8.m1.4.4"></divide><apply id="S3.E8.m1.2.2.2.cmml" xref="S3.E8.m1.2.2.2"><times id="S3.E8.m1.2.2.2.3.cmml" xref="S3.E8.m1.2.2.2.3"></times><ci id="S3.E8.m1.2.2.2.4.cmml" xref="S3.E8.m1.2.2.2.4">𝑃</ci><apply id="S3.E8.m1.2.2.2.2.1.1.cmml" xref="S3.E8.m1.2.2.2.2.1"><csymbol cd="latexml" id="S3.E8.m1.2.2.2.2.1.1.3.cmml" xref="S3.E8.m1.2.2.2.2.1.1.3">conditional</csymbol><ci id="S3.E8.m1.2.2.2.2.1.1.4.cmml" xref="S3.E8.m1.2.2.2.2.1.1.4">𝑣</ci><list id="S3.E8.m1.2.2.2.2.1.1.2.3.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2"><apply id="S3.E8.m1.2.2.2.2.1.1.1.1.1.cmml" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E8.m1.2.2.2.2.1.1.1.1.1.1.cmml" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1">subscript</csymbol><ci id="S3.E8.m1.2.2.2.2.1.1.1.1.1.2.cmml" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S3.E8.m1.2.2.2.2.1.1.1.1.1.3.cmml" xref="S3.E8.m1.2.2.2.2.1.1.1.1.1.3">1</cn></apply><ci id="S3.E8.m1.1.1.1.1.cmml" xref="S3.E8.m1.1.1.1.1">…</ci><apply id="S3.E8.m1.2.2.2.2.1.1.2.2.2.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2"><csymbol cd="ambiguous" id="S3.E8.m1.2.2.2.2.1.1.2.2.2.1.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2">subscript</csymbol><ci id="S3.E8.m1.2.2.2.2.1.1.2.2.2.2.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.2">𝑥</ci><apply id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3"><minus id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.1.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.1"></minus><ci id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.2.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.2">𝑡</ci><cn type="integer" id="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.3.cmml" xref="S3.E8.m1.2.2.2.2.1.1.2.2.2.3.3">1</cn></apply></apply></list></apply></apply><apply id="S3.E8.m1.4.4.4.cmml" xref="S3.E8.m1.4.4.4"><apply id="S3.E8.m1.4.4.4.3.cmml" xref="S3.E8.m1.4.4.4.3"><csymbol cd="ambiguous" id="S3.E8.m1.4.4.4.3.1.cmml" xref="S3.E8.m1.4.4.4.3">subscript</csymbol><sum id="S3.E8.m1.4.4.4.3.2.cmml" xref="S3.E8.m1.4.4.4.3.2"></sum><apply id="S3.E8.m1.4.4.4.3.3.cmml" xref="S3.E8.m1.4.4.4.3.3"><in id="S3.E8.m1.4.4.4.3.3.1.cmml" xref="S3.E8.m1.4.4.4.3.3.1"></in><apply id="S3.E8.m1.4.4.4.3.3.2.cmml" xref="S3.E8.m1.4.4.4.3.3.2"><csymbol cd="ambiguous" id="S3.E8.m1.4.4.4.3.3.2.1.cmml" xref="S3.E8.m1.4.4.4.3.3.2">superscript</csymbol><ci id="S3.E8.m1.4.4.4.3.3.2.2.cmml" xref="S3.E8.m1.4.4.4.3.3.2.2">𝑣</ci><ci id="S3.E8.m1.4.4.4.3.3.2.3.cmml" xref="S3.E8.m1.4.4.4.3.3.2.3">′</ci></apply><apply id="S3.E8.m1.4.4.4.3.3.3.cmml" xref="S3.E8.m1.4.4.4.3.3.3"><csymbol cd="ambiguous" id="S3.E8.m1.4.4.4.3.3.3.1.cmml" xref="S3.E8.m1.4.4.4.3.3.3">subscript</csymbol><ci id="S3.E8.m1.4.4.4.3.3.3.2.cmml" xref="S3.E8.m1.4.4.4.3.3.3.2">𝒱</ci><apply id="S3.E8.m1.4.4.4.3.3.3.3.cmml" xref="S3.E8.m1.4.4.4.3.3.3.3"><times id="S3.E8.m1.4.4.4.3.3.3.3.1.cmml" xref="S3.E8.m1.4.4.4.3.3.3.3.1"></times><ci id="S3.E8.m1.4.4.4.3.3.3.3.2.cmml" xref="S3.E8.m1.4.4.4.3.3.3.3.2">𝑚</ci><ci id="S3.E8.m1.4.4.4.3.3.3.3.3.cmml" xref="S3.E8.m1.4.4.4.3.3.3.3.3">𝑖</ci><ci id="S3.E8.m1.4.4.4.3.3.3.3.4.cmml" xref="S3.E8.m1.4.4.4.3.3.3.3.4">𝑛</ci></apply></apply></apply></apply><apply id="S3.E8.m1.4.4.4.2.cmml" xref="S3.E8.m1.4.4.4.2"><times id="S3.E8.m1.4.4.4.2.2.cmml" xref="S3.E8.m1.4.4.4.2.2"></times><ci id="S3.E8.m1.4.4.4.2.3.cmml" xref="S3.E8.m1.4.4.4.2.3">𝑃</ci><apply id="S3.E8.m1.4.4.4.2.1.1.1.cmml" xref="S3.E8.m1.4.4.4.2.1.1"><csymbol cd="latexml" id="S3.E8.m1.4.4.4.2.1.1.1.3.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.3">conditional</csymbol><apply id="S3.E8.m1.4.4.4.2.1.1.1.4.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.4"><csymbol cd="ambiguous" id="S3.E8.m1.4.4.4.2.1.1.1.4.1.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.4">superscript</csymbol><ci id="S3.E8.m1.4.4.4.2.1.1.1.4.2.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.4.2">𝑣</ci><ci id="S3.E8.m1.4.4.4.2.1.1.1.4.3.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.4.3">′</ci></apply><list id="S3.E8.m1.4.4.4.2.1.1.1.2.3.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2"><apply id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1"><csymbol cd="ambiguous" id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.1.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1">subscript</csymbol><ci id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.2.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.2">𝑥</ci><cn type="integer" id="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.3.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.1.1.1.3">1</cn></apply><ci id="S3.E8.m1.3.3.3.1.cmml" xref="S3.E8.m1.3.3.3.1">…</ci><apply id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2"><csymbol cd="ambiguous" id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.1.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2">subscript</csymbol><ci id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.2.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.2">𝑥</ci><apply id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3"><minus id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.1.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.1"></minus><ci id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.2.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.2">𝑡</ci><cn type="integer" id="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.3.cmml" xref="S3.E8.m1.4.4.4.2.1.1.1.2.2.2.3.3">1</cn></apply></apply></list></apply></apply></apply></apply><ci id="S3.E8.m1.4.5.4.2a.cmml" xref="S3.E8.m1.4.5.4.2"><mtext id="S3.E8.m1.4.5.4.2.cmml" xref="S3.E8.m1.4.5.4.2"> for </mtext></ci><ci id="S3.E8.m1.4.5.4.3.cmml" xref="S3.E8.m1.4.5.4.3">𝑣</ci></apply></apply><apply id="S3.E8.m1.4.5c.cmml" xref="S3.E8.m1.4.5"><in id="S3.E8.m1.4.5.5.cmml" xref="S3.E8.m1.4.5.5"></in><share href="#S3.E8.m1.4.5.4.cmml" id="S3.E8.m1.4.5d.cmml" xref="S3.E8.m1.4.5"></share><apply id="S3.E8.m1.4.5.6.cmml" xref="S3.E8.m1.4.5.6"><csymbol cd="ambiguous" id="S3.E8.m1.4.5.6.1.cmml" xref="S3.E8.m1.4.5.6">subscript</csymbol><ci id="S3.E8.m1.4.5.6.2.cmml" xref="S3.E8.m1.4.5.6.2">𝒱</ci><apply id="S3.E8.m1.4.5.6.3.cmml" xref="S3.E8.m1.4.5.6.3"><times id="S3.E8.m1.4.5.6.3.1.cmml" xref="S3.E8.m1.4.5.6.3.1"></times><ci id="S3.E8.m1.4.5.6.3.2.cmml" xref="S3.E8.m1.4.5.6.3.2">𝑚</ci><ci id="S3.E8.m1.4.5.6.3.3.cmml" xref="S3.E8.m1.4.5.6.3.3">𝑖</ci><ci id="S3.E8.m1.4.5.6.3.4.cmml" xref="S3.E8.m1.4.5.6.3.4">𝑛</ci></apply></apply></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.E8.m1.4c">\hat{x}_{t}\sim\frac{P(v|x_{1},\dots,x_{t-1})}{\sum_{v^{\prime}\in\mathcal{V}_{min}}P(v^{\prime}|x_{1},\dots,x_{t-1})}\text{ for }v\in\mathcal{V}_{min}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
<td rowspan="1" class="ltx_eqn_cell ltx_eqn_eqno ltx_align_middle ltx_align_right"><span class="ltx_tag ltx_tag_equation ltx_align_right">(8)</span></td>
</tr></tbody>
</table>
</div>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Case Study</h2>

<div id="S4.p1" class="ltx_para">
<p id="S4.p1.2" class="ltx_p">In this section, we inspect the behavior of min-<math id="S4.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.p1.1.m1.1a"><mi id="S4.p1.1.m1.1.1" xref="S4.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.p1.1.m1.1b"><ci id="S4.p1.1.m1.1.1.cmml" xref="S4.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p1.1.m1.1c">p</annotation></semantics></math> with top-<math id="S4.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.p1.2.m2.1a"><mi id="S4.p1.2.m2.1.1" xref="S4.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.p1.2.m2.1b"><ci id="S4.p1.2.m2.1.1.cmml" xref="S4.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p1.2.m2.1c">p</annotation></semantics></math> when both methods are used in conjunction with temperature. Briefly:</p>
<ul id="S4.I1" class="ltx_itemize">
<li id="S4.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i1.p1" class="ltx_para">
<p id="S4.I1.i1.p1.1" class="ltx_p"><span id="S4.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">High-confidence predictions:</span> When the model assigns high probability to a particular next token, min-<math id="S4.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.I1.i1.p1.1.m1.1a"><mi id="S4.I1.i1.p1.1.m1.1.1" xref="S4.I1.i1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.I1.i1.p1.1.m1.1b"><ci id="S4.I1.i1.p1.1.m1.1.1.cmml" xref="S4.I1.i1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.I1.i1.p1.1.m1.1c">p</annotation></semantics></math> filters out low-probability alternatives. This preserves coherence by avoiding sampling from the "unreliable tail" that could derail generation.</p>
</div>
</li>
<li id="S4.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i2.p1" class="ltx_para">
<p id="S4.I1.i2.p1.1" class="ltx_p"><span id="S4.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Low-confidence predictions:</span> When there is no clear front-runner, min-<math id="S4.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.I1.i2.p1.1.m1.1a"><mi id="S4.I1.i2.p1.1.m1.1.1" xref="S4.I1.i2.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.I1.i2.p1.1.m1.1b"><ci id="S4.I1.i2.p1.1.m1.1.1.cmml" xref="S4.I1.i2.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.I1.i2.p1.1.m1.1c">p</annotation></semantics></math> relaxes its filter. This allows the model to sample from a more diverse set of plausible continuations, enabling creativity in open-ended settings.</p>
</div>
</li>
</ul>
</div>
<div id="S4.p2" class="ltx_para">
<p id="S4.p2.1" class="ltx_p">We illustrate these differences in the following two examples<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>Interactive demo available at <a target="_blank" href="https://artefact2.github.io/llm-sampling/index.xhtml" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://artefact2.github.io/llm-sampling/index.xhtml</a>. Demo not affliated with authors.</span></span></span>. Table <a href="#S4.T1" title="Table 1 ‣ 4.2 Case 2: High-Certainty Next Token ‣ 4 Case Study ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> shows a case where the next token probabilities are relatively low, while Table <a href="#S4.T2" title="Table 2 ‣ 4.2 Case 2: High-Certainty Next Token ‣ 4 Case Study ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> shows a case with a clear high probability next token.</p>
</div>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Case 1: Low-Certainty Next Token</h3>

<div id="S4.SS1.p1" class="ltx_para">
<p id="S4.SS1.p1.5" class="ltx_p">Table <a href="#S4.T1" title="Table 1 ‣ 4.2 Case 2: High-Certainty Next Token ‣ 4 Case Study ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> shows the token probabilities for a creative writing prompt where there are many plausible ways to continue the story. At temperature <math id="S4.SS1.p1.1.m1.1" class="ltx_Math" alttext="\tau=1" display="inline"><semantics id="S4.SS1.p1.1.m1.1a"><mrow id="S4.SS1.p1.1.m1.1.1" xref="S4.SS1.p1.1.m1.1.1.cmml"><mi id="S4.SS1.p1.1.m1.1.1.2" xref="S4.SS1.p1.1.m1.1.1.2.cmml">τ</mi><mo id="S4.SS1.p1.1.m1.1.1.1" xref="S4.SS1.p1.1.m1.1.1.1.cmml">=</mo><mn id="S4.SS1.p1.1.m1.1.1.3" xref="S4.SS1.p1.1.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p1.1.m1.1b"><apply id="S4.SS1.p1.1.m1.1.1.cmml" xref="S4.SS1.p1.1.m1.1.1"><eq id="S4.SS1.p1.1.m1.1.1.1.cmml" xref="S4.SS1.p1.1.m1.1.1.1"></eq><ci id="S4.SS1.p1.1.m1.1.1.2.cmml" xref="S4.SS1.p1.1.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.SS1.p1.1.m1.1.1.3.cmml" xref="S4.SS1.p1.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p1.1.m1.1c">\tau=1</annotation></semantics></math>, the highest probability token "was" only has a 70.3% probability. When the temperature is increased to <math id="S4.SS1.p1.2.m2.1" class="ltx_Math" alttext="\tau=3" display="inline"><semantics id="S4.SS1.p1.2.m2.1a"><mrow id="S4.SS1.p1.2.m2.1.1" xref="S4.SS1.p1.2.m2.1.1.cmml"><mi id="S4.SS1.p1.2.m2.1.1.2" xref="S4.SS1.p1.2.m2.1.1.2.cmml">τ</mi><mo id="S4.SS1.p1.2.m2.1.1.1" xref="S4.SS1.p1.2.m2.1.1.1.cmml">=</mo><mn id="S4.SS1.p1.2.m2.1.1.3" xref="S4.SS1.p1.2.m2.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.SS1.p1.2.m2.1b"><apply id="S4.SS1.p1.2.m2.1.1.cmml" xref="S4.SS1.p1.2.m2.1.1"><eq id="S4.SS1.p1.2.m2.1.1.1.cmml" xref="S4.SS1.p1.2.m2.1.1.1"></eq><ci id="S4.SS1.p1.2.m2.1.1.2.cmml" xref="S4.SS1.p1.2.m2.1.1.2">𝜏</ci><cn type="integer" id="S4.SS1.p1.2.m2.1.1.3.cmml" xref="S4.SS1.p1.2.m2.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p1.2.m2.1c">\tau=3</annotation></semantics></math>, this drops to 11.9% as probability mass shifts to other candidate tokens. In this low-certainty case, applying top-<math id="S4.SS1.p1.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS1.p1.3.m3.1a"><mi id="S4.SS1.p1.3.m3.1.1" xref="S4.SS1.p1.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p1.3.m3.1b"><ci id="S4.SS1.p1.3.m3.1.1.cmml" xref="S4.SS1.p1.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p1.3.m3.1c">p</annotation></semantics></math> and min-<math id="S4.SS1.p1.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS1.p1.4.m4.1a"><mi id="S4.SS1.p1.4.m4.1.1" xref="S4.SS1.p1.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p1.4.m4.1b"><ci id="S4.SS1.p1.4.m4.1.1.cmml" xref="S4.SS1.p1.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p1.4.m4.1c">p</annotation></semantics></math> to the post-temperature distribution yields similar results, as there is no clear top token for min-<math id="S4.SS1.p1.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS1.p1.5.m5.1a"><mi id="S4.SS1.p1.5.m5.1.1" xref="S4.SS1.p1.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS1.p1.5.m5.1b"><ci id="S4.SS1.p1.5.m5.1.1.cmml" xref="S4.SS1.p1.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS1.p1.5.m5.1c">p</annotation></semantics></math> to prioritize.</p>
</div>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Case 2: High-Certainty Next Token</h3>

<div id="S4.SS2.p1" class="ltx_para">
<p id="S4.SS2.p1.2" class="ltx_p">In contrast, Table <a href="#S4.T2" title="Table 2 ‣ 4.2 Case 2: High-Certainty Next Token ‣ 4 Case Study ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> shows a case where using min-<math id="S4.SS2.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS2.p1.1.m1.1a"><mi id="S4.SS2.p1.1.m1.1.1" xref="S4.SS2.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.p1.1.m1.1b"><ci id="S4.SS2.p1.1.m1.1.1.cmml" xref="S4.SS2.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p1.1.m1.1c">p</annotation></semantics></math> has a clear impact. The prompt is a factual statement about rainbows, with "light"being the obvious next token with 98.25% probability when temperature <math id="S4.SS2.p1.2.m2.1" class="ltx_Math" alttext="\tau=1" display="inline"><semantics id="S4.SS2.p1.2.m2.1a"><mrow id="S4.SS2.p1.2.m2.1.1" xref="S4.SS2.p1.2.m2.1.1.cmml"><mi id="S4.SS2.p1.2.m2.1.1.2" xref="S4.SS2.p1.2.m2.1.1.2.cmml">τ</mi><mo id="S4.SS2.p1.2.m2.1.1.1" xref="S4.SS2.p1.2.m2.1.1.1.cmml">=</mo><mn id="S4.SS2.p1.2.m2.1.1.3" xref="S4.SS2.p1.2.m2.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.SS2.p1.2.m2.1b"><apply id="S4.SS2.p1.2.m2.1.1.cmml" xref="S4.SS2.p1.2.m2.1.1"><eq id="S4.SS2.p1.2.m2.1.1.1.cmml" xref="S4.SS2.p1.2.m2.1.1.1"></eq><ci id="S4.SS2.p1.2.m2.1.1.2.cmml" xref="S4.SS2.p1.2.m2.1.1.2">𝜏</ci><cn type="integer" id="S4.SS2.p1.2.m2.1.1.3.cmml" xref="S4.SS2.p1.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p1.2.m2.1c">\tau=1</annotation></semantics></math>.</p>
</div>
<div id="S4.SS2.p2" class="ltx_para">
<p id="S4.SS2.p2.2" class="ltx_p">Raising the temperature to <math id="S4.SS2.p2.1.m1.1" class="ltx_Math" alttext="\tau=3" display="inline"><semantics id="S4.SS2.p2.1.m1.1a"><mrow id="S4.SS2.p2.1.m1.1.1" xref="S4.SS2.p2.1.m1.1.1.cmml"><mi id="S4.SS2.p2.1.m1.1.1.2" xref="S4.SS2.p2.1.m1.1.1.2.cmml">τ</mi><mo id="S4.SS2.p2.1.m1.1.1.1" xref="S4.SS2.p2.1.m1.1.1.1.cmml">=</mo><mn id="S4.SS2.p2.1.m1.1.1.3" xref="S4.SS2.p2.1.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.SS2.p2.1.m1.1b"><apply id="S4.SS2.p2.1.m1.1.1.cmml" xref="S4.SS2.p2.1.m1.1.1"><eq id="S4.SS2.p2.1.m1.1.1.1.cmml" xref="S4.SS2.p2.1.m1.1.1.1"></eq><ci id="S4.SS2.p2.1.m1.1.1.2.cmml" xref="S4.SS2.p2.1.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.SS2.p2.1.m1.1.1.3.cmml" xref="S4.SS2.p2.1.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p2.1.m1.1c">\tau=3</annotation></semantics></math> flattens out the probability distribution, but "light" remains dominant at 34.41%. When applying top-<math id="S4.SS2.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS2.p2.2.m2.1a"><mi id="S4.SS2.p2.2.m2.1.1" xref="S4.SS2.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.p2.2.m2.1b"><ci id="S4.SS2.p2.2.m2.1.1.cmml" xref="S4.SS2.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p2.2.m2.1c">p</annotation></semantics></math>, several additional tokens like "water", "sunshine", "a" still remain in the sampling pool at ~2-3% probability each. While these are vaguely plausible, they are much lower quality continuations compared to "light".</p>
</div>
<div id="S4.SS2.p3" class="ltx_para">
<p id="S4.SS2.p3.3" class="ltx_p">Min-<math id="S4.SS2.p3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS2.p3.1.m1.1a"><mi id="S4.SS2.p3.1.m1.1.1" xref="S4.SS2.p3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.p3.1.m1.1b"><ci id="S4.SS2.p3.1.m1.1.1.cmml" xref="S4.SS2.p3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p3.1.m1.1c">p</annotation></semantics></math>, on the other hand, truncates much more aggressively. Setting a 10% base probability threshold when "light" has 34.41% probability leads min-<math id="S4.SS2.p3.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS2.p3.2.m2.1a"><mi id="S4.SS2.p3.2.m2.1.1" xref="S4.SS2.p3.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.p3.2.m2.1b"><ci id="S4.SS2.p3.2.m2.1.1.cmml" xref="S4.SS2.p3.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p3.2.m2.1c">p</annotation></semantics></math> to only include "light" and "sunlight" in its sampling pool, as all other tokens have less than the scaled 3.441 percent probability threshold. This showcases min-<math id="S4.SS2.p3.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS2.p3.3.m3.1a"><mi id="S4.SS2.p3.3.m3.1.1" xref="S4.SS2.p3.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS2.p3.3.m3.1b"><ci id="S4.SS2.p3.3.m3.1.1.cmml" xref="S4.SS2.p3.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS2.p3.3.m3.1c">p</annotation></semantics></math>’s ability to maintain coherence at high temperatures by calibrating its cutoff to the strength of the top candidate.</p>
</div>
<figure id="S4.T1" class="ltx_table">
<table id="S4.T1.8" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S4.T1.8.9.1" class="ltx_tr" style="background-color:#DFDFDF;">
<th id="S4.T1.8.9.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_l ltx_border_r ltx_border_t" colspan="5">
<span id="S4.T1.8.9.1.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#DFDFDF;">
<span id="S4.T1.8.9.1.1.1.1" class="ltx_p" style="width:209.1pt;"><span id="S4.T1.8.9.1.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#DFDFDF;">Prompt</span></span>
</span>
</th>
</tr>
<tr id="S4.T1.8.10.2" class="ltx_tr" style="background-color:#DFDFDF;">
<th id="S4.T1.8.10.2.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_l ltx_border_r" colspan="5">
<span id="S4.T1.8.10.2.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#DFDFDF;">
<span id="S4.T1.8.10.2.1.1.1" class="ltx_p" style="width:209.1pt;">"You will pay for what you have done," she hissed, her blade flashing in the moonlight. The battle that ensued _____</span>
</span>
</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S4.T1.2.2" class="ltx_tr">
<th id="S4.T1.2.2.3" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_t">
<span id="S4.T1.2.2.3.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T1.2.2.3.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.2.2.3.1.1.1" class="ltx_text ltx_font_bold">Sampling</span></span>
</span>
</th>
<td id="S4.T1.2.2.4" class="ltx_td ltx_align_center ltx_border_t" colspan="2"><span id="S4.T1.2.2.4.1" class="ltx_text ltx_font_bold">Temperature</span></td>
<td id="S4.T1.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.1.1" class="ltx_text ltx_font_bold">+top-<math id="S4.T1.1.1.1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T1.1.1.1.1.m1.1a"><mi id="S4.T1.1.1.1.1.m1.1.1" xref="S4.T1.1.1.1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T1.1.1.1.1.m1.1b"><ci id="S4.T1.1.1.1.1.m1.1.1.cmml" xref="S4.T1.1.1.1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.1.1.1.1.m1.1c">p</annotation></semantics></math></span></td>
<td id="S4.T1.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.2.2.2.1" class="ltx_text ltx_font_bold">+min-<math id="S4.T1.2.2.2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T1.2.2.2.1.m1.1a"><mi id="S4.T1.2.2.2.1.m1.1.1" xref="S4.T1.2.2.2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T1.2.2.2.1.m1.1b"><ci id="S4.T1.2.2.2.1.m1.1.1.cmml" xref="S4.T1.2.2.2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.2.2.2.1.m1.1c">p</annotation></semantics></math></span></td>
</tr>
<tr id="S4.T1.6.6" class="ltx_tr">
<th id="S4.T1.6.6.5" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T1.6.6.5.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T1.6.6.5.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.6.6.5.1.1.1" class="ltx_text ltx_font_bold">Method</span></span>
</span>
</th>
<td id="S4.T1.3.3.1" class="ltx_td ltx_align_center"><math id="S4.T1.3.3.1.m1.1" class="ltx_Math" alttext="\tau\!=\!1" display="inline"><semantics id="S4.T1.3.3.1.m1.1a"><mrow id="S4.T1.3.3.1.m1.1.1" xref="S4.T1.3.3.1.m1.1.1.cmml"><mi id="S4.T1.3.3.1.m1.1.1.2" xref="S4.T1.3.3.1.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T1.3.3.1.m1.1.1.1" xref="S4.T1.3.3.1.m1.1.1.1.cmml">=</mo><mn id="S4.T1.3.3.1.m1.1.1.3" xref="S4.T1.3.3.1.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T1.3.3.1.m1.1b"><apply id="S4.T1.3.3.1.m1.1.1.cmml" xref="S4.T1.3.3.1.m1.1.1"><eq id="S4.T1.3.3.1.m1.1.1.1.cmml" xref="S4.T1.3.3.1.m1.1.1.1"></eq><ci id="S4.T1.3.3.1.m1.1.1.2.cmml" xref="S4.T1.3.3.1.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T1.3.3.1.m1.1.1.3.cmml" xref="S4.T1.3.3.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.3.3.1.m1.1c">\tau\!=\!1</annotation></semantics></math></td>
<td id="S4.T1.4.4.2" class="ltx_td ltx_align_center"><math id="S4.T1.4.4.2.m1.1" class="ltx_Math" alttext="\tau\!=\!3" display="inline"><semantics id="S4.T1.4.4.2.m1.1a"><mrow id="S4.T1.4.4.2.m1.1.1" xref="S4.T1.4.4.2.m1.1.1.cmml"><mi id="S4.T1.4.4.2.m1.1.1.2" xref="S4.T1.4.4.2.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T1.4.4.2.m1.1.1.1" xref="S4.T1.4.4.2.m1.1.1.1.cmml">=</mo><mn id="S4.T1.4.4.2.m1.1.1.3" xref="S4.T1.4.4.2.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T1.4.4.2.m1.1b"><apply id="S4.T1.4.4.2.m1.1.1.cmml" xref="S4.T1.4.4.2.m1.1.1"><eq id="S4.T1.4.4.2.m1.1.1.1.cmml" xref="S4.T1.4.4.2.m1.1.1.1"></eq><ci id="S4.T1.4.4.2.m1.1.1.2.cmml" xref="S4.T1.4.4.2.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T1.4.4.2.m1.1.1.3.cmml" xref="S4.T1.4.4.2.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.4.4.2.m1.1c">\tau\!=\!3</annotation></semantics></math></td>
<td id="S4.T1.5.5.3" class="ltx_td ltx_align_center"><math id="S4.T1.5.5.3.m1.1" class="ltx_Math" alttext="\tau\!=\!3" display="inline"><semantics id="S4.T1.5.5.3.m1.1a"><mrow id="S4.T1.5.5.3.m1.1.1" xref="S4.T1.5.5.3.m1.1.1.cmml"><mi id="S4.T1.5.5.3.m1.1.1.2" xref="S4.T1.5.5.3.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T1.5.5.3.m1.1.1.1" xref="S4.T1.5.5.3.m1.1.1.1.cmml">=</mo><mn id="S4.T1.5.5.3.m1.1.1.3" xref="S4.T1.5.5.3.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T1.5.5.3.m1.1b"><apply id="S4.T1.5.5.3.m1.1.1.cmml" xref="S4.T1.5.5.3.m1.1.1"><eq id="S4.T1.5.5.3.m1.1.1.1.cmml" xref="S4.T1.5.5.3.m1.1.1.1"></eq><ci id="S4.T1.5.5.3.m1.1.1.2.cmml" xref="S4.T1.5.5.3.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T1.5.5.3.m1.1.1.3.cmml" xref="S4.T1.5.5.3.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.5.5.3.m1.1c">\tau\!=\!3</annotation></semantics></math></td>
<td id="S4.T1.6.6.4" class="ltx_td ltx_align_center"><math id="S4.T1.6.6.4.m1.1" class="ltx_Math" alttext="\tau\!=\!3" display="inline"><semantics id="S4.T1.6.6.4.m1.1a"><mrow id="S4.T1.6.6.4.m1.1.1" xref="S4.T1.6.6.4.m1.1.1.cmml"><mi id="S4.T1.6.6.4.m1.1.1.2" xref="S4.T1.6.6.4.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T1.6.6.4.m1.1.1.1" xref="S4.T1.6.6.4.m1.1.1.1.cmml">=</mo><mn id="S4.T1.6.6.4.m1.1.1.3" xref="S4.T1.6.6.4.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T1.6.6.4.m1.1b"><apply id="S4.T1.6.6.4.m1.1.1.cmml" xref="S4.T1.6.6.4.m1.1.1"><eq id="S4.T1.6.6.4.m1.1.1.1.cmml" xref="S4.T1.6.6.4.m1.1.1.1"></eq><ci id="S4.T1.6.6.4.m1.1.1.2.cmml" xref="S4.T1.6.6.4.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T1.6.6.4.m1.1.1.3.cmml" xref="S4.T1.6.6.4.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.6.6.4.m1.1c">\tau\!=\!3</annotation></semantics></math></td>
</tr>
<tr id="S4.T1.8.8" class="ltx_tr">
<th id="S4.T1.8.8.3" class="ltx_td ltx_align_top ltx_th ltx_th_row"></th>
<td id="S4.T1.8.8.4" class="ltx_td"></td>
<td id="S4.T1.8.8.5" class="ltx_td"></td>
<td id="S4.T1.7.7.1" class="ltx_td ltx_align_center"><math id="S4.T1.7.7.1.m1.1" class="ltx_Math" alttext="p\!=\!0.9" display="inline"><semantics id="S4.T1.7.7.1.m1.1a"><mrow id="S4.T1.7.7.1.m1.1.1" xref="S4.T1.7.7.1.m1.1.1.cmml"><mi id="S4.T1.7.7.1.m1.1.1.2" xref="S4.T1.7.7.1.m1.1.1.2.cmml">p</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T1.7.7.1.m1.1.1.1" xref="S4.T1.7.7.1.m1.1.1.1.cmml">=</mo><mn id="S4.T1.7.7.1.m1.1.1.3" xref="S4.T1.7.7.1.m1.1.1.3.cmml">0.9</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T1.7.7.1.m1.1b"><apply id="S4.T1.7.7.1.m1.1.1.cmml" xref="S4.T1.7.7.1.m1.1.1"><eq id="S4.T1.7.7.1.m1.1.1.1.cmml" xref="S4.T1.7.7.1.m1.1.1.1"></eq><ci id="S4.T1.7.7.1.m1.1.1.2.cmml" xref="S4.T1.7.7.1.m1.1.1.2">𝑝</ci><cn type="float" id="S4.T1.7.7.1.m1.1.1.3.cmml" xref="S4.T1.7.7.1.m1.1.1.3">0.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.7.7.1.m1.1c">p\!=\!0.9</annotation></semantics></math></td>
<td id="S4.T1.8.8.2" class="ltx_td ltx_align_center"><math id="S4.T1.8.8.2.m1.1" class="ltx_Math" alttext="p\!=\!0.1" display="inline"><semantics id="S4.T1.8.8.2.m1.1a"><mrow id="S4.T1.8.8.2.m1.1.1" xref="S4.T1.8.8.2.m1.1.1.cmml"><mi id="S4.T1.8.8.2.m1.1.1.2" xref="S4.T1.8.8.2.m1.1.1.2.cmml">p</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T1.8.8.2.m1.1.1.1" xref="S4.T1.8.8.2.m1.1.1.1.cmml">=</mo><mn id="S4.T1.8.8.2.m1.1.1.3" xref="S4.T1.8.8.2.m1.1.1.3.cmml">0.1</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T1.8.8.2.m1.1b"><apply id="S4.T1.8.8.2.m1.1.1.cmml" xref="S4.T1.8.8.2.m1.1.1"><eq id="S4.T1.8.8.2.m1.1.1.1.cmml" xref="S4.T1.8.8.2.m1.1.1.1"></eq><ci id="S4.T1.8.8.2.m1.1.1.2.cmml" xref="S4.T1.8.8.2.m1.1.1.2">𝑝</ci><cn type="float" id="S4.T1.8.8.2.m1.1.1.3.cmml" xref="S4.T1.8.8.2.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.8.8.2.m1.1c">p\!=\!0.1</annotation></semantics></math></td>
</tr>
<tr id="S4.T1.8.11.1" class="ltx_tr">
<th id="S4.T1.8.11.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_t">
<span id="S4.T1.8.11.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T1.8.11.1.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.11.1.1.1.1.1" class="ltx_text ltx_font_bold">was</span></span>
</span>
</th>
<td id="S4.T1.8.11.1.2" class="ltx_td ltx_align_center ltx_border_t">70.3</td>
<td id="S4.T1.8.11.1.3" class="ltx_td ltx_align_center ltx_border_t">11.9</td>
<td id="S4.T1.8.11.1.4" class="ltx_td ltx_align_center ltx_border_t">13.1</td>
<td id="S4.T1.8.11.1.5" class="ltx_td ltx_align_center ltx_border_t">18.5</td>
</tr>
<tr id="S4.T1.8.12.2" class="ltx_tr" style="background-color:#F2F2F2;">
<th id="S4.T1.8.12.2.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T1.8.12.2.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#F2F2F2;">
<span id="S4.T1.8.12.2.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.12.2.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">lasted</span></span>
</span>
</th>
<td id="S4.T1.8.12.2.2" class="ltx_td ltx_align_center"><span id="S4.T1.8.12.2.2.1" class="ltx_text" style="background-color:#F2F2F2;">9.5</span></td>
<td id="S4.T1.8.12.2.3" class="ltx_td ltx_align_center"><span id="S4.T1.8.12.2.3.1" class="ltx_text" style="background-color:#F2F2F2;">6.1</span></td>
<td id="S4.T1.8.12.2.4" class="ltx_td ltx_align_center"><span id="S4.T1.8.12.2.4.1" class="ltx_text" style="background-color:#F2F2F2;">6.7</span></td>
<td id="S4.T1.8.12.2.5" class="ltx_td ltx_align_center"><span id="S4.T1.8.12.2.5.1" class="ltx_text" style="background-color:#F2F2F2;">9.5</span></td>
</tr>
<tr id="S4.T1.8.13.3" class="ltx_tr">
<th id="S4.T1.8.13.3.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T1.8.13.3.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T1.8.13.3.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.13.3.1.1.1.1" class="ltx_text ltx_font_bold">between</span></span>
</span>
</th>
<td id="S4.T1.8.13.3.2" class="ltx_td ltx_align_center">6.2</td>
<td id="S4.T1.8.13.3.3" class="ltx_td ltx_align_center">5.3</td>
<td id="S4.T1.8.13.3.4" class="ltx_td ltx_align_center">5.9</td>
<td id="S4.T1.8.13.3.5" class="ltx_td ltx_align_center">8.2</td>
</tr>
<tr id="S4.T1.8.14.4" class="ltx_tr" style="background-color:#F2F2F2;">
<th id="S4.T1.8.14.4.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T1.8.14.4.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#F2F2F2;">
<span id="S4.T1.8.14.4.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.14.4.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">left</span></span>
</span>
</th>
<td id="S4.T1.8.14.4.2" class="ltx_td ltx_align_center"><span id="S4.T1.8.14.4.2.1" class="ltx_text" style="background-color:#F2F2F2;">4.5</span></td>
<td id="S4.T1.8.14.4.3" class="ltx_td ltx_align_center"><span id="S4.T1.8.14.4.3.1" class="ltx_text" style="background-color:#F2F2F2;">4.8</span></td>
<td id="S4.T1.8.14.4.4" class="ltx_td ltx_align_center"><span id="S4.T1.8.14.4.4.1" class="ltx_text" style="background-color:#F2F2F2;">5.3</span></td>
<td id="S4.T1.8.14.4.5" class="ltx_td ltx_align_center"><span id="S4.T1.8.14.4.5.1" class="ltx_text" style="background-color:#F2F2F2;">7.4</span></td>
</tr>
<tr id="S4.T1.8.15.5" class="ltx_tr">
<th id="S4.T1.8.15.5.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T1.8.15.5.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T1.8.15.5.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.15.5.1.1.1.1" class="ltx_text ltx_font_bold">would</span></span>
</span>
</th>
<td id="S4.T1.8.15.5.2" class="ltx_td ltx_align_center">3.2</td>
<td id="S4.T1.8.15.5.3" class="ltx_td ltx_align_center">4.3</td>
<td id="S4.T1.8.15.5.4" class="ltx_td ltx_align_center">4.7</td>
<td id="S4.T1.8.15.5.5" class="ltx_td ltx_align_center">6.6</td>
</tr>
<tr id="S4.T1.8.16.6" class="ltx_tr" style="background-color:#F2F2F2;">
<th id="S4.T1.8.16.6.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T1.8.16.6.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#F2F2F2;">
<span id="S4.T1.8.16.6.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.16.6.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">could</span></span>
</span>
</th>
<td id="S4.T1.8.16.6.2" class="ltx_td ltx_align_center"><span id="S4.T1.8.16.6.2.1" class="ltx_text" style="background-color:#F2F2F2;">1.2</span></td>
<td id="S4.T1.8.16.6.3" class="ltx_td ltx_align_center"><span id="S4.T1.8.16.6.3.1" class="ltx_text" style="background-color:#F2F2F2;">3.1</span></td>
<td id="S4.T1.8.16.6.4" class="ltx_td ltx_align_center"><span id="S4.T1.8.16.6.4.1" class="ltx_text" style="background-color:#F2F2F2;">3.4</span></td>
<td id="S4.T1.8.16.6.5" class="ltx_td ltx_align_center"><span id="S4.T1.8.16.6.5.1" class="ltx_text" style="background-color:#F2F2F2;">4.8</span></td>
</tr>
<tr id="S4.T1.8.17.7" class="ltx_tr">
<th id="S4.T1.8.17.7.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_b">
<span id="S4.T1.8.17.7.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T1.8.17.7.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T1.8.17.7.1.1.1.1" class="ltx_text ltx_font_bold">seemed</span></span>
</span>
</th>
<td id="S4.T1.8.17.7.2" class="ltx_td ltx_align_center ltx_border_b">0.5</td>
<td id="S4.T1.8.17.7.3" class="ltx_td ltx_align_center ltx_border_b">2.3</td>
<td id="S4.T1.8.17.7.4" class="ltx_td ltx_align_center ltx_border_b">2.5</td>
<td id="S4.T1.8.17.7.5" class="ltx_td ltx_align_center ltx_border_b">3.5</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 1: </span>Comparison between temperature sampling, top-<math id="S4.T1.13.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T1.13.m1.1b"><mi id="S4.T1.13.m1.1.1" xref="S4.T1.13.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T1.13.m1.1c"><ci id="S4.T1.13.m1.1.1.cmml" xref="S4.T1.13.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.13.m1.1d">p</annotation></semantics></math>, min-<math id="S4.T1.14.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T1.14.m2.1b"><mi id="S4.T1.14.m2.1.1" xref="S4.T1.14.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T1.14.m2.1c"><ci id="S4.T1.14.m2.1.1.cmml" xref="S4.T1.14.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.14.m2.1d">p</annotation></semantics></math> sampling at higher temperatures, for a low-certainty case. In cases where the tokens are relatively low-certainty, min-<math id="S4.T1.15.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T1.15.m3.1b"><mi id="S4.T1.15.m3.1.1" xref="S4.T1.15.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T1.15.m3.1c"><ci id="S4.T1.15.m3.1.1.cmml" xref="S4.T1.15.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.15.m3.1d">p</annotation></semantics></math> does not differ significantly from top-<math id="S4.T1.16.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T1.16.m4.1b"><mi id="S4.T1.16.m4.1.1" xref="S4.T1.16.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T1.16.m4.1c"><ci id="S4.T1.16.m4.1.1.cmml" xref="S4.T1.16.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T1.16.m4.1d">p</annotation></semantics></math>.</figcaption>
</figure>
<figure id="S4.T2" class="ltx_table">
<table id="S4.T2.8" class="ltx_tabular ltx_guessed_headers ltx_align_middle">
<thead class="ltx_thead">
<tr id="S4.T2.8.9.1" class="ltx_tr" style="background-color:#DFDFDF;">
<th id="S4.T2.8.9.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_l ltx_border_r ltx_border_t" colspan="5">
<span id="S4.T2.8.9.1.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#DFDFDF;">
<span id="S4.T2.8.9.1.1.1.1" class="ltx_p" style="width:209.1pt;"><span id="S4.T2.8.9.1.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#DFDFDF;">Prompt</span></span>
</span>
</th>
</tr>
<tr id="S4.T2.8.10.2" class="ltx_tr" style="background-color:#DFDFDF;">
<th id="S4.T2.8.10.2.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_l ltx_border_r" colspan="5">
<span id="S4.T2.8.10.2.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#DFDFDF;">
<span id="S4.T2.8.10.2.1.1.1" class="ltx_p" style="width:209.1pt;">A rainbow is an optically brilliant meteorological event, resulting from the refraction, reflection, and dispersion of _____</span>
</span>
</th>
</tr>
</thead>
<tbody class="ltx_tbody">
<tr id="S4.T2.2.2" class="ltx_tr">
<th id="S4.T2.2.2.3" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_t">
<span id="S4.T2.2.2.3.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T2.2.2.3.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.2.2.3.1.1.1" class="ltx_text ltx_font_bold">Sampling</span></span>
</span>
</th>
<td id="S4.T2.2.2.4" class="ltx_td ltx_align_center ltx_border_t" colspan="2"><span id="S4.T2.2.2.4.1" class="ltx_text ltx_font_bold">Temperature</span></td>
<td id="S4.T2.1.1.1" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T2.1.1.1.1" class="ltx_text ltx_font_bold">+top-<math id="S4.T2.1.1.1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.1.1.1.1.m1.1a"><mi id="S4.T2.1.1.1.1.m1.1.1" xref="S4.T2.1.1.1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.1.1.1.1.m1.1b"><ci id="S4.T2.1.1.1.1.m1.1.1.cmml" xref="S4.T2.1.1.1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.1.1.1.1.m1.1c">p</annotation></semantics></math></span></td>
<td id="S4.T2.2.2.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T2.2.2.2.1" class="ltx_text ltx_font_bold">+min-<math id="S4.T2.2.2.2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.2.2.2.1.m1.1a"><mi id="S4.T2.2.2.2.1.m1.1.1" xref="S4.T2.2.2.2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.2.2.2.1.m1.1b"><ci id="S4.T2.2.2.2.1.m1.1.1.cmml" xref="S4.T2.2.2.2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.2.2.2.1.m1.1c">p</annotation></semantics></math></span></td>
</tr>
<tr id="S4.T2.6.6" class="ltx_tr">
<th id="S4.T2.6.6.5" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T2.6.6.5.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T2.6.6.5.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.6.6.5.1.1.1" class="ltx_text ltx_font_bold">Method</span></span>
</span>
</th>
<td id="S4.T2.3.3.1" class="ltx_td ltx_align_center"><math id="S4.T2.3.3.1.m1.1" class="ltx_Math" alttext="\tau\!=\!1" display="inline"><semantics id="S4.T2.3.3.1.m1.1a"><mrow id="S4.T2.3.3.1.m1.1.1" xref="S4.T2.3.3.1.m1.1.1.cmml"><mi id="S4.T2.3.3.1.m1.1.1.2" xref="S4.T2.3.3.1.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T2.3.3.1.m1.1.1.1" xref="S4.T2.3.3.1.m1.1.1.1.cmml">=</mo><mn id="S4.T2.3.3.1.m1.1.1.3" xref="S4.T2.3.3.1.m1.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.3.3.1.m1.1b"><apply id="S4.T2.3.3.1.m1.1.1.cmml" xref="S4.T2.3.3.1.m1.1.1"><eq id="S4.T2.3.3.1.m1.1.1.1.cmml" xref="S4.T2.3.3.1.m1.1.1.1"></eq><ci id="S4.T2.3.3.1.m1.1.1.2.cmml" xref="S4.T2.3.3.1.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T2.3.3.1.m1.1.1.3.cmml" xref="S4.T2.3.3.1.m1.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.3.3.1.m1.1c">\tau\!=\!1</annotation></semantics></math></td>
<td id="S4.T2.4.4.2" class="ltx_td ltx_align_center"><math id="S4.T2.4.4.2.m1.1" class="ltx_Math" alttext="\tau\!=\!3" display="inline"><semantics id="S4.T2.4.4.2.m1.1a"><mrow id="S4.T2.4.4.2.m1.1.1" xref="S4.T2.4.4.2.m1.1.1.cmml"><mi id="S4.T2.4.4.2.m1.1.1.2" xref="S4.T2.4.4.2.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T2.4.4.2.m1.1.1.1" xref="S4.T2.4.4.2.m1.1.1.1.cmml">=</mo><mn id="S4.T2.4.4.2.m1.1.1.3" xref="S4.T2.4.4.2.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.4.4.2.m1.1b"><apply id="S4.T2.4.4.2.m1.1.1.cmml" xref="S4.T2.4.4.2.m1.1.1"><eq id="S4.T2.4.4.2.m1.1.1.1.cmml" xref="S4.T2.4.4.2.m1.1.1.1"></eq><ci id="S4.T2.4.4.2.m1.1.1.2.cmml" xref="S4.T2.4.4.2.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T2.4.4.2.m1.1.1.3.cmml" xref="S4.T2.4.4.2.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.4.4.2.m1.1c">\tau\!=\!3</annotation></semantics></math></td>
<td id="S4.T2.5.5.3" class="ltx_td ltx_align_center"><math id="S4.T2.5.5.3.m1.1" class="ltx_Math" alttext="\tau\!=\!3" display="inline"><semantics id="S4.T2.5.5.3.m1.1a"><mrow id="S4.T2.5.5.3.m1.1.1" xref="S4.T2.5.5.3.m1.1.1.cmml"><mi id="S4.T2.5.5.3.m1.1.1.2" xref="S4.T2.5.5.3.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T2.5.5.3.m1.1.1.1" xref="S4.T2.5.5.3.m1.1.1.1.cmml">=</mo><mn id="S4.T2.5.5.3.m1.1.1.3" xref="S4.T2.5.5.3.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.5.5.3.m1.1b"><apply id="S4.T2.5.5.3.m1.1.1.cmml" xref="S4.T2.5.5.3.m1.1.1"><eq id="S4.T2.5.5.3.m1.1.1.1.cmml" xref="S4.T2.5.5.3.m1.1.1.1"></eq><ci id="S4.T2.5.5.3.m1.1.1.2.cmml" xref="S4.T2.5.5.3.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T2.5.5.3.m1.1.1.3.cmml" xref="S4.T2.5.5.3.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.5.5.3.m1.1c">\tau\!=\!3</annotation></semantics></math></td>
<td id="S4.T2.6.6.4" class="ltx_td ltx_align_center"><math id="S4.T2.6.6.4.m1.1" class="ltx_Math" alttext="\tau\!=\!3" display="inline"><semantics id="S4.T2.6.6.4.m1.1a"><mrow id="S4.T2.6.6.4.m1.1.1" xref="S4.T2.6.6.4.m1.1.1.cmml"><mi id="S4.T2.6.6.4.m1.1.1.2" xref="S4.T2.6.6.4.m1.1.1.2.cmml">τ</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T2.6.6.4.m1.1.1.1" xref="S4.T2.6.6.4.m1.1.1.1.cmml">=</mo><mn id="S4.T2.6.6.4.m1.1.1.3" xref="S4.T2.6.6.4.m1.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.6.6.4.m1.1b"><apply id="S4.T2.6.6.4.m1.1.1.cmml" xref="S4.T2.6.6.4.m1.1.1"><eq id="S4.T2.6.6.4.m1.1.1.1.cmml" xref="S4.T2.6.6.4.m1.1.1.1"></eq><ci id="S4.T2.6.6.4.m1.1.1.2.cmml" xref="S4.T2.6.6.4.m1.1.1.2">𝜏</ci><cn type="integer" id="S4.T2.6.6.4.m1.1.1.3.cmml" xref="S4.T2.6.6.4.m1.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.6.6.4.m1.1c">\tau\!=\!3</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.8.8" class="ltx_tr">
<th id="S4.T2.8.8.3" class="ltx_td ltx_align_top ltx_th ltx_th_row"></th>
<td id="S4.T2.8.8.4" class="ltx_td"></td>
<td id="S4.T2.8.8.5" class="ltx_td"></td>
<td id="S4.T2.7.7.1" class="ltx_td ltx_align_center"><math id="S4.T2.7.7.1.m1.1" class="ltx_Math" alttext="p\!=\!0.9" display="inline"><semantics id="S4.T2.7.7.1.m1.1a"><mrow id="S4.T2.7.7.1.m1.1.1" xref="S4.T2.7.7.1.m1.1.1.cmml"><mi id="S4.T2.7.7.1.m1.1.1.2" xref="S4.T2.7.7.1.m1.1.1.2.cmml">p</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T2.7.7.1.m1.1.1.1" xref="S4.T2.7.7.1.m1.1.1.1.cmml">=</mo><mn id="S4.T2.7.7.1.m1.1.1.3" xref="S4.T2.7.7.1.m1.1.1.3.cmml">0.9</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.7.7.1.m1.1b"><apply id="S4.T2.7.7.1.m1.1.1.cmml" xref="S4.T2.7.7.1.m1.1.1"><eq id="S4.T2.7.7.1.m1.1.1.1.cmml" xref="S4.T2.7.7.1.m1.1.1.1"></eq><ci id="S4.T2.7.7.1.m1.1.1.2.cmml" xref="S4.T2.7.7.1.m1.1.1.2">𝑝</ci><cn type="float" id="S4.T2.7.7.1.m1.1.1.3.cmml" xref="S4.T2.7.7.1.m1.1.1.3">0.9</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.7.7.1.m1.1c">p\!=\!0.9</annotation></semantics></math></td>
<td id="S4.T2.8.8.2" class="ltx_td ltx_align_center"><math id="S4.T2.8.8.2.m1.1" class="ltx_Math" alttext="p\!=\!0.1" display="inline"><semantics id="S4.T2.8.8.2.m1.1a"><mrow id="S4.T2.8.8.2.m1.1.1" xref="S4.T2.8.8.2.m1.1.1.cmml"><mi id="S4.T2.8.8.2.m1.1.1.2" xref="S4.T2.8.8.2.m1.1.1.2.cmml">p</mi><mo lspace="0.108em" rspace="0.108em" id="S4.T2.8.8.2.m1.1.1.1" xref="S4.T2.8.8.2.m1.1.1.1.cmml">=</mo><mn id="S4.T2.8.8.2.m1.1.1.3" xref="S4.T2.8.8.2.m1.1.1.3.cmml">0.1</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.8.8.2.m1.1b"><apply id="S4.T2.8.8.2.m1.1.1.cmml" xref="S4.T2.8.8.2.m1.1.1"><eq id="S4.T2.8.8.2.m1.1.1.1.cmml" xref="S4.T2.8.8.2.m1.1.1.1"></eq><ci id="S4.T2.8.8.2.m1.1.1.2.cmml" xref="S4.T2.8.8.2.m1.1.1.2">𝑝</ci><cn type="float" id="S4.T2.8.8.2.m1.1.1.3.cmml" xref="S4.T2.8.8.2.m1.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.8.8.2.m1.1c">p\!=\!0.1</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.8.11.1" class="ltx_tr">
<th id="S4.T2.8.11.1.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_t">
<span id="S4.T2.8.11.1.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T2.8.11.1.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.8.11.1.1.1.1.1" class="ltx_text ltx_font_bold">light</span></span>
</span>
</th>
<td id="S4.T2.8.11.1.2" class="ltx_td ltx_align_center ltx_border_t">98.25</td>
<td id="S4.T2.8.11.1.3" class="ltx_td ltx_align_center ltx_border_t">34.41</td>
<td id="S4.T2.8.11.1.4" class="ltx_td ltx_align_center ltx_border_t">38.18</td>
<td id="S4.T2.8.11.1.5" class="ltx_td ltx_align_center ltx_border_t">80.90</td>
</tr>
<tr id="S4.T2.8.12.2" class="ltx_tr" style="background-color:#F2F2F2;">
<th id="S4.T2.8.12.2.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T2.8.12.2.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#F2F2F2;">
<span id="S4.T2.8.12.2.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.8.12.2.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">sunlight</span></span>
</span>
</th>
<td id="S4.T2.8.12.2.2" class="ltx_td ltx_align_center"><span id="S4.T2.8.12.2.2.1" class="ltx_text" style="background-color:#F2F2F2;">1.29</span></td>
<td id="S4.T2.8.12.2.3" class="ltx_td ltx_align_center"><span id="S4.T2.8.12.2.3.1" class="ltx_text" style="background-color:#F2F2F2;">8.12</span></td>
<td id="S4.T2.8.12.2.4" class="ltx_td ltx_align_center"><span id="S4.T2.8.12.2.4.1" class="ltx_text" style="background-color:#F2F2F2;">9.01</span></td>
<td id="S4.T2.8.12.2.5" class="ltx_td ltx_align_center"><span id="S4.T2.8.12.2.5.1" class="ltx_text" style="background-color:#F2F2F2;">19.10</span></td>
</tr>
<tr id="S4.T2.8.13.3" class="ltx_tr">
<th id="S4.T2.8.13.3.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T2.8.13.3.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T2.8.13.3.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.8.13.3.1.1.1.1" class="ltx_text ltx_font_bold">water</span></span>
</span>
</th>
<td id="S4.T2.8.13.3.2" class="ltx_td ltx_align_center">0.10</td>
<td id="S4.T2.8.13.3.3" class="ltx_td ltx_align_center">3.44</td>
<td id="S4.T2.8.13.3.4" class="ltx_td ltx_align_center">3.82</td>
<td id="S4.T2.8.13.3.5" class="ltx_td ltx_align_center">0.00</td>
</tr>
<tr id="S4.T2.8.14.4" class="ltx_tr" style="background-color:#F2F2F2;">
<th id="S4.T2.8.14.4.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T2.8.14.4.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#F2F2F2;">
<span id="S4.T2.8.14.4.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.8.14.4.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">sunshine</span></span>
</span>
</th>
<td id="S4.T2.8.14.4.2" class="ltx_td ltx_align_center"><span id="S4.T2.8.14.4.2.1" class="ltx_text" style="background-color:#F2F2F2;">0.06</span></td>
<td id="S4.T2.8.14.4.3" class="ltx_td ltx_align_center"><span id="S4.T2.8.14.4.3.1" class="ltx_text" style="background-color:#F2F2F2;">2.89</span></td>
<td id="S4.T2.8.14.4.4" class="ltx_td ltx_align_center"><span id="S4.T2.8.14.4.4.1" class="ltx_text" style="background-color:#F2F2F2;">3.21</span></td>
<td id="S4.T2.8.14.4.5" class="ltx_td ltx_align_center"><span id="S4.T2.8.14.4.5.1" class="ltx_text" style="background-color:#F2F2F2;">0.00</span></td>
</tr>
<tr id="S4.T2.8.15.5" class="ltx_tr">
<th id="S4.T2.8.15.5.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row">
<span id="S4.T2.8.15.5.1.1" class="ltx_inline-block ltx_align_top">
<span id="S4.T2.8.15.5.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.8.15.5.1.1.1.1" class="ltx_text ltx_font_bold">a</span></span>
</span>
</th>
<td id="S4.T2.8.15.5.2" class="ltx_td ltx_align_center">0.05</td>
<td id="S4.T2.8.15.5.3" class="ltx_td ltx_align_center">2.71</td>
<td id="S4.T2.8.15.5.4" class="ltx_td ltx_align_center">3.01</td>
<td id="S4.T2.8.15.5.5" class="ltx_td ltx_align_center">0.00</td>
</tr>
<tr id="S4.T2.8.16.6" class="ltx_tr" style="background-color:#F2F2F2;">
<th id="S4.T2.8.16.6.1" class="ltx_td ltx_align_justify ltx_align_top ltx_th ltx_th_row ltx_border_b">
<span id="S4.T2.8.16.6.1.1" class="ltx_inline-block ltx_align_top" style="background-color:#F2F2F2;">
<span id="S4.T2.8.16.6.1.1.1" class="ltx_p" style="width:41.3pt;"><span id="S4.T2.8.16.6.1.1.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">moisture</span></span>
</span>
</th>
<td id="S4.T2.8.16.6.2" class="ltx_td ltx_align_center ltx_border_b"><span id="S4.T2.8.16.6.2.1" class="ltx_text" style="background-color:#F2F2F2;">0.05</span></td>
<td id="S4.T2.8.16.6.3" class="ltx_td ltx_align_center ltx_border_b"><span id="S4.T2.8.16.6.3.1" class="ltx_text" style="background-color:#F2F2F2;">2.70</span></td>
<td id="S4.T2.8.16.6.4" class="ltx_td ltx_align_center ltx_border_b"><span id="S4.T2.8.16.6.4.1" class="ltx_text" style="background-color:#F2F2F2;">3.00</span></td>
<td id="S4.T2.8.16.6.5" class="ltx_td ltx_align_center ltx_border_b"><span id="S4.T2.8.16.6.5.1" class="ltx_text" style="background-color:#F2F2F2;">0.00</span></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_table">Table 2: </span>Comparison between temperature sampling, top-<math id="S4.T2.16.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.16.m1.1b"><mi id="S4.T2.16.m1.1.1" xref="S4.T2.16.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.16.m1.1c"><ci id="S4.T2.16.m1.1.1.cmml" xref="S4.T2.16.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.16.m1.1d">p</annotation></semantics></math>, min-<math id="S4.T2.17.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.17.m2.1b"><mi id="S4.T2.17.m2.1.1" xref="S4.T2.17.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.17.m2.1c"><ci id="S4.T2.17.m2.1.1.cmml" xref="S4.T2.17.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.17.m2.1d">p</annotation></semantics></math> sampling at higher temperatures, for a high-certainty case. In cases where the tokens are high-certainty, min-<math id="S4.T2.18.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.18.m3.1b"><mi id="S4.T2.18.m3.1.1" xref="S4.T2.18.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.18.m3.1c"><ci id="S4.T2.18.m3.1.1.cmml" xref="S4.T2.18.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.18.m3.1d">p</annotation></semantics></math> filters for the high probability token more aggressively than top-<math id="S4.T2.19.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.19.m4.1b"><mi id="S4.T2.19.m4.1.1" xref="S4.T2.19.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.19.m4.1c"><ci id="S4.T2.19.m4.1.1.cmml" xref="S4.T2.19.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.19.m4.1d">p</annotation></semantics></math>. In this case, when <math id="S4.T2.20.m5.1" class="ltx_Math" alttext="\tau=3" display="inline"><semantics id="S4.T2.20.m5.1b"><mrow id="S4.T2.20.m5.1.1" xref="S4.T2.20.m5.1.1.cmml"><mi id="S4.T2.20.m5.1.1.2" xref="S4.T2.20.m5.1.1.2.cmml">τ</mi><mo id="S4.T2.20.m5.1.1.1" xref="S4.T2.20.m5.1.1.1.cmml">=</mo><mn id="S4.T2.20.m5.1.1.3" xref="S4.T2.20.m5.1.1.3.cmml">3</mn></mrow><annotation-xml encoding="MathML-Content" id="S4.T2.20.m5.1c"><apply id="S4.T2.20.m5.1.1.cmml" xref="S4.T2.20.m5.1.1"><eq id="S4.T2.20.m5.1.1.1.cmml" xref="S4.T2.20.m5.1.1.1"></eq><ci id="S4.T2.20.m5.1.1.2.cmml" xref="S4.T2.20.m5.1.1.2">𝜏</ci><cn type="integer" id="S4.T2.20.m5.1.1.3.cmml" xref="S4.T2.20.m5.1.1.3">3</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.20.m5.1d">\tau=3</annotation></semantics></math>, top-<math id="S4.T2.21.m6.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.21.m6.1b"><mi id="S4.T2.21.m6.1.1" xref="S4.T2.21.m6.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.21.m6.1c"><ci id="S4.T2.21.m6.1.1.cmml" xref="S4.T2.21.m6.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.21.m6.1d">p</annotation></semantics></math> lets initially low probability tokens (&lt;0.10% probability) into its nucleus (~3% probability), while min-<math id="S4.T2.22.m7.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.T2.22.m7.1b"><mi id="S4.T2.22.m7.1.1" xref="S4.T2.22.m7.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.T2.22.m7.1c"><ci id="S4.T2.22.m7.1.1.cmml" xref="S4.T2.22.m7.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.22.m7.1d">p</annotation></semantics></math> filters them out entirely.</figcaption>
</figure>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>Discussion</h3>

<div id="S4.SS3.p1" class="ltx_para">
<p id="S4.SS3.p1.2" class="ltx_p">Our case studies demonstrate that min-<math id="S4.SS3.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p1.1.m1.1a"><mi id="S4.SS3.p1.1.m1.1.1" xref="S4.SS3.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.1.m1.1b"><ci id="S4.SS3.p1.1.m1.1.1.cmml" xref="S4.SS3.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.1.m1.1c">p</annotation></semantics></math> sampling’s token choices differ most notably from top-<math id="S4.SS3.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p1.2.m2.1a"><mi id="S4.SS3.p1.2.m2.1.1" xref="S4.SS3.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p1.2.m2.1b"><ci id="S4.SS3.p1.2.m2.1.1.cmml" xref="S4.SS3.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p1.2.m2.1c">p</annotation></semantics></math> sampling when selecting high-probability tokens at elevated temperatures. This has important implications for improving the performance of language models in high-temperature settings:</p>
</div>
<div id="S4.SS3.p2" class="ltx_para">
<p id="S4.SS3.p2.5" class="ltx_p"><span id="S4.SS3.p2.5.1" class="ltx_text ltx_font_bold">Maintaining accuracy in reasoning tasks.</span> For tasks that require factual and logical reasoning, such as mathematics, science, programming, and recalling specific facts, min-<math id="S4.SS3.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p2.1.m1.1a"><mi id="S4.SS3.p2.1.m1.1.1" xref="S4.SS3.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.1.m1.1b"><ci id="S4.SS3.p2.1.m1.1.1.cmml" xref="S4.SS3.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.1.m1.1c">p</annotation></semantics></math> significantly reduces the accuracy tradeoff at high temperatures. In the rainbow example (Table <a href="#S4.T2" title="Table 2 ‣ 4.2 Case 2: High-Certainty Next Token ‣ 4 Case Study ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>), 100% of min-<math id="S4.SS3.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p2.2.m2.1a"><mi id="S4.SS3.p2.2.m2.1.1" xref="S4.SS3.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.2.m2.1b"><ci id="S4.SS3.p2.2.m2.1.1.cmml" xref="S4.SS3.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.2.m2.1c">p</annotation></semantics></math>’s probability distribution ("light" and "sunlight") can be considered accurate, with the most relevant answer "light" comprising 80.90%. In contrast, only 47.19% of top-<math id="S4.SS3.p2.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p2.3.m3.1a"><mi id="S4.SS3.p2.3.m3.1.1" xref="S4.SS3.p2.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.3.m3.1b"><ci id="S4.SS3.p2.3.m3.1.1.cmml" xref="S4.SS3.p2.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.3.m3.1c">p</annotation></semantics></math>’s distribution is relevant to light. This suggests that min-<math id="S4.SS3.p2.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p2.4.m4.1a"><mi id="S4.SS3.p2.4.m4.1.1" xref="S4.SS3.p2.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.4.m4.1b"><ci id="S4.SS3.p2.4.m4.1.1.cmml" xref="S4.SS3.p2.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.4.m4.1c">p</annotation></semantics></math> can enable language models to tackle reasoning tasks at temperature settings previously considered unsuitable. We further evaluate min-<math id="S4.SS3.p2.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p2.5.m5.1a"><mi id="S4.SS3.p2.5.m5.1.1" xref="S4.SS3.p2.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p2.5.m5.1b"><ci id="S4.SS3.p2.5.m5.1.1.cmml" xref="S4.SS3.p2.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p2.5.m5.1c">p</annotation></semantics></math>’s reasoning capabilities in Section <a href="#S5" title="5 Experiments ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</p>
</div>
<div id="S4.SS3.p3" class="ltx_para">
<p id="S4.SS3.p3.1" class="ltx_p"><span id="S4.SS3.p3.1.1" class="ltx_text ltx_font_bold">Mitigating cascading errors.</span> Language models construct longform outputs by sequentially selecting tokens, with previous choices influencing future decisions. Consequently, erroneous token selections can compound, increasing the likelihood of further inaccurate or incoherent choices. The rainbow prompt illustrates this, as it contains multiple technical terms ("meteorological", "refraction", "dispersion") that require high-accuracy token selections to maintain semantic coherence and factual correctness. Min-<math id="S4.SS3.p3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S4.SS3.p3.1.m1.1a"><mi id="S4.SS3.p3.1.m1.1.1" xref="S4.SS3.p3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S4.SS3.p3.1.m1.1b"><ci id="S4.SS3.p3.1.m1.1.1.cmml" xref="S4.SS3.p3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.SS3.p3.1.m1.1c">p</annotation></semantics></math>’s ability to robustly select high-confidence tokens, even at high temperatures, helps prevent such cascading failures.</p>
</div>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Experiments</h2>

<div id="S5.p1" class="ltx_para">
<p id="S5.p1.2" class="ltx_p">To establish min-<math id="S5.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.p1.1.m1.1a"><mi id="S5.p1.1.m1.1.1" xref="S5.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.p1.1.m1.1b"><ci id="S5.p1.1.m1.1.1.cmml" xref="S5.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.1.m1.1c">p</annotation></semantics></math> sampling as a viable alternative to the state-of-the-art top-<math id="S5.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.p1.2.m2.1a"><mi id="S5.p1.2.m2.1.1" xref="S5.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.p1.2.m2.1b"><ci id="S5.p1.2.m2.1.1.cmml" xref="S5.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.p1.2.m2.1c">p</annotation></semantics></math> (nucleus) sampling, we conduct experiments to demonstrate that:</p>
<ol id="S5.I1" class="ltx_enumerate">
<li id="S5.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">1.</span> 
<div id="S5.I1.i1.p1" class="ltx_para">
<p id="S5.I1.i1.p1.3" class="ltx_p">Min-<math id="S5.I1.i1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.I1.i1.p1.1.m1.1a"><mi id="S5.I1.i1.p1.1.m1.1.1" xref="S5.I1.i1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.I1.i1.p1.1.m1.1b"><ci id="S5.I1.i1.p1.1.m1.1.1.cmml" xref="S5.I1.i1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.I1.i1.p1.1.m1.1c">p</annotation></semantics></math> achieves equal or better benchmark performance compared to top-<math id="S5.I1.i1.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.I1.i1.p1.2.m2.1a"><mi id="S5.I1.i1.p1.2.m2.1.1" xref="S5.I1.i1.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.I1.i1.p1.2.m2.1b"><ci id="S5.I1.i1.p1.2.m2.1.1.cmml" xref="S5.I1.i1.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.I1.i1.p1.2.m2.1c">p</annotation></semantics></math> at standard temperature settings (<math id="S5.I1.i1.p1.3.m3.2" class="ltx_Math" alttext="\tau\in[0,1]" display="inline"><semantics id="S5.I1.i1.p1.3.m3.2a"><mrow id="S5.I1.i1.p1.3.m3.2.3" xref="S5.I1.i1.p1.3.m3.2.3.cmml"><mi id="S5.I1.i1.p1.3.m3.2.3.2" xref="S5.I1.i1.p1.3.m3.2.3.2.cmml">τ</mi><mo id="S5.I1.i1.p1.3.m3.2.3.1" xref="S5.I1.i1.p1.3.m3.2.3.1.cmml">∈</mo><mrow id="S5.I1.i1.p1.3.m3.2.3.3.2" xref="S5.I1.i1.p1.3.m3.2.3.3.1.cmml"><mo stretchy="false" id="S5.I1.i1.p1.3.m3.2.3.3.2.1" xref="S5.I1.i1.p1.3.m3.2.3.3.1.cmml">[</mo><mn id="S5.I1.i1.p1.3.m3.1.1" xref="S5.I1.i1.p1.3.m3.1.1.cmml">0</mn><mo id="S5.I1.i1.p1.3.m3.2.3.3.2.2" xref="S5.I1.i1.p1.3.m3.2.3.3.1.cmml">,</mo><mn id="S5.I1.i1.p1.3.m3.2.2" xref="S5.I1.i1.p1.3.m3.2.2.cmml">1</mn><mo stretchy="false" id="S5.I1.i1.p1.3.m3.2.3.3.2.3" xref="S5.I1.i1.p1.3.m3.2.3.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S5.I1.i1.p1.3.m3.2b"><apply id="S5.I1.i1.p1.3.m3.2.3.cmml" xref="S5.I1.i1.p1.3.m3.2.3"><in id="S5.I1.i1.p1.3.m3.2.3.1.cmml" xref="S5.I1.i1.p1.3.m3.2.3.1"></in><ci id="S5.I1.i1.p1.3.m3.2.3.2.cmml" xref="S5.I1.i1.p1.3.m3.2.3.2">𝜏</ci><interval closure="closed" id="S5.I1.i1.p1.3.m3.2.3.3.1.cmml" xref="S5.I1.i1.p1.3.m3.2.3.3.2"><cn type="integer" id="S5.I1.i1.p1.3.m3.1.1.cmml" xref="S5.I1.i1.p1.3.m3.1.1">0</cn><cn type="integer" id="S5.I1.i1.p1.3.m3.2.2.cmml" xref="S5.I1.i1.p1.3.m3.2.2">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.I1.i1.p1.3.m3.2c">\tau\in[0,1]</annotation></semantics></math>).</p>
</div>
</li>
<li id="S5.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">2.</span> 
<div id="S5.I1.i2.p1" class="ltx_para">
<p id="S5.I1.i2.p1.2" class="ltx_p">Min-<math id="S5.I1.i2.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.I1.i2.p1.1.m1.1a"><mi id="S5.I1.i2.p1.1.m1.1.1" xref="S5.I1.i2.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.I1.i2.p1.1.m1.1b"><ci id="S5.I1.i2.p1.1.m1.1.1.cmml" xref="S5.I1.i2.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.I1.i2.p1.1.m1.1c">p</annotation></semantics></math> handles the creativity-coherence tradeoff well at higher temperatures (<math id="S5.I1.i2.p1.2.m2.1" class="ltx_Math" alttext="\tau&gt;1" display="inline"><semantics id="S5.I1.i2.p1.2.m2.1a"><mrow id="S5.I1.i2.p1.2.m2.1.1" xref="S5.I1.i2.p1.2.m2.1.1.cmml"><mi id="S5.I1.i2.p1.2.m2.1.1.2" xref="S5.I1.i2.p1.2.m2.1.1.2.cmml">τ</mi><mo id="S5.I1.i2.p1.2.m2.1.1.1" xref="S5.I1.i2.p1.2.m2.1.1.1.cmml">&gt;</mo><mn id="S5.I1.i2.p1.2.m2.1.1.3" xref="S5.I1.i2.p1.2.m2.1.1.3.cmml">1</mn></mrow><annotation-xml encoding="MathML-Content" id="S5.I1.i2.p1.2.m2.1b"><apply id="S5.I1.i2.p1.2.m2.1.1.cmml" xref="S5.I1.i2.p1.2.m2.1.1"><gt id="S5.I1.i2.p1.2.m2.1.1.1.cmml" xref="S5.I1.i2.p1.2.m2.1.1.1"></gt><ci id="S5.I1.i2.p1.2.m2.1.1.2.cmml" xref="S5.I1.i2.p1.2.m2.1.1.2">𝜏</ci><cn type="integer" id="S5.I1.i2.p1.2.m2.1.1.3.cmml" xref="S5.I1.i2.p1.2.m2.1.1.3">1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S5.I1.i2.p1.2.m2.1c">\tau&gt;1</annotation></semantics></math>) by achieving more diverse outputs while maintaining coherence, as demonstrated by improved performance on reasoning benchmarks at high temperatures.</p>
</div>
</li>
</ol>
</div>
<section id="S5.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.1 </span>Experimental Setup</h3>

<div id="S5.SS1.p1" class="ltx_para">
<p id="S5.SS1.p1.2" class="ltx_p">We evaluate the performance of min-<math id="S5.SS1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS1.p1.1.m1.1a"><mi id="S5.SS1.p1.1.m1.1.1" xref="S5.SS1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p1.1.m1.1b"><ci id="S5.SS1.p1.1.m1.1.1.cmml" xref="S5.SS1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p1.1.m1.1c">p</annotation></semantics></math> sampling against top-<math id="S5.SS1.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS1.p1.2.m2.1a"><mi id="S5.SS1.p1.2.m2.1.1" xref="S5.SS1.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p1.2.m2.1b"><ci id="S5.SS1.p1.2.m2.1.1.cmml" xref="S5.SS1.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p1.2.m2.1c">p</annotation></semantics></math> sampling on three generative language model tasks using a Mistral 7B <cite class="ltx_cite ltx_citemacro_citep">(Jiang et al., <a href="#bib.bib9" title="" class="ltx_ref">2023</a>)</cite> model:</p>
<ul id="S5.I2" class="ltx_itemize">
<li id="S5.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S5.I2.i1.p1" class="ltx_para">
<p id="S5.I2.i1.p1.1" class="ltx_p"><span id="S5.I2.i1.p1.1.1" class="ltx_text ltx_font_bold">Graduate-level reasoning:</span> Google-Proof Q&amp;A Benchmark - Main, Generative (GPQA Main) - 5-shot <cite class="ltx_cite ltx_citemacro_citep">(Rein et al., <a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite></p>
</div>
</li>
<li id="S5.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S5.I2.i2.p1" class="ltx_para">
<p id="S5.I2.i2.p1.1" class="ltx_p"><span id="S5.I2.i2.p1.1.1" class="ltx_text ltx_font_bold">Grade school math:</span> GSM8K Chain-of-Thought (GSM8K COT) - 8-shot <cite class="ltx_cite ltx_citemacro_citep">(Cobbe et al., <a href="#bib.bib3" title="" class="ltx_ref">2021</a>)</cite></p>
</div>
</li>
<li id="S5.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S5.I2.i3.p1" class="ltx_para">
<p id="S5.I2.i3.p1.1" class="ltx_p"><span id="S5.I2.i3.p1.1.1" class="ltx_text ltx_font_bold">Creative writing:</span> AlpacaEval Creative Writing <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib11" title="" class="ltx_ref">2023b</a>)</cite></p>
</div>
</li>
</ul>
<p id="S5.SS1.p1.3" class="ltx_p">While 7B parameter models typically achieve less than 50% accuracy on these challenging benchmarks, this is sufficient for comparing the relative performance between truncation sampling methods.</p>
</div>
<div id="S5.SS1.p2" class="ltx_para">
<p id="S5.SS1.p2.4" class="ltx_p">We primarily focus on comparing min-<math id="S5.SS1.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS1.p2.1.m1.1a"><mi id="S5.SS1.p2.1.m1.1.1" xref="S5.SS1.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.1.m1.1b"><ci id="S5.SS1.p2.1.m1.1.1.cmml" xref="S5.SS1.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.1.m1.1c">p</annotation></semantics></math> values in the range of 0.05 to 0.1 with top-<math id="S5.SS1.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS1.p2.2.m2.1a"><mi id="S5.SS1.p2.2.m2.1.1" xref="S5.SS1.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.2.m2.1b"><ci id="S5.SS1.p2.2.m2.1.1.cmml" xref="S5.SS1.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.2.m2.1c">p</annotation></semantics></math> values between 0.9 and 0.95. These are common settings preferred by users, although they are not directly comparable. In general, a higher min-<math id="S5.SS1.p2.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS1.p2.3.m3.1a"><mi id="S5.SS1.p2.3.m3.1.1" xref="S5.SS1.p2.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.3.m3.1b"><ci id="S5.SS1.p2.3.m3.1.1.cmml" xref="S5.SS1.p2.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.3.m3.1c">p</annotation></semantics></math> value or a lower top-<math id="S5.SS1.p2.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS1.p2.4.m4.1a"><mi id="S5.SS1.p2.4.m4.1.1" xref="S5.SS1.p2.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS1.p2.4.m4.1b"><ci id="S5.SS1.p2.4.m4.1.1.cmml" xref="S5.SS1.p2.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS1.p2.4.m4.1c">p</annotation></semantics></math> value results in more selective token choices and a bias towards more likely outputs.</p>
</div>
<div id="S5.SS1.p3" class="ltx_para">
<p id="S5.SS1.p3.1" class="ltx_p">For GPQA and GSM8K, we used the EleutherAI Evaluation Harness <cite class="ltx_cite ltx_citemacro_citep">(Gao et al., <a href="#bib.bib5" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
</section>
<section id="S5.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.2 </span>Graduate-Level Reasoning (GPQA Main)</h3>

<div id="S5.SS2.p1" class="ltx_para">
<p id="S5.SS2.p1.1" class="ltx_p">To evaluate min-<math id="S5.SS2.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS2.p1.1.m1.1a"><mi id="S5.SS2.p1.1.m1.1.1" xref="S5.SS2.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS2.p1.1.m1.1b"><ci id="S5.SS2.p1.1.m1.1.1.cmml" xref="S5.SS2.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS2.p1.1.m1.1c">p</annotation></semantics></math>’s impact on graduate-level reasoning, we employ the Google-Proof Q&amp;A Benchmark (GPQA) <cite class="ltx_cite ltx_citemacro_citep">(Rein et al., <a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite>, a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. GPQA is designed to test difficult, graduate-level domain expertise with minimal dataset leakage, making it particularly suitable for our experiments, as Mistral 7B was released in September 2023, two months before GPQA’s public release.</p>
</div>
<div id="S5.SS2.p2" class="ltx_para">
<p id="S5.SS2.p2.1" class="ltx_p"><span id="S5.SS2.p2.1.1" class="ltx_text ltx_font_bold">5-shot prompting.</span> Initial experiments revealed that the baseline Mistral 7B model at temperature 1 with zero-shot prompting was unable to answer any question correctly (0% accuracy). Few-shot prompting proved necessary to obtain statistically significant and meaningful comparisons. After discussions with GPQA’s lead author regarding standard few-shot settings for a 7B parameter model, we opted for 5-shot prompting in this evaluation.</p>
</div>
</section>
<section id="S5.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.3 </span>Grade School Math (GSM8K COT)</h3>

<div id="S5.SS3.p1" class="ltx_para">
<p id="S5.SS3.p1.1" class="ltx_p">To evaluate min-<math id="S5.SS3.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS3.p1.1.m1.1a"><mi id="S5.SS3.p1.1.m1.1.1" xref="S5.SS3.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS3.p1.1.m1.1b"><ci id="S5.SS3.p1.1.m1.1.1.cmml" xref="S5.SS3.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p1.1.m1.1c">p</annotation></semantics></math>’s impact on mathematical reasoning, we employ the GS8MK Chain-of-Thought (GSM8K COT) dataset, which consists of 8.5K high-quality, linguistically diverse grade school math word problems. We consider this a rigorous, falsifiable, quantitative test of reasoning that 7B parameter models can somewhat manage, albeit with significant performance degradations at higher temperatures. Our prompting set-up is as follows:</p>
</div>
<div id="S5.SS3.p2" class="ltx_para">
<p id="S5.SS3.p2.1" class="ltx_p"><span id="S5.SS3.p2.1.1" class="ltx_text ltx_font_bold">8-shot prompting.</span> Initial experiments revealed that the baseline Mistral 7B model at temperature 1 with zero-shot prompting was unable to answer any question correctly (0% accuracy). Few-shot prompting proved necessary to obtain statistically significant and meaningful comparisons. We chose 8-shot prompting, as it is the recommended baseline on the EleutherAI Evaluations Harness and widely used in reporting GSM8K COT results.</p>
</div>
<div id="S5.SS3.p3" class="ltx_para">
<p id="S5.SS3.p3.1" class="ltx_p"><span id="S5.SS3.p3.1.1" class="ltx_text ltx_font_bold">Chain-of-Thought (COT).</span> We utilize Chain-of-Thought prompting for two reasons. First, baseline Mistral 7B scored too low on GSM8K 8-shot for meaningful comparisons ( 25%), while GSM8K COT 8-shot yielded a more helpful baseline of  38% at temperature 1. Second, COT more accurately simulates real-world environments that require multi-step reasoning.</p>
</div>
<div id="S5.SS3.p4" class="ltx_para">
<p id="S5.SS3.p4.1" class="ltx_p"><span id="S5.SS3.p4.1.1" class="ltx_text ltx_font_bold">Exact Match.</span> We primarily report Exact Match scores, in line with standard reporting practices, although Flexible Match scores generally favor min-<math id="S5.SS3.p4.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS3.p4.1.m1.1a"><mi id="S5.SS3.p4.1.m1.1.1" xref="S5.SS3.p4.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS3.p4.1.m1.1b"><ci id="S5.SS3.p4.1.m1.1.1.cmml" xref="S5.SS3.p4.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS3.p4.1.m1.1c">p</annotation></semantics></math> even more than Exact Match scores.</p>
</div>
</section>
<section id="S5.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.4 </span>Creative Writing (AlpacaEval Creative Writing)</h3>

<div id="S5.SS4.p1" class="ltx_para">
<p id="S5.SS4.p1.1" class="ltx_p">To evaluate min-<math id="S5.SS4.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS4.p1.1.m1.1a"><mi id="S5.SS4.p1.1.m1.1.1" xref="S5.SS4.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p1.1.m1.1b"><ci id="S5.SS4.p1.1.m1.1.1.cmml" xref="S5.SS4.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p1.1.m1.1c">p</annotation></semantics></math>’s impact on creative writing tasks, we employ the AlpacaEval Creative Writing Benchmark <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib11" title="" class="ltx_ref">2023b</a>)</cite>. AlpacaEval is an LLM-based automatic evaluation framework that is fast, cost-effective, replicable, and validated against 20K human annotations. The Creative Writing Benchmark utilizes a frontier LLM evaluator (in this case, Anthropic’s Claude) to select between two outputs generated from creative writing prompts.</p>
</div>
<div id="S5.SS4.p2" class="ltx_para">
<p id="S5.SS4.p2.2" class="ltx_p">We chose this benchmark due to the significant number of users within the open-source community who have qualitatively reported a preference for min-<math id="S5.SS4.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS4.p2.1.m1.1a"><mi id="S5.SS4.p2.1.m1.1.1" xref="S5.SS4.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p2.1.m1.1b"><ci id="S5.SS4.p2.1.m1.1.1.cmml" xref="S5.SS4.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p2.1.m1.1c">p</annotation></semantics></math> sampling on high-temperature creative writing tasks. This preference has contributed to min-<math id="S5.SS4.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS4.p2.2.m2.1a"><mi id="S5.SS4.p2.2.m2.1.1" xref="S5.SS4.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p2.2.m2.1b"><ci id="S5.SS4.p2.2.m2.1.1.cmml" xref="S5.SS4.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p2.2.m2.1c">p</annotation></semantics></math>’s rapid adoption within the open-source language model community. At the time of writing, we do not have access to other creativity or creative writing evaluation datasets, whether automated or human-annotated.</p>
</div>
<div id="S5.SS4.p3" class="ltx_para">
<p id="S5.SS4.p3.1" class="ltx_p">We note that these AlpacaEval Creative Writing results are primarily reproduced with permission from another independent evaluator within the open-source community <cite class="ltx_cite ltx_citemacro_citep">(Gusev, <a href="#bib.bib6" title="" class="ltx_ref">2023</a>)</cite>. This independent evaluator is one of many open-source community members who have taken interest in min-<math id="S5.SS4.p3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S5.SS4.p3.1.m1.1a"><mi id="S5.SS4.p3.1.m1.1.1" xref="S5.SS4.p3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S5.SS4.p3.1.m1.1b"><ci id="S5.SS4.p3.1.m1.1.1.cmml" xref="S5.SS4.p3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S5.SS4.p3.1.m1.1c">p</annotation></semantics></math> within the months after its release, conducting their own evaluation which we then reproduced via the same procedure.</p>
</div>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Results</h2>

<div id="S6.p1" class="ltx_para">
<p id="S6.p1.2" class="ltx_p">We find that min-<math id="S6.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.p1.1.m1.1a"><mi id="S6.p1.1.m1.1.1" xref="S6.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.p1.1.m1.1b"><ci id="S6.p1.1.m1.1.1.cmml" xref="S6.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.p1.1.m1.1c">p</annotation></semantics></math> sampling outperforms top-<math id="S6.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.p1.2.m2.1a"><mi id="S6.p1.2.m2.1.1" xref="S6.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.p1.2.m2.1b"><ci id="S6.p1.2.m2.1.1.cmml" xref="S6.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.p1.2.m2.1c">p</annotation></semantics></math> across various temperature settings and benchmarks, maintaining coherence while allowing for increased diversity at higher temperatures (&gt;1).</p>
</div>
<section id="S6.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">6.1 </span>Graduate-Level Reasoning (GPQA Main)</h3>

<div id="S6.SS1.p1" class="ltx_para">
<p id="S6.SS1.p1.2" class="ltx_p">Figure <a href="#S6.F3" title="Figure 3 ‣ 6.2 Grade School Math (GSM8K COT) ‣ 6 Results ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> presents the results of the GPQA Main Generative evaluation. We observe that min-<math id="S6.SS1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS1.p1.1.m1.1a"><mi id="S6.SS1.p1.1.m1.1.1" xref="S6.SS1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS1.p1.1.m1.1b"><ci id="S6.SS1.p1.1.m1.1.1.cmml" xref="S6.SS1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS1.p1.1.m1.1c">p</annotation></semantics></math> sampling slightly outperforms top-<math id="S6.SS1.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS1.p1.2.m2.1a"><mi id="S6.SS1.p1.2.m2.1.1" xref="S6.SS1.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS1.p1.2.m2.1b"><ci id="S6.SS1.p1.2.m2.1.1.cmml" xref="S6.SS1.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS1.p1.2.m2.1c">p</annotation></semantics></math> sampling at a temperature of 1.</p>
</div>
<div id="S6.SS1.p2" class="ltx_para">
<p id="S6.SS1.p2.2" class="ltx_p">As the temperature increases, both truncation sampling methods experience a degradation in benchmark performance. However, min-<math id="S6.SS1.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS1.p2.1.m1.1a"><mi id="S6.SS1.p2.1.m1.1.1" xref="S6.SS1.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS1.p2.1.m1.1b"><ci id="S6.SS1.p2.1.m1.1.1.cmml" xref="S6.SS1.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS1.p2.1.m1.1c">p</annotation></semantics></math>’s performance degradation is noticeably lower compared to top-<math id="S6.SS1.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS1.p2.2.m2.1a"><mi id="S6.SS1.p2.2.m2.1.1" xref="S6.SS1.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS1.p2.2.m2.1b"><ci id="S6.SS1.p2.2.m2.1.1.cmml" xref="S6.SS1.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS1.p2.2.m2.1c">p</annotation></semantics></math>, indicating that it more reliably answers graduate-level reasoning questions at higher temperatures.</p>
</div>
<div id="S6.SS1.p3" class="ltx_para">
<p id="S6.SS1.p3.2" class="ltx_p">It is notable that min-<math id="S6.SS1.p3.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS1.p3.1.m1.1a"><mi id="S6.SS1.p3.1.m1.1.1" xref="S6.SS1.p3.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS1.p3.1.m1.1b"><ci id="S6.SS1.p3.1.m1.1.1.cmml" xref="S6.SS1.p3.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS1.p3.1.m1.1c">p</annotation></semantics></math> sampling exhibits a slight improvement in reasoning performance between temperatures 1 and 3. This suggests that the increased diversity in token selection introduced by min-<math id="S6.SS1.p3.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS1.p3.2.m2.1a"><mi id="S6.SS1.p3.2.m2.1.1" xref="S6.SS1.p3.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS1.p3.2.m2.1b"><ci id="S6.SS1.p3.2.m2.1.1.cmml" xref="S6.SS1.p3.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS1.p3.2.m2.1c">p</annotation></semantics></math> at these temperature levels may be beneficial for complex reasoning tasks, as it allows the model to explore a broader range of plausible solutions without significantly sacrificing coherence.</p>
</div>
</section>
<section id="S6.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">6.2 </span>Grade School Math (GSM8K COT)</h3>

<div id="S6.SS2.p1" class="ltx_para">
<p id="S6.SS2.p1.4" class="ltx_p">Figure <a href="#S6.F2" title="Figure 2 ‣ 6.2 Grade School Math (GSM8K COT) ‣ 6 Results ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> shows the results of the GSM8K Chain-of-Thought evaluation. Similar to the GPQA results, min-<math id="S6.SS2.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS2.p1.1.m1.1a"><mi id="S6.SS2.p1.1.m1.1.1" xref="S6.SS2.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS2.p1.1.m1.1b"><ci id="S6.SS2.p1.1.m1.1.1.cmml" xref="S6.SS2.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS2.p1.1.m1.1c">p</annotation></semantics></math> sampling slightly outperforms top-<math id="S6.SS2.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS2.p1.2.m2.1a"><mi id="S6.SS2.p1.2.m2.1.1" xref="S6.SS2.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS2.p1.2.m2.1b"><ci id="S6.SS2.p1.2.m2.1.1.cmml" xref="S6.SS2.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS2.p1.2.m2.1c">p</annotation></semantics></math> sampling at a temperature of 1. As the temperature increases, both truncation sampling methods experience a decline in benchmark performance. However, min-<math id="S6.SS2.p1.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS2.p1.3.m3.1a"><mi id="S6.SS2.p1.3.m3.1.1" xref="S6.SS2.p1.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS2.p1.3.m3.1b"><ci id="S6.SS2.p1.3.m3.1.1.cmml" xref="S6.SS2.p1.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS2.p1.3.m3.1c">p</annotation></semantics></math>’s performance degradation is noticeably lower compared to top-<math id="S6.SS2.p1.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS2.p1.4.m4.1a"><mi id="S6.SS2.p1.4.m4.1.1" xref="S6.SS2.p1.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS2.p1.4.m4.1b"><ci id="S6.SS2.p1.4.m4.1.1.cmml" xref="S6.SS2.p1.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS2.p1.4.m4.1c">p</annotation></semantics></math>, suggesting that it more reliably derives the correct answer to mathematical word problems at higher temperatures.</p>
</div>
<figure id="S6.F2" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x5.png" id="S6.F2.g1" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x6.png" id="S6.F2.g2" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x7.png" id="S6.F2.g3" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x8.png" id="S6.F2.g4" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 2: </span>Results of running min-<math id="S6.F2.3.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.F2.3.m1.1b"><mi id="S6.F2.3.m1.1.1" xref="S6.F2.3.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.F2.3.m1.1c"><ci id="S6.F2.3.m1.1.1.cmml" xref="S6.F2.3.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.F2.3.m1.1d">p</annotation></semantics></math> vs top-<math id="S6.F2.4.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.F2.4.m2.1b"><mi id="S6.F2.4.m2.1.1" xref="S6.F2.4.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.F2.4.m2.1c"><ci id="S6.F2.4.m2.1.1.cmml" xref="S6.F2.4.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.F2.4.m2.1d">p</annotation></semantics></math> on GSM8K. The control method used is pure sampling. A single plot with all results is available in Figure <a href="#A2.F4" title="Figure 4 ‣ Appendix B Full Plots for GSM8K/GPQA results ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>.</figcaption>
</figure>
<figure id="S6.F3" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x9.png" id="S6.F3.g1" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x10.png" id="S6.F3.g2" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x11.png" id="S6.F3.g3" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1"><img src="/html/2407.01082/assets/x12.png" id="S6.F3.g4" class="ltx_graphics ltx_centering ltx_figure_panel ltx_img_landscape" width="369" height="241" alt="Refer to caption"></div>
</div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 3: </span>Results of running min-<math id="S6.F3.3.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.F3.3.m1.1b"><mi id="S6.F3.3.m1.1.1" xref="S6.F3.3.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.F3.3.m1.1c"><ci id="S6.F3.3.m1.1.1.cmml" xref="S6.F3.3.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.F3.3.m1.1d">p</annotation></semantics></math> vs top-<math id="S6.F3.4.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.F3.4.m2.1b"><mi id="S6.F3.4.m2.1.1" xref="S6.F3.4.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.F3.4.m2.1c"><ci id="S6.F3.4.m2.1.1.cmml" xref="S6.F3.4.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.F3.4.m2.1d">p</annotation></semantics></math> on GPQA. The control method used is pure sampling. A single plot with all results is available in Figure <a href="#A2.F5" title="Figure 5 ‣ Appendix B Full Plots for GSM8K/GPQA results ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</figcaption>
</figure>
<div id="S6.SS2.p2" class="ltx_para">
<p id="S6.SS2.p2.2" class="ltx_p">These results demonstrate min-<math id="S6.SS2.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS2.p2.1.m1.1a"><mi id="S6.SS2.p2.1.m1.1.1" xref="S6.SS2.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS2.p2.1.m1.1b"><ci id="S6.SS2.p2.1.m1.1.1.cmml" xref="S6.SS2.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS2.p2.1.m1.1c">p</annotation></semantics></math>’s ability to maintain higher levels of coherence and accuracy in multi-step reasoning tasks, even when the diversity of token selection is increased through higher temperature settings. This finding aligns with our hypothesis that min-<math id="S6.SS2.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS2.p2.2.m2.1a"><mi id="S6.SS2.p2.2.m2.1.1" xref="S6.SS2.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS2.p2.2.m2.1b"><ci id="S6.SS2.p2.2.m2.1.1.cmml" xref="S6.SS2.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS2.p2.2.m2.1c">p</annotation></semantics></math> sampling can better navigate the creativity-coherence tradeoff in complex reasoning scenarios.</p>
</div>
</section>
<section id="S6.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">6.3 </span>AlpacaEval</h3>

<div id="S6.SS3.p1" class="ltx_para">
<p id="S6.SS3.p1.4" class="ltx_p">Table <a href="#S6.T3" title="Table 3 ‣ 6.3 AlpacaEval ‣ 6 Results ‣ Min P Sampling: Balancing Creativity and Coherence at High Temperature" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a> shows that min-<math id="S6.SS3.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS3.p1.1.m1.1a"><mi id="S6.SS3.p1.1.m1.1.1" xref="S6.SS3.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS3.p1.1.m1.1b"><ci id="S6.SS3.p1.1.m1.1.1.cmml" xref="S6.SS3.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS3.p1.1.m1.1c">p</annotation></semantics></math> displays strong performance on the AlpacaEval Creative Writing Benchmark, especially at <math id="S6.SS3.p1.2.m2.1" class="ltx_Math" alttext="p=0.1" display="inline"><semantics id="S6.SS3.p1.2.m2.1a"><mrow id="S6.SS3.p1.2.m2.1.1" xref="S6.SS3.p1.2.m2.1.1.cmml"><mi id="S6.SS3.p1.2.m2.1.1.2" xref="S6.SS3.p1.2.m2.1.1.2.cmml">p</mi><mo id="S6.SS3.p1.2.m2.1.1.1" xref="S6.SS3.p1.2.m2.1.1.1.cmml">=</mo><mn id="S6.SS3.p1.2.m2.1.1.3" xref="S6.SS3.p1.2.m2.1.1.3.cmml">0.1</mn></mrow><annotation-xml encoding="MathML-Content" id="S6.SS3.p1.2.m2.1b"><apply id="S6.SS3.p1.2.m2.1.1.cmml" xref="S6.SS3.p1.2.m2.1.1"><eq id="S6.SS3.p1.2.m2.1.1.1.cmml" xref="S6.SS3.p1.2.m2.1.1.1"></eq><ci id="S6.SS3.p1.2.m2.1.1.2.cmml" xref="S6.SS3.p1.2.m2.1.1.2">𝑝</ci><cn type="float" id="S6.SS3.p1.2.m2.1.1.3.cmml" xref="S6.SS3.p1.2.m2.1.1.3">0.1</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS3.p1.2.m2.1c">p=0.1</annotation></semantics></math> at <math id="S6.SS3.p1.3.m3.1" class="ltx_Math" alttext="\tau=1.5" display="inline"><semantics id="S6.SS3.p1.3.m3.1a"><mrow id="S6.SS3.p1.3.m3.1.1" xref="S6.SS3.p1.3.m3.1.1.cmml"><mi id="S6.SS3.p1.3.m3.1.1.2" xref="S6.SS3.p1.3.m3.1.1.2.cmml">τ</mi><mo id="S6.SS3.p1.3.m3.1.1.1" xref="S6.SS3.p1.3.m3.1.1.1.cmml">=</mo><mn id="S6.SS3.p1.3.m3.1.1.3" xref="S6.SS3.p1.3.m3.1.1.3.cmml">1.5</mn></mrow><annotation-xml encoding="MathML-Content" id="S6.SS3.p1.3.m3.1b"><apply id="S6.SS3.p1.3.m3.1.1.cmml" xref="S6.SS3.p1.3.m3.1.1"><eq id="S6.SS3.p1.3.m3.1.1.1.cmml" xref="S6.SS3.p1.3.m3.1.1.1"></eq><ci id="S6.SS3.p1.3.m3.1.1.2.cmml" xref="S6.SS3.p1.3.m3.1.1.2">𝜏</ci><cn type="float" id="S6.SS3.p1.3.m3.1.1.3.cmml" xref="S6.SS3.p1.3.m3.1.1.3">1.5</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="S6.SS3.p1.3.m3.1c">\tau=1.5</annotation></semantics></math>, a temperature where top-<math id="S6.SS3.p1.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.SS3.p1.4.m4.1a"><mi id="S6.SS3.p1.4.m4.1.1" xref="S6.SS3.p1.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.SS3.p1.4.m4.1b"><ci id="S6.SS3.p1.4.m4.1.1.cmml" xref="S6.SS3.p1.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.SS3.p1.4.m4.1c">p</annotation></semantics></math> struggles to produce coherent creative writing.</p>
</div>
<figure id="S6.T3" class="ltx_table">
<table id="S6.T3.3" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S6.T3.3.1.1" class="ltx_tr">
<td id="S6.T3.3.1.1.1" class="ltx_td ltx_align_center ltx_border_t">Temp</td>
<td id="S6.T3.3.1.1.2" class="ltx_td ltx_align_center ltx_border_t">Constraints</td>
<td id="S6.T3.3.1.1.3" class="ltx_td ltx_align_center ltx_border_t">Winrate</td>
<td id="S6.T3.3.1.1.4" class="ltx_td ltx_align_center ltx_border_t">Winrate</td>
</tr>
<tr id="S6.T3.3.2.2" class="ltx_tr">
<td id="S6.T3.3.2.2.1" class="ltx_td"></td>
<td id="S6.T3.3.2.2.2" class="ltx_td"></td>
<td id="S6.T3.3.2.2.3" class="ltx_td ltx_align_center">(LC)</td>
<td id="S6.T3.3.2.2.4" class="ltx_td"></td>
</tr>
<tr id="S6.T3.3.3.3" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.3.3.1" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.3.3.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">1.5</span></td>
<td id="S6.T3.3.3.3.2" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.3.3.2.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">min_p=0.1</span></td>
<td id="S6.T3.3.3.3.3" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.3.3.3.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">58.12</span></td>
<td id="S6.T3.3.3.3.4" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.3.3.4.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">56.54</span></td>
</tr>
<tr id="S6.T3.3.4.4" class="ltx_tr">
<td id="S6.T3.3.4.4.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.4.4.1.1" class="ltx_text ltx_font_bold">1.0</span></td>
<td id="S6.T3.3.4.4.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.4.4.2.1" class="ltx_text ltx_font_bold">min_p=0.05</span></td>
<td id="S6.T3.3.4.4.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.4.4.3.1" class="ltx_text ltx_font_bold">55.07</span></td>
<td id="S6.T3.3.4.4.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.4.4.4.1" class="ltx_text ltx_font_bold">52.01</span></td>
</tr>
<tr id="S6.T3.3.5.5" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.5.5.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.5.5.1.1" class="ltx_text" style="background-color:#F2F2F2;">0.8</span></td>
<td id="S6.T3.3.5.5.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.5.5.2.1" class="ltx_text" style="background-color:#F2F2F2;">top_p=0.98</span></td>
<td id="S6.T3.3.5.5.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.5.5.3.1" class="ltx_text" style="background-color:#F2F2F2;">54.65</span></td>
<td id="S6.T3.3.5.5.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.5.5.4.1" class="ltx_text" style="background-color:#F2F2F2;">51.29</span></td>
</tr>
<tr id="S6.T3.3.6.6" class="ltx_tr">
<td id="S6.T3.3.6.6.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.6.6.1.1" class="ltx_text ltx_font_bold">1.0</span></td>
<td id="S6.T3.3.6.6.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.6.6.2.1" class="ltx_text ltx_font_bold">min_p=0.1</span></td>
<td id="S6.T3.3.6.6.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.6.6.3.1" class="ltx_text ltx_font_bold">53.24</span></td>
<td id="S6.T3.3.6.6.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.6.6.4.1" class="ltx_text ltx_font_bold">50.14</span></td>
</tr>
<tr id="S6.T3.3.7.7" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.7.7.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.7.7.1.1" class="ltx_text" style="background-color:#F2F2F2;">1.0</span></td>
<td id="S6.T3.3.7.7.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.7.7.2.1" class="ltx_text" style="background-color:#F2F2F2;">top_p=0.98</span></td>
<td id="S6.T3.3.7.7.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.7.7.3.1" class="ltx_text" style="background-color:#F2F2F2;">53.00</span></td>
<td id="S6.T3.3.7.7.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.7.7.4.1" class="ltx_text" style="background-color:#F2F2F2;">50.43</span></td>
</tr>
<tr id="S6.T3.3.8.8" class="ltx_tr">
<td id="S6.T3.3.8.8.1" class="ltx_td ltx_align_center">1.0</td>
<td id="S6.T3.3.8.8.2" class="ltx_td ltx_align_center">top_p=0.9</td>
<td id="S6.T3.3.8.8.3" class="ltx_td ltx_align_center">52.07</td>
<td id="S6.T3.3.8.8.4" class="ltx_td ltx_align_center">50.07</td>
</tr>
<tr id="S6.T3.3.9.9" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.9.9.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.9.9.1.1" class="ltx_text" style="background-color:#F2F2F2;">0.8</span></td>
<td id="S6.T3.3.9.9.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.9.9.2.1" class="ltx_text" style="background-color:#F2F2F2;">top_p=0.95</span></td>
<td id="S6.T3.3.9.9.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.9.9.3.1" class="ltx_text" style="background-color:#F2F2F2;">51.80</span></td>
<td id="S6.T3.3.9.9.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.9.9.4.1" class="ltx_text" style="background-color:#F2F2F2;">50.22</span></td>
</tr>
<tr id="S6.T3.3.10.10" class="ltx_tr">
<td id="S6.T3.3.10.10.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.10.10.1.1" class="ltx_text ltx_font_bold">1.0</span></td>
<td id="S6.T3.3.10.10.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.10.10.2.1" class="ltx_text ltx_font_bold">min_p=0.02</span></td>
<td id="S6.T3.3.10.10.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.10.10.3.1" class="ltx_text ltx_font_bold">51.62</span></td>
<td id="S6.T3.3.10.10.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.10.10.4.1" class="ltx_text ltx_font_bold">50.43</span></td>
</tr>
<tr id="S6.T3.3.11.11" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.11.11.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.11.11.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">1.0</span></td>
<td id="S6.T3.3.11.11.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.11.11.2.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">min_p=0.02</span></td>
<td id="S6.T3.3.11.11.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.11.11.3.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">51.46</span></td>
<td id="S6.T3.3.11.11.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.11.11.4.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">48.85</span></td>
</tr>
<tr id="S6.T3.3.12.12" class="ltx_tr">
<td id="S6.T3.3.12.12.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.12.12.1.1" class="ltx_text ltx_font_bold">0.8</span></td>
<td id="S6.T3.3.12.12.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.12.12.2.1" class="ltx_text ltx_font_bold">min_p=0.05</span></td>
<td id="S6.T3.3.12.12.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.12.12.3.1" class="ltx_text ltx_font_bold">50.99</span></td>
<td id="S6.T3.3.12.12.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.12.12.4.1" class="ltx_text ltx_font_bold">47.84</span></td>
</tr>
<tr id="S6.T3.3.13.13" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.13.13.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.13.13.1.1" class="ltx_text" style="background-color:#F2F2F2;">0.8</span></td>
<td id="S6.T3.3.13.13.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.13.13.2.1" class="ltx_text" style="background-color:#F2F2F2;">top_p=0.95</span></td>
<td id="S6.T3.3.13.13.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.13.13.3.1" class="ltx_text" style="background-color:#F2F2F2;">50.76</span></td>
<td id="S6.T3.3.13.13.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.13.13.4.1" class="ltx_text" style="background-color:#F2F2F2;">48.78</span></td>
</tr>
<tr id="S6.T3.3.14.14" class="ltx_tr">
<td id="S6.T3.3.14.14.1" class="ltx_td ltx_align_center ltx_border_t">1.0</td>
<td id="S6.T3.3.14.14.2" class="ltx_td ltx_border_t"></td>
<td id="S6.T3.3.14.14.3" class="ltx_td ltx_align_center ltx_border_t">50.0</td>
<td id="S6.T3.3.14.14.4" class="ltx_td ltx_align_center ltx_border_t">50.0</td>
</tr>
<tr id="S6.T3.3.15.15" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.15.15.1" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.15.15.1.1" class="ltx_text" style="background-color:#F2F2F2;">0.0</span></td>
<td id="S6.T3.3.15.15.2" class="ltx_td ltx_border_t"></td>
<td id="S6.T3.3.15.15.3" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.15.15.3.1" class="ltx_text" style="background-color:#F2F2F2;">49.90</span></td>
<td id="S6.T3.3.15.15.4" class="ltx_td ltx_align_center ltx_border_t"><span id="S6.T3.3.15.15.4.1" class="ltx_text" style="background-color:#F2F2F2;">46.64</span></td>
</tr>
<tr id="S6.T3.3.16.16" class="ltx_tr">
<td id="S6.T3.3.16.16.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.16.16.1.1" class="ltx_text ltx_font_bold">1.5</span></td>
<td id="S6.T3.3.16.16.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.16.16.2.1" class="ltx_text ltx_font_bold">min_p=0.05</span></td>
<td id="S6.T3.3.16.16.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.16.16.3.1" class="ltx_text ltx_font_bold">48.13</span></td>
<td id="S6.T3.3.16.16.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.16.16.4.1" class="ltx_text ltx_font_bold">48.57</span></td>
</tr>
<tr id="S6.T3.3.17.17" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.17.17.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.17.17.1.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">1.5</span></td>
<td id="S6.T3.3.17.17.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.17.17.2.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">min_p=0.02</span></td>
<td id="S6.T3.3.17.17.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.17.17.3.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">42.09</span></td>
<td id="S6.T3.3.17.17.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.17.17.4.1" class="ltx_text ltx_font_bold" style="background-color:#F2F2F2;">44.83</span></td>
</tr>
<tr id="S6.T3.3.18.18" class="ltx_tr">
<td id="S6.T3.3.18.18.1" class="ltx_td ltx_align_center">1.0</td>
<td id="S6.T3.3.18.18.2" class="ltx_td ltx_align_center">mirostat_tau=4.0</td>
<td id="S6.T3.3.18.18.3" class="ltx_td ltx_align_center">16.04</td>
<td id="S6.T3.3.18.18.4" class="ltx_td ltx_align_center">16.69</td>
</tr>
<tr id="S6.T3.3.19.19" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.19.19.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.19.19.1.1" class="ltx_text" style="background-color:#F2F2F2;">1.0</span></td>
<td id="S6.T3.3.19.19.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.19.19.2.1" class="ltx_text" style="background-color:#F2F2F2;">mirostat_tau=5.0</span></td>
<td id="S6.T3.3.19.19.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.19.19.3.1" class="ltx_text" style="background-color:#F2F2F2;">16.03</span></td>
<td id="S6.T3.3.19.19.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.19.19.4.1" class="ltx_text" style="background-color:#F2F2F2;">16.40</span></td>
</tr>
<tr id="S6.T3.3.20.20" class="ltx_tr">
<td id="S6.T3.3.20.20.1" class="ltx_td ltx_align_center">1.5</td>
<td id="S6.T3.3.20.20.2" class="ltx_td"></td>
<td id="S6.T3.3.20.20.3" class="ltx_td ltx_align_center">FAIL</td>
<td id="S6.T3.3.20.20.4" class="ltx_td ltx_align_center">FAIL</td>
</tr>
<tr id="S6.T3.3.21.21" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.21.21.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.21.21.1.1" class="ltx_text" style="background-color:#F2F2F2;">1.5</span></td>
<td id="S6.T3.3.21.21.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.21.21.2.1" class="ltx_text" style="background-color:#F2F2F2;">top_p=0.98</span></td>
<td id="S6.T3.3.21.21.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.21.21.3.1" class="ltx_text" style="background-color:#F2F2F2;">FAIL</span></td>
<td id="S6.T3.3.21.21.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.21.21.4.1" class="ltx_text" style="background-color:#F2F2F2;">FAIL</span></td>
</tr>
<tr id="S6.T3.3.22.22" class="ltx_tr">
<td id="S6.T3.3.22.22.1" class="ltx_td ltx_align_center">1.5</td>
<td id="S6.T3.3.22.22.2" class="ltx_td ltx_align_center">top_p=0.95</td>
<td id="S6.T3.3.22.22.3" class="ltx_td ltx_align_center">FAIL</td>
<td id="S6.T3.3.22.22.4" class="ltx_td ltx_align_center">FAIL</td>
</tr>
<tr id="S6.T3.3.23.23" class="ltx_tr" style="background-color:#F2F2F2;">
<td id="S6.T3.3.23.23.1" class="ltx_td ltx_align_center"><span id="S6.T3.3.23.23.1.1" class="ltx_text" style="background-color:#F2F2F2;">1.5</span></td>
<td id="S6.T3.3.23.23.2" class="ltx_td ltx_align_center"><span id="S6.T3.3.23.23.2.1" class="ltx_text" style="background-color:#F2F2F2;">top_p=0.9</span></td>
<td id="S6.T3.3.23.23.3" class="ltx_td ltx_align_center"><span id="S6.T3.3.23.23.3.1" class="ltx_text" style="background-color:#F2F2F2;">FAIL</span></td>
<td id="S6.T3.3.23.23.4" class="ltx_td ltx_align_center"><span id="S6.T3.3.23.23.4.1" class="ltx_text" style="background-color:#F2F2F2;">FAIL</span></td>
</tr>
<tr id="S6.T3.3.24.24" class="ltx_tr">
<td id="S6.T3.3.24.24.1" class="ltx_td ltx_align_center ltx_border_b">1.5</td>
<td id="S6.T3.3.24.24.2" class="ltx_td ltx_align_center ltx_border_b">top_p=0.8</td>
<td id="S6.T3.3.24.24.3" class="ltx_td ltx_align_center ltx_border_b">FAIL</td>
<td id="S6.T3.3.24.24.4" class="ltx_td ltx_align_center ltx_border_b">FAIL</td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 3: </span>AlpacaEval on creative writing prompts with different sampling parameters. LC-winrate is winrate with length control, which alleviates the length biases of LLM-based judging. Min-<math id="S6.T3.2.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S6.T3.2.m1.1b"><mi id="S6.T3.2.m1.1.1" xref="S6.T3.2.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S6.T3.2.m1.1c"><ci id="S6.T3.2.m1.1.1.cmml" xref="S6.T3.2.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S6.T3.2.m1.1d">p</annotation></semantics></math> results are in <span id="S6.T3.5.1" class="ltx_text ltx_font_bold">bold</span>.</figcaption>
</figure>
</section>
</section>
<section id="S7" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">7 </span>Different Min-<math id="S7.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.1.m1.1b"><mi id="S7.1.m1.1.1" xref="S7.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.1.m1.1c"><ci id="S7.1.m1.1.1.cmml" xref="S7.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.1.m1.1d">p</annotation></semantics></math> Configurations</h2>

<div id="S7.p1" class="ltx_para">
<p id="S7.p1.4" class="ltx_p">As of June 2024, users of open-source LLM inference engines generally recommend min-<math id="S7.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p1.1.m1.1a"><mi id="S7.p1.1.m1.1.1" xref="S7.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p1.1.m1.1b"><ci id="S7.p1.1.m1.1.1.cmml" xref="S7.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.1.m1.1c">p</annotation></semantics></math> settings of 0.05-0.1 and temperature  1 for normal usage, temperature 1-1.5 for varied text generation while balancing factual and logical tasks, and min-<math id="S7.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p1.2.m2.1a"><mi id="S7.p1.2.m2.1.1" xref="S7.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p1.2.m2.1b"><ci id="S7.p1.2.m2.1.1.cmml" xref="S7.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.2.m2.1c">p</annotation></semantics></math> 0.05-0.1 with temperature 2-5 for roleplay and creative writing. Higher values of min-<math id="S7.p1.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p1.3.m3.1a"><mi id="S7.p1.3.m3.1.1" xref="S7.p1.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p1.3.m3.1b"><ci id="S7.p1.3.m3.1.1.cmml" xref="S7.p1.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.3.m3.1c">p</annotation></semantics></math> base percentage (e.g., min-<math id="S7.p1.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p1.4.m4.1a"><mi id="S7.p1.4.m4.1.1" xref="S7.p1.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p1.4.m4.1b"><ci id="S7.p1.4.m4.1.1.cmml" xref="S7.p1.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p1.4.m4.1c">p</annotation></semantics></math> = 0.9) result in significantly reduced degradation for temperature scaling values &gt;20.</p>
</div>
<div id="S7.p2" class="ltx_para">
<p id="S7.p2.3" class="ltx_p"><span id="S7.p2.3.1" class="ltx_text ltx_font_bold">Sampling Order.</span> Min-<math id="S7.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p2.1.m1.1a"><mi id="S7.p2.1.m1.1.1" xref="S7.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p2.1.m1.1b"><ci id="S7.p2.1.m1.1.1.cmml" xref="S7.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p2.1.m1.1c">p</annotation></semantics></math> can be used with any other sampling method, in any order. Applying min-<math id="S7.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p2.2.m2.1a"><mi id="S7.p2.2.m2.1.1" xref="S7.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p2.2.m2.1b"><ci id="S7.p2.2.m2.1.1.cmml" xref="S7.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p2.2.m2.1c">p</annotation></semantics></math> after temperature scaling is recommended for steerability. Applying min-<math id="S7.p2.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S7.p2.3.m3.1a"><mi id="S7.p2.3.m3.1.1" xref="S7.p2.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S7.p2.3.m3.1b"><ci id="S7.p2.3.m3.1.1.cmml" xref="S7.p2.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S7.p2.3.m3.1c">p</annotation></semantics></math> before temperature scaling results in the same token pool as temperature 1, with temperature only shifting relative probabilities, leading to higher accuracy for high-certainty tokens but constrained diversity.</p>
</div>
<div id="S7.p3" class="ltx_para">
<p id="S7.p3.1" class="ltx_p"><span id="S7.p3.1.1" class="ltx_text ltx_font_bold">Combining Sampling Methods.</span> Combining multiple sampling methods is possible, it has not been thoroughly tested, and no particular benefits have been observed or reported, likely because sampling methods are designed to be used in isolation.</p>
</div>
</section>
<section id="S8" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">8 </span>Related Work</h2>

<div id="S8.p1" class="ltx_para">
<p id="S8.p1.2" class="ltx_p">In addition to temperature sampling and top-<math id="S8.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S8.p1.1.m1.1a"><mi id="S8.p1.1.m1.1.1" xref="S8.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S8.p1.1.m1.1b"><ci id="S8.p1.1.m1.1.1.cmml" xref="S8.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p1.1.m1.1c">p</annotation></semantics></math> sampling, several alternative sampling methods have been proposed, which <cite class="ltx_cite ltx_citemacro_citet">Shi et al. (<a href="#bib.bib15" title="" class="ltx_ref">2024</a>)</cite> classify as deterministic and stochastic. This section focuses on stochastic sampling methods, as min-<math id="S8.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S8.p1.2.m2.1a"><mi id="S8.p1.2.m2.1.1" xref="S8.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S8.p1.2.m2.1b"><ci id="S8.p1.2.m2.1.1.cmml" xref="S8.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p1.2.m2.1c">p</annotation></semantics></math> sampling falls under this category.</p>
</div>
<div id="S8.p2" class="ltx_para">
<p id="S8.p2.2" class="ltx_p"><span id="S8.p2.1.1" class="ltx_text ltx_font_bold">Top-<math id="S8.p2.1.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S8.p2.1.1.m1.1a"><mi id="S8.p2.1.1.m1.1.1" xref="S8.p2.1.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S8.p2.1.1.m1.1b"><ci id="S8.p2.1.1.m1.1.1.cmml" xref="S8.p2.1.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p2.1.1.m1.1c">k</annotation></semantics></math> sampling</span> <cite class="ltx_cite ltx_citemacro_citep">(Fan et al., <a href="#bib.bib4" title="" class="ltx_ref">2018</a>)</cite> is a truncation-based method that restricts the sampling pool to the <math id="S8.p2.2.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S8.p2.2.m1.1a"><mi id="S8.p2.2.m1.1.1" xref="S8.p2.2.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S8.p2.2.m1.1b"><ci id="S8.p2.2.m1.1.1.cmml" xref="S8.p2.2.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p2.2.m1.1c">k</annotation></semantics></math> most probable tokens at each step.</p>
</div>
<div id="S8.p3" class="ltx_para">
<p id="S8.p3.2" class="ltx_p"><span id="S8.p3.2.1" class="ltx_text ltx_font_bold">Mirostat sampling</span> <cite class="ltx_cite ltx_citemacro_citep">(Basu et al., <a href="#bib.bib2" title="" class="ltx_ref">2021</a>)</cite> builds upon top-<math id="S8.p3.1.m1.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S8.p3.1.m1.1a"><mi id="S8.p3.1.m1.1.1" xref="S8.p3.1.m1.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S8.p3.1.m1.1b"><ci id="S8.p3.1.m1.1.1.cmml" xref="S8.p3.1.m1.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p3.1.m1.1c">k</annotation></semantics></math> sampling by dynamically adjusting the value of <math id="S8.p3.2.m2.1" class="ltx_Math" alttext="k" display="inline"><semantics id="S8.p3.2.m2.1a"><mi id="S8.p3.2.m2.1.1" xref="S8.p3.2.m2.1.1.cmml">k</mi><annotation-xml encoding="MathML-Content" id="S8.p3.2.m2.1b"><ci id="S8.p3.2.m2.1.1.cmml" xref="S8.p3.2.m2.1.1">𝑘</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p3.2.m2.1c">k</annotation></semantics></math> at each generation step to maintain a target perplexity level, enabling the generation of coherent and diverse text without requiring manual tuning of the sampling parameters.</p>
</div>
<div id="S8.p4" class="ltx_para">
<p id="S8.p4.1" class="ltx_p"><span id="S8.p4.1.1" class="ltx_text ltx_font_bold">Locally typical sampling</span> <cite class="ltx_cite ltx_citemacro_citep">(Meister et al., <a href="#bib.bib12" title="" class="ltx_ref">2023</a>)</cite> is a sampling method that limits the sampling pool to words with negative log-probability within a certain range from the conditional entropy of the distribution at that time step, which helps to avoid both highly unlikely and overly generic tokens.</p>
</div>
<div id="S8.p5" class="ltx_para">
<p id="S8.p5.1" class="ltx_p"><span id="S8.p5.1.1" class="ltx_text ltx_font_bold">Contrastive decoding</span> <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib10" title="" class="ltx_ref">2023a</a>)</cite> uses an auxiliary language model to penalize tokens that are unlikely under the main language model, which guides the selection of higher-quality text while ensuring the generated output remains plausible.</p>
</div>
<div id="S8.p6" class="ltx_para">
<p id="S8.p6.1" class="ltx_p"><math id="S8.p6.1.m1.1" class="ltx_Math" alttext="\eta" display="inline"><semantics id="S8.p6.1.m1.1a"><mi id="S8.p6.1.m1.1.1" xref="S8.p6.1.m1.1.1.cmml">η</mi><annotation-xml encoding="MathML-Content" id="S8.p6.1.m1.1b"><ci id="S8.p6.1.m1.1.1.cmml" xref="S8.p6.1.m1.1.1">𝜂</ci></annotation-xml><annotation encoding="application/x-tex" id="S8.p6.1.m1.1c">\eta</annotation></semantics></math><span id="S8.p6.1.1" class="ltx_text ltx_font_bold">-sampling</span> <cite class="ltx_cite ltx_citemacro_citep">(Hewitt et al., <a href="#bib.bib7" title="" class="ltx_ref">2022</a>)</cite> is an entropy-dependent truncation method that sets a probability threshold based on the entropy of the conditional distribution at each step, truncating words below this threshold.</p>
</div>
</section>
<section id="S9" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">9 </span>Conclusion</h2>

<div id="S9.p1" class="ltx_para">
<p id="S9.p1.5" class="ltx_p">In this work, we introduced min-<math id="S9.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p1.1.m1.1a"><mi id="S9.p1.1.m1.1.1" xref="S9.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p1.1.m1.1b"><ci id="S9.p1.1.m1.1.1.cmml" xref="S9.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p1.1.m1.1c">p</annotation></semantics></math> sampling, a novel dynamic truncation sampling method that aims to balance creativity and coherence in text generation from large language models, particularly at high temperatures. Through experiments on multiple LLM benchmarks, including graduate-level reasoning (GPQA), grade school math (GSM8K), and creative writing (AlpacaEval), we demonstrated that min-<math id="S9.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p1.2.m2.1a"><mi id="S9.p1.2.m2.1.1" xref="S9.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p1.2.m2.1b"><ci id="S9.p1.2.m2.1.1.cmml" xref="S9.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p1.2.m2.1c">p</annotation></semantics></math> outperforms the popular top-<math id="S9.p1.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p1.3.m3.1a"><mi id="S9.p1.3.m3.1.1" xref="S9.p1.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p1.3.m3.1b"><ci id="S9.p1.3.m3.1.1.cmml" xref="S9.p1.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p1.3.m3.1c">p</annotation></semantics></math> sampling method in maintaining coherence while enabling more diverse outputs at temperatures above 1. The rapid adoption of min-<math id="S9.p1.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p1.4.m4.1a"><mi id="S9.p1.4.m4.1.1" xref="S9.p1.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p1.4.m4.1b"><ci id="S9.p1.4.m4.1.1.cmml" xref="S9.p1.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p1.4.m4.1c">p</annotation></semantics></math> by the open-source LLM community further validates its practical utility and potential. By effectively navigating the trade-off between creativity and coherence, min-<math id="S9.p1.5.m5.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p1.5.m5.1a"><mi id="S9.p1.5.m5.1.1" xref="S9.p1.5.m5.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p1.5.m5.1b"><ci id="S9.p1.5.m5.1.1.cmml" xref="S9.p1.5.m5.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p1.5.m5.1c">p</annotation></semantics></math> sampling opens up new possibilities for high-quality open-ended text generation.</p>
</div>
<div id="S9.p2" class="ltx_para">
<p id="S9.p2.2" class="ltx_p"><span id="S9.p2.2.1" class="ltx_text ltx_font_bold">Further Work.</span> Future research directions for min-<math id="S9.p2.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p2.1.m1.1a"><mi id="S9.p2.1.m1.1.1" xref="S9.p2.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p2.1.m1.1b"><ci id="S9.p2.1.m1.1.1.cmml" xref="S9.p2.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p2.1.m1.1c">p</annotation></semantics></math> sampling include investigating its performance on a wider range of NLP tasks, such as dialogue generation, story completion, and language translation. A more rigorous theoretical analysis of min-<math id="S9.p2.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.p2.2.m2.1a"><mi id="S9.p2.2.m2.1.1" xref="S9.p2.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.p2.2.m2.1b"><ci id="S9.p2.2.m2.1.1.cmml" xref="S9.p2.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.p2.2.m2.1c">p</annotation></semantics></math>’s behavior may provide valuable insights for developing even more effective sampling strategies.</p>
</div>
<section id="S9.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">9.1 </span>Limitations</h3>

<div id="S9.SS1.p1" class="ltx_para">
<p id="S9.SS1.p1.4" class="ltx_p">Although our study of min-<math id="S9.SS1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.SS1.p1.1.m1.1a"><mi id="S9.SS1.p1.1.m1.1.1" xref="S9.SS1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.SS1.p1.1.m1.1b"><ci id="S9.SS1.p1.1.m1.1.1.cmml" xref="S9.SS1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.SS1.p1.1.m1.1c">p</annotation></semantics></math> has shown promising results, it is not without limitations. In particular, we focus on a single model architecture (Mistral 7B) and a limited set of benchmarks. To establish the generality of min-<math id="S9.SS1.p1.2.m2.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.SS1.p1.2.m2.1a"><mi id="S9.SS1.p1.2.m2.1.1" xref="S9.SS1.p1.2.m2.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.SS1.p1.2.m2.1b"><ci id="S9.SS1.p1.2.m2.1.1.cmml" xref="S9.SS1.p1.2.m2.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.SS1.p1.2.m2.1c">p</annotation></semantics></math>’s benefits, evaluating its performance across a diverse range of model sizes, architectures, and tasks is important. Furthermore, the AlpacaEval Creative Writing benchmark, while cost-effective and validated against human judgments, may not fully capture all aspects of creativity in text generation. Human evaluation studies could provide additional insights into the quality and creativity of text generated using min-<math id="S9.SS1.p1.3.m3.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.SS1.p1.3.m3.1a"><mi id="S9.SS1.p1.3.m3.1.1" xref="S9.SS1.p1.3.m3.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.SS1.p1.3.m3.1b"><ci id="S9.SS1.p1.3.m3.1.1.cmml" xref="S9.SS1.p1.3.m3.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.SS1.p1.3.m3.1c">p</annotation></semantics></math> compared to other sampling methods using the same models. At the same time, the results position min-<math id="S9.SS1.p1.4.m4.1" class="ltx_Math" alttext="p" display="inline"><semantics id="S9.SS1.p1.4.m4.1a"><mi id="S9.SS1.p1.4.m4.1.1" xref="S9.SS1.p1.4.m4.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="S9.SS1.p1.4.m4.1b"><ci id="S9.SS1.p1.4.m4.1.1.cmml" xref="S9.SS1.p1.4.m4.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="S9.SS1.p1.4.m4.1c">p</annotation></semantics></math> as a computationally cheap alternative to nucleus sampling, especially when creativity and higher diversity of the generated text are important.</p>
</div>
</section>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ackley et al. (1985)</span>
<span class="ltx_bibblock">
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. 1985.

</span>
<span class="ltx_bibblock">A learning algorithm for boltzmann machines.

</span>
<span class="ltx_bibblock"><em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">Cognitive science</em>, 9(1):147–169.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Basu et al. (2021)</span>
<span class="ltx_bibblock">
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2007.14966" title="" class="ltx_ref ltx_href">Mirostat: A neural text decoding algorithm that directly controls perplexity</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2007.14966.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cobbe et al. (2021)</span>
<span class="ltx_bibblock">
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2110.14168" title="" class="ltx_ref ltx_href">Training verifiers to solve math word problems</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2110.14168.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Fan et al. (2018)</span>
<span class="ltx_bibblock">
Angela Fan, Mike Lewis, and Yann Dauphin. 2018.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/P18-1082" title="" class="ltx_ref ltx_href">Hierarchical neural story generation</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pages 889–898, Melbourne, Australia. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gao et al. (2023)</span>
<span class="ltx_bibblock">
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.5281/zenodo.10256836" title="" class="ltx_ref ltx_href">A framework for few-shot language model evaluation</a>.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gusev (2023)</span>
<span class="ltx_bibblock">
Ilya Gusev. 2023.

</span>
<span class="ltx_bibblock">Quantitative evaluation of modern llm sampling techniques.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://github.com/IlyaGusev/quest" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/IlyaGusev/quest</a>.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hewitt et al. (2022)</span>
<span class="ltx_bibblock">
John Hewitt, Christopher Manning, and Percy Liang. 2022.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2022.findings-emnlp.249" title="" class="ltx_ref ltx_href">Truncation sampling as language model desmoothing</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">Findings of the Association for Computational Linguistics: EMNLP 2022</em>, pages 3414–3427, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Holtzman et al. (2020)</span>
<span class="ltx_bibblock">
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.

</span>
<span class="ltx_bibblock">The curious case of neural text degeneration.

</span>
<span class="ltx_bibblock"><em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">ICLR 2020</em>.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jiang et al. (2023)</span>
<span class="ltx_bibblock">
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2310.06825" title="" class="ltx_ref ltx_href">Mistral 7b</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2310.06825.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023a)</span>
<span class="ltx_bibblock">
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023a.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.18653/v1/2023.acl-long.687" title="" class="ltx_ref ltx_href">Contrastive decoding: Open-ended text generation as optimization</a>.

</span>
<span class="ltx_bibblock">In <em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, pages 12286–12312, Toronto, Canada. Association for Computational Linguistics.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023b)</span>
<span class="ltx_bibblock">
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023b.

</span>
<span class="ltx_bibblock">Alpacaeval: An automatic evaluator of instruction-following models.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://github.com/tatsu-lab/alpaca_eval" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/tatsu-lab/alpaca_eval</a>.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Meister et al. (2023)</span>
<span class="ltx_bibblock">
Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://doi.org/10.1162/tacl_a_00536" title="" class="ltx_ref ltx_href">Locally typical sampling</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">Transactions of the Association for Computational Linguistics</em>, 11:102–121.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">OpenAI API (2024)</span>
<span class="ltx_bibblock">
OpenAI API. 2024.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://platform.openai.com/docs/api-reference/completions/create" title="" class="ltx_ref ltx_href">Openai api reference - completions</a>.

</span>
<span class="ltx_bibblock">Accessed: 2024-06-10.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Rein et al. (2023)</span>
<span class="ltx_bibblock">
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023.

</span>
<span class="ltx_bibblock"><a target="_blank" href="https://arxiv.org/abs/2311.12022" title="" class="ltx_ref ltx_href">Gpqa: A graduate-level google-proof q&amp;a benchmark</a>.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">Preprint</em>, arXiv:2311.12022.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Shi et al. (2024)</span>
<span class="ltx_bibblock">
Chufan Shi, Haoran Yang, Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam. 2024.

</span>
<span class="ltx_bibblock">A thorough examination of decoding methods in the era of llms.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.06925</em>.

</span>
</li>
</ul>
</section>
<div id="p2" class="ltx_para">
<p id="p2.1" class="ltx_p"><span id="p2.1.1" class="ltx_text ltx_font_bold">AI Assistance.</span> AI Assistance was used to check for typos and grammatical errors in writing, as well as propose better phrasings for existing text.</p>
</div>
<div class="ltx_pagination ltx_role_newpage"></div>
<div class="ltx_pagination ltx_role_newpage"></div>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span>Min-<math id="A1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="A1.1.m1.1b"><mi id="A1.1.m1.1.1" xref="A1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="A1.1.m1.1c"><ci id="A1.1.m1.1.1.cmml" xref="A1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.1.m1.1d">p</annotation></semantics></math> Implementation and Documentation</h2>

<div id="A1.p1" class="ltx_para">
<p id="A1.p1.1" class="ltx_p">Below is the implementation code for min-<math id="A1.p1.1.m1.1" class="ltx_Math" alttext="p" display="inline"><semantics id="A1.p1.1.m1.1a"><mi id="A1.p1.1.m1.1.1" xref="A1.p1.1.m1.1.1.cmml">p</mi><annotation-xml encoding="MathML-Content" id="A1.p1.1.m1.1b"><ci id="A1.p1.1.m1.1.1.cmml" xref="A1.p1.1.m1.1.1">𝑝</ci></annotation-xml><annotation encoding="application/x-tex" id="A1.p1.1.m1.1c">p</annotation></semantics></math> truncation sampling as detailed in the Huggingface Transformers library, with range exception handling and keeping minimum tokens to prevent errors.</p>
</div>
<div id="A1.p2" class="ltx_para">
<p id="A1.p2.1" class="ltx_p">This implementation code, along with other similar implementations in other open-source inference engines, logs of automated evaluations for GPQA, GSM8K Chain-of-Thought and AlpacaEval Creative Writing is available at <a target="_blank" href="https://github.com/menhguin/minp_paper/" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/menhguin/minp_paper/</a>.</p>
</div>
<div id="A1.p3" class="ltx_para">
<div id="A1.p3.1" class="ltx_listing ltx_lst_language_Python ltx_lst_numbers_left ltx_lstlisting ltx_framed ltx_framed_rectangle ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,Y2xhc3MgTWluUExvZ2l0c1dhcnBlcihMb2dpdHNXYXJwZXIpOgogICAgZGVmIF9faW5pdF9fKHNlbGYsIG1pbl9wOiBmbG9hdCwgZmlsdGVyX3ZhbHVlOiBmbG9hdCA9IC1mbG9hdCgiSW5mIiksIG1pbl90b2tlbnNfdG9fa2VlcDogaW50ID0gMSk6CiAgICAgICAgaWYgbWluX3AgPCAwIG9yIG1pbl9wID4gMS4wOgogICAgICAgICAgICByYWlzZSBWYWx1ZUVycm9yKGYiYG1pbl9wYCBoYXMgdG8gYmUgYSBmbG9hdCA+PSAwIGFuZCA8PSAxLCBidXQgaXMge21pbl9wfSIpCiAgICAgICAgaWYgbm90IGlzaW5zdGFuY2UobWluX3Rva2Vuc190b19rZWVwLCBpbnQpIG9yIChtaW5fdG9rZW5zX3RvX2tlZXAgPCAxKToKICAgICAgICAgICAgcmFpc2UgVmFsdWVFcnJvcihmImBtaW5fdG9rZW5zX3RvX2tlZXBgIGhhcyB0byBiZSBhIHBvc2l0aXZlIGludGVnZXIsIGJ1dCBpcyB7bWluX3Rva2Vuc190b19rZWVwfSIpCgogICAgICAgIHNlbGYubWluX3AgPSBtaW5fcAogICAgICAgIHNlbGYuZmlsdGVyX3ZhbHVlID0gZmlsdGVyX3ZhbHVlCiAgICAgICAgc2VsZi5taW5fdG9rZW5zX3RvX2tlZXAgPSBtaW5fdG9rZW5zX3RvX2tlZXAKCiAgICBkZWYgX19jYWxsX18oc2VsZiwgaW5wdXRfaWRzOiB0b3JjaC5Mb25nVGVuc29yLCBzY29yZXM6IHRvcmNoLkZsb2F0VGVuc29yKSAtPiB0b3JjaC5GbG9hdFRlbnNvcjoKICAgICAgICAjIENvbnZlcnQgbG9naXRzIHRvIHByb2JhYmlsaXRpZXMKICAgICAgICBwcm9icyA9IHRvcmNoLnNvZnRtYXgoc2NvcmVzLCBkaW09LTEpCiAgICAgICAgIyBHZXQgdGhlIHByb2JhYmlsaXR5IG9mIHRoZSB0b3AgdG9rZW4gZm9yIGVhY2ggc2VxdWVuY2UgaW4gdGhlIGJhdGNoCiAgICAgICAgdG9wX3Byb2JzLCBfID0gcHJvYnMubWF4KGRpbT0tMSwga2VlcGRpbT1UcnVlKQogICAgICAgICMgQ2FsY3VsYXRlIHRoZSBhY3R1YWwgbWluX3AgdGhyZXNob2xkIGJ5IHNjYWxpbmcgbWluX3Agd2l0aCB0aGUgdG9wIHRva2VuJ3MgcHJvYmFiaWxpdHkKICAgICAgICBzY2FsZWRfbWluX3AgPSBzZWxmLm1pbl9wICogdG9wX3Byb2JzCiAgICAgICAgIyBDcmVhdGUgYSBtYXNrIGZvciB0b2tlbnMgdGhhdCBoYXZlIGEgcHJvYmFiaWxpdHkgbGVzcyB0aGFuIHRoZSBzY2FsZWQgbWluX3AKICAgICAgICB0b2tlbnNfdG9fcmVtb3ZlID0gcHJvYnMgPCBzY2FsZWRfbWluX3AKCiAgICAgICAgc29ydGVkX2luZGljZXMgPSB0b3JjaC5hcmdzb3J0KHNjb3JlcywgZGVzY2VuZGluZz1UcnVlLCBkaW09LTEpCiAgICAgICAgc29ydGVkX2luZGljZXNfdG9fcmVtb3ZlID0gdG9yY2guZ2F0aGVyKHRva2Vuc190b19yZW1vdmUsIGRpbT0tMSwgaW5kZXg9c29ydGVkX2luZGljZXMpCiAgICAgICAgIyBLZWVwIGF0IGxlYXN0IG1pbl90b2tlbnNfdG9fa2VlcAogICAgICAgIHNvcnRlZF9pbmRpY2VzX3RvX3JlbW92ZVsuLi4sIDogc2VsZi5taW5fdG9rZW5zX3RvX2tlZXBdID0gRmFsc2UKCiAgICAgICAgaW5kaWNlc190b19yZW1vdmUgPSBzb3J0ZWRfaW5kaWNlc190b19yZW1vdmUuc2NhdHRlcigxLCBzb3J0ZWRfaW5kaWNlcywgc29ydGVkX2luZGljZXNfdG9fcmVtb3ZlKQogICAgICAgIHNjb3Jlc19wcm9jZXNzZWQgPSBzY29yZXMubWFza2VkX2ZpbGwoaW5kaWNlc190b19yZW1vdmUsIHNlbGYuZmlsdGVyX3ZhbHVlKQogICAgICAgIHJldHVybiBzY29yZXNfcHJvY2Vzc2Vk" download="">⬇</a></div>
<div id="lstnumberx1" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">1</span><span id="lstnumberx1.1" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">class</span><span id="lstnumberx1.2" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx1.3" class="ltx_text ltx_lst_identifier ltx_font_typewriter">MinPLogitsWarper</span><span id="lstnumberx1.4" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx1.5" class="ltx_text ltx_lst_identifier ltx_font_typewriter">LogitsWarper</span><span id="lstnumberx1.6" class="ltx_text ltx_font_typewriter">):</span>
</div>
<div id="lstnumberx2" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">2</span><span id="lstnumberx2.1" class="ltx_text ltx_lst_space ltx_font_typewriter">    </span><span id="lstnumberx2.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">def</span><span id="lstnumberx2.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">__init__</span><span id="lstnumberx2.5" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx2.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx2.7" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx2.8" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.9" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_p</span><span id="lstnumberx2.10" class="ltx_text ltx_font_typewriter">:</span><span id="lstnumberx2.11" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.12" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">float</span><span id="lstnumberx2.13" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx2.14" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.15" class="ltx_text ltx_lst_identifier ltx_font_typewriter">filter_value</span><span id="lstnumberx2.16" class="ltx_text ltx_font_typewriter">:</span><span id="lstnumberx2.17" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.18" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">float</span><span id="lstnumberx2.19" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.20" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx2.21" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.22" class="ltx_text ltx_font_typewriter">-</span><span id="lstnumberx2.23" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">float</span><span id="lstnumberx2.24" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx2.25" class="ltx_text ltx_lst_string ltx_font_typewriter"><span id="lstnumberx2.25.1" class="ltx_text" style="color:#FF0000;">"Inf"</span></span><span id="lstnumberx2.26" class="ltx_text ltx_font_typewriter">),</span><span id="lstnumberx2.27" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.28" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_tokens_to_keep</span><span id="lstnumberx2.29" class="ltx_text ltx_font_typewriter">:</span><span id="lstnumberx2.30" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.31" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">int</span><span id="lstnumberx2.32" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.33" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx2.34" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx2.35" class="ltx_text ltx_font_typewriter">1):</span>
</div>
<div id="lstnumberx3" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">3</span><span id="lstnumberx3.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx3.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">if</span><span id="lstnumberx3.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_p</span><span id="lstnumberx3.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.6" class="ltx_text ltx_font_typewriter">&lt;</span><span id="lstnumberx3.7" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.8" class="ltx_text ltx_font_typewriter">0</span><span id="lstnumberx3.9" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.10" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">or</span><span id="lstnumberx3.11" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.12" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_p</span><span id="lstnumberx3.13" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.14" class="ltx_text ltx_font_typewriter">&gt;</span><span id="lstnumberx3.15" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx3.16" class="ltx_text ltx_font_typewriter">1.0:</span>
</div>
<div id="lstnumberx4" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">4</span><span id="lstnumberx4.1" class="ltx_text ltx_lst_space ltx_font_typewriter">            </span><span id="lstnumberx4.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">raise</span><span id="lstnumberx4.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx4.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">ValueError</span><span id="lstnumberx4.5" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx4.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">f</span><span id="lstnumberx4.7" class="ltx_text ltx_lst_string ltx_font_typewriter"><span id="lstnumberx4.7.1" class="ltx_text" style="color:#FF0000;">"‘min_p‘<span id="lstnumberx4.7.1.1" class="ltx_text ltx_lst_space"> </span>has<span id="lstnumberx4.7.1.2" class="ltx_text ltx_lst_space"> </span>to<span id="lstnumberx4.7.1.3" class="ltx_text ltx_lst_space"> </span>be<span id="lstnumberx4.7.1.4" class="ltx_text ltx_lst_space"> </span>a<span id="lstnumberx4.7.1.5" class="ltx_text ltx_lst_space"> </span>float<span id="lstnumberx4.7.1.6" class="ltx_text ltx_lst_space"> </span>&gt;=<span id="lstnumberx4.7.1.7" class="ltx_text ltx_lst_space"> </span>0<span id="lstnumberx4.7.1.8" class="ltx_text ltx_lst_space"> </span>and<span id="lstnumberx4.7.1.9" class="ltx_text ltx_lst_space"> </span>&lt;=<span id="lstnumberx4.7.1.10" class="ltx_text ltx_lst_space"> </span>1,<span id="lstnumberx4.7.1.11" class="ltx_text ltx_lst_space"> </span>but<span id="lstnumberx4.7.1.12" class="ltx_text ltx_lst_space"> </span>is<span id="lstnumberx4.7.1.13" class="ltx_text ltx_lst_space"> </span>{min_p}"</span></span><span id="lstnumberx4.8" class="ltx_text ltx_font_typewriter">)</span>
</div>
<div id="lstnumberx5" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">5</span><span id="lstnumberx5.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx5.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">if</span><span id="lstnumberx5.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.4" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">not</span><span id="lstnumberx5.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.6" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">isinstance</span><span id="lstnumberx5.7" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx5.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_tokens_to_keep</span><span id="lstnumberx5.9" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx5.10" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.11" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">int</span><span id="lstnumberx5.12" class="ltx_text ltx_font_typewriter">)</span><span id="lstnumberx5.13" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.14" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">or</span><span id="lstnumberx5.15" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.16" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx5.17" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_tokens_to_keep</span><span id="lstnumberx5.18" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.19" class="ltx_text ltx_font_typewriter">&lt;</span><span id="lstnumberx5.20" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx5.21" class="ltx_text ltx_font_typewriter">1):</span>
</div>
<div id="lstnumberx6" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">6</span><span id="lstnumberx6.1" class="ltx_text ltx_lst_space ltx_font_typewriter">            </span><span id="lstnumberx6.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">raise</span><span id="lstnumberx6.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx6.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">ValueError</span><span id="lstnumberx6.5" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx6.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">f</span><span id="lstnumberx6.7" class="ltx_text ltx_lst_string ltx_font_typewriter"><span id="lstnumberx6.7.1" class="ltx_text" style="color:#FF0000;">"‘min_tokens_to_keep‘<span id="lstnumberx6.7.1.1" class="ltx_text ltx_lst_space"> </span>has<span id="lstnumberx6.7.1.2" class="ltx_text ltx_lst_space"> </span>to<span id="lstnumberx6.7.1.3" class="ltx_text ltx_lst_space"> </span>be<span id="lstnumberx6.7.1.4" class="ltx_text ltx_lst_space"> </span>a<span id="lstnumberx6.7.1.5" class="ltx_text ltx_lst_space"> </span>positive<span id="lstnumberx6.7.1.6" class="ltx_text ltx_lst_space"> </span>integer,<span id="lstnumberx6.7.1.7" class="ltx_text ltx_lst_space"> </span>but<span id="lstnumberx6.7.1.8" class="ltx_text ltx_lst_space"> </span>is<span id="lstnumberx6.7.1.9" class="ltx_text ltx_lst_space"> </span>{min_tokens_to_keep}"</span></span><span id="lstnumberx6.8" class="ltx_text ltx_font_typewriter">)</span>
</div>
<div id="lstnumberx7" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">7</span>
</div>
<div id="lstnumberx8" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">8</span><span id="lstnumberx8.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx8.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx8.3" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx8.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_p</span><span id="lstnumberx8.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx8.6" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx8.7" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx8.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_p</span>
</div>
<div id="lstnumberx9" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">9</span><span id="lstnumberx9.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx9.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx9.3" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx9.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">filter_value</span><span id="lstnumberx9.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx9.6" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx9.7" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx9.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">filter_value</span>
</div>
<div id="lstnumberx10" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">10</span><span id="lstnumberx10.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx10.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx10.3" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx10.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_tokens_to_keep</span><span id="lstnumberx10.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx10.6" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx10.7" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx10.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_tokens_to_keep</span>
</div>
<div id="lstnumberx11" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">11</span>
</div>
<div id="lstnumberx12" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">12</span><span id="lstnumberx12.1" class="ltx_text ltx_lst_space ltx_font_typewriter">    </span><span id="lstnumberx12.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">def</span><span id="lstnumberx12.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">__call__</span><span id="lstnumberx12.5" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx12.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx12.7" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx12.8" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.9" class="ltx_text ltx_lst_identifier ltx_font_typewriter">input_ids</span><span id="lstnumberx12.10" class="ltx_text ltx_font_typewriter">:</span><span id="lstnumberx12.11" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.12" class="ltx_text ltx_lst_identifier ltx_font_typewriter">torch</span><span id="lstnumberx12.13" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx12.14" class="ltx_text ltx_lst_identifier ltx_font_typewriter">LongTensor</span><span id="lstnumberx12.15" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx12.16" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.17" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scores</span><span id="lstnumberx12.18" class="ltx_text ltx_font_typewriter">:</span><span id="lstnumberx12.19" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.20" class="ltx_text ltx_lst_identifier ltx_font_typewriter">torch</span><span id="lstnumberx12.21" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx12.22" class="ltx_text ltx_lst_identifier ltx_font_typewriter">FloatTensor</span><span id="lstnumberx12.23" class="ltx_text ltx_font_typewriter">)</span><span id="lstnumberx12.24" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.25" class="ltx_text ltx_font_typewriter">-&gt;</span><span id="lstnumberx12.26" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx12.27" class="ltx_text ltx_lst_identifier ltx_font_typewriter">torch</span><span id="lstnumberx12.28" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx12.29" class="ltx_text ltx_lst_identifier ltx_font_typewriter">FloatTensor</span><span id="lstnumberx12.30" class="ltx_text ltx_font_typewriter">:</span>
</div>
<div id="lstnumberx13" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">13</span><span id="lstnumberx13.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx13.2" class="ltx_text ltx_lst_comment ltx_font_typewriter" style="color:#808080;">#<span id="lstnumberx13.2.1" class="ltx_text ltx_lst_space"> </span>Convert<span id="lstnumberx13.2.2" class="ltx_text ltx_lst_space"> </span>logits<span id="lstnumberx13.2.3" class="ltx_text ltx_lst_space"> </span>to<span id="lstnumberx13.2.4" class="ltx_text ltx_lst_space"> </span>probabilities</span>
</div>
<div id="lstnumberx14" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">14</span><span id="lstnumberx14.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx14.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">probs</span><span id="lstnumberx14.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx14.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx14.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx14.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">torch</span><span id="lstnumberx14.7" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx14.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">softmax</span><span id="lstnumberx14.9" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx14.10" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scores</span><span id="lstnumberx14.11" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx14.12" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx14.13" class="ltx_text ltx_lst_identifier ltx_font_typewriter">dim</span><span id="lstnumberx14.14" class="ltx_text ltx_font_typewriter">=-1)</span>
</div>
<div id="lstnumberx15" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">15</span><span id="lstnumberx15.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx15.2" class="ltx_text ltx_lst_comment ltx_font_typewriter" style="color:#808080;">#<span id="lstnumberx15.2.1" class="ltx_text ltx_lst_space"> </span>Get<span id="lstnumberx15.2.2" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx15.2.3" class="ltx_text ltx_lst_space"> </span>probability<span id="lstnumberx15.2.4" class="ltx_text ltx_lst_space"> </span>of<span id="lstnumberx15.2.5" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx15.2.6" class="ltx_text ltx_lst_space"> </span>top<span id="lstnumberx15.2.7" class="ltx_text ltx_lst_space"> </span>token<span id="lstnumberx15.2.8" class="ltx_text ltx_lst_space"> </span>for<span id="lstnumberx15.2.9" class="ltx_text ltx_lst_space"> </span>each<span id="lstnumberx15.2.10" class="ltx_text ltx_lst_space"> </span>sequence<span id="lstnumberx15.2.11" class="ltx_text ltx_lst_space"> </span>in<span id="lstnumberx15.2.12" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx15.2.13" class="ltx_text ltx_lst_space"> </span>batch</span>
</div>
<div id="lstnumberx16" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">16</span><span id="lstnumberx16.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx16.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">top_probs</span><span id="lstnumberx16.3" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx16.4" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx16.5" class="ltx_text ltx_lst_identifier ltx_font_typewriter">_</span><span id="lstnumberx16.6" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx16.7" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx16.8" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx16.9" class="ltx_text ltx_lst_identifier ltx_font_typewriter">probs</span><span id="lstnumberx16.10" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx16.11" class="ltx_text ltx_lst_keyword ltx_lst_keywords2 ltx_font_typewriter" style="color:#0000FF;">max</span><span id="lstnumberx16.12" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx16.13" class="ltx_text ltx_lst_identifier ltx_font_typewriter">dim</span><span id="lstnumberx16.14" class="ltx_text ltx_font_typewriter">=-1,</span><span id="lstnumberx16.15" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx16.16" class="ltx_text ltx_lst_identifier ltx_font_typewriter">keepdim</span><span id="lstnumberx16.17" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx16.18" class="ltx_text ltx_lst_identifier ltx_font_typewriter">True</span><span id="lstnumberx16.19" class="ltx_text ltx_font_typewriter">)</span>
</div>
<div id="lstnumberx17" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">17</span><span id="lstnumberx17.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx17.2" class="ltx_text ltx_lst_comment ltx_font_typewriter" style="color:#808080;">#<span id="lstnumberx17.2.1" class="ltx_text ltx_lst_space"> </span>Calculate<span id="lstnumberx17.2.2" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx17.2.3" class="ltx_text ltx_lst_space"> </span>actual<span id="lstnumberx17.2.4" class="ltx_text ltx_lst_space"> </span>min_p<span id="lstnumberx17.2.5" class="ltx_text ltx_lst_space"> </span>threshold<span id="lstnumberx17.2.6" class="ltx_text ltx_lst_space"> </span>by<span id="lstnumberx17.2.7" class="ltx_text ltx_lst_space"> </span>scaling<span id="lstnumberx17.2.8" class="ltx_text ltx_lst_space"> </span>min_p<span id="lstnumberx17.2.9" class="ltx_text ltx_lst_space"> </span>with<span id="lstnumberx17.2.10" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx17.2.11" class="ltx_text ltx_lst_space"> </span>top<span id="lstnumberx17.2.12" class="ltx_text ltx_lst_space"> </span>token’s<span id="lstnumberx17.2.13" class="ltx_text ltx_lst_space"> </span>probability</span>
</div>
<div id="lstnumberx18" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">18</span><span id="lstnumberx18.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx18.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scaled_min_p</span><span id="lstnumberx18.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx18.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx18.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx18.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx18.7" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx18.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_p</span><span id="lstnumberx18.9" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx18.10" class="ltx_text ltx_font_typewriter">*</span><span id="lstnumberx18.11" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx18.12" class="ltx_text ltx_lst_identifier ltx_font_typewriter">top_probs</span>
</div>
<div id="lstnumberx19" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">19</span><span id="lstnumberx19.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx19.2" class="ltx_text ltx_lst_comment ltx_font_typewriter" style="color:#808080;">#<span id="lstnumberx19.2.1" class="ltx_text ltx_lst_space"> </span>Create<span id="lstnumberx19.2.2" class="ltx_text ltx_lst_space"> </span>a<span id="lstnumberx19.2.3" class="ltx_text ltx_lst_space"> </span>mask<span id="lstnumberx19.2.4" class="ltx_text ltx_lst_space"> </span>for<span id="lstnumberx19.2.5" class="ltx_text ltx_lst_space"> </span>tokens<span id="lstnumberx19.2.6" class="ltx_text ltx_lst_space"> </span>that<span id="lstnumberx19.2.7" class="ltx_text ltx_lst_space"> </span>have<span id="lstnumberx19.2.8" class="ltx_text ltx_lst_space"> </span>a<span id="lstnumberx19.2.9" class="ltx_text ltx_lst_space"> </span>probability<span id="lstnumberx19.2.10" class="ltx_text ltx_lst_space"> </span>less<span id="lstnumberx19.2.11" class="ltx_text ltx_lst_space"> </span>than<span id="lstnumberx19.2.12" class="ltx_text ltx_lst_space"> </span>the<span id="lstnumberx19.2.13" class="ltx_text ltx_lst_space"> </span>scaled<span id="lstnumberx19.2.14" class="ltx_text ltx_lst_space"> </span>min_p</span>
</div>
<div id="lstnumberx20" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">20</span><span id="lstnumberx20.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx20.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">tokens_to_remove</span><span id="lstnumberx20.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx20.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx20.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx20.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">probs</span><span id="lstnumberx20.7" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx20.8" class="ltx_text ltx_font_typewriter">&lt;</span><span id="lstnumberx20.9" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx20.10" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scaled_min_p</span>
</div>
<div id="lstnumberx21" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">21</span>
</div>
<div id="lstnumberx22" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">22</span><span id="lstnumberx22.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx22.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices</span><span id="lstnumberx22.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx22.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx22.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx22.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">torch</span><span id="lstnumberx22.7" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx22.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">argsort</span><span id="lstnumberx22.9" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx22.10" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scores</span><span id="lstnumberx22.11" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx22.12" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx22.13" class="ltx_text ltx_lst_identifier ltx_font_typewriter">descending</span><span id="lstnumberx22.14" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx22.15" class="ltx_text ltx_lst_identifier ltx_font_typewriter">True</span><span id="lstnumberx22.16" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx22.17" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx22.18" class="ltx_text ltx_lst_identifier ltx_font_typewriter">dim</span><span id="lstnumberx22.19" class="ltx_text ltx_font_typewriter">=-1)</span>
</div>
<div id="lstnumberx23" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">23</span><span id="lstnumberx23.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx23.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices_to_remove</span><span id="lstnumberx23.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx23.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx23.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx23.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">torch</span><span id="lstnumberx23.7" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx23.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">gather</span><span id="lstnumberx23.9" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx23.10" class="ltx_text ltx_lst_identifier ltx_font_typewriter">tokens_to_remove</span><span id="lstnumberx23.11" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx23.12" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx23.13" class="ltx_text ltx_lst_identifier ltx_font_typewriter">dim</span><span id="lstnumberx23.14" class="ltx_text ltx_font_typewriter">=-1,</span><span id="lstnumberx23.15" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx23.16" class="ltx_text ltx_lst_identifier ltx_font_typewriter">index</span><span id="lstnumberx23.17" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx23.18" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices</span><span id="lstnumberx23.19" class="ltx_text ltx_font_typewriter">)</span>
</div>
<div id="lstnumberx24" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">24</span><span id="lstnumberx24.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx24.2" class="ltx_text ltx_lst_comment ltx_font_typewriter" style="color:#808080;">#<span id="lstnumberx24.2.1" class="ltx_text ltx_lst_space"> </span>Keep<span id="lstnumberx24.2.2" class="ltx_text ltx_lst_space"> </span>at<span id="lstnumberx24.2.3" class="ltx_text ltx_lst_space"> </span>least<span id="lstnumberx24.2.4" class="ltx_text ltx_lst_space"> </span>min_tokens_to_keep</span>
</div>
<div id="lstnumberx25" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">25</span><span id="lstnumberx25.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx25.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices_to_remove</span><span id="lstnumberx25.3" class="ltx_text ltx_font_typewriter">[...,</span><span id="lstnumberx25.4" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx25.5" class="ltx_text ltx_font_typewriter">:</span><span id="lstnumberx25.6" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx25.7" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx25.8" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx25.9" class="ltx_text ltx_lst_identifier ltx_font_typewriter">min_tokens_to_keep</span><span id="lstnumberx25.10" class="ltx_text ltx_font_typewriter">]</span><span id="lstnumberx25.11" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx25.12" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx25.13" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx25.14" class="ltx_text ltx_lst_identifier ltx_font_typewriter">False</span>
</div>
<div id="lstnumberx26" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">26</span>
</div>
<div id="lstnumberx27" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">27</span><span id="lstnumberx27.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx27.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">indices_to_remove</span><span id="lstnumberx27.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx27.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx27.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx27.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices_to_remove</span><span id="lstnumberx27.7" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx27.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scatter</span><span id="lstnumberx27.9" class="ltx_text ltx_font_typewriter">(1,</span><span id="lstnumberx27.10" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx27.11" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices</span><span id="lstnumberx27.12" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx27.13" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx27.14" class="ltx_text ltx_lst_identifier ltx_font_typewriter">sorted_indices_to_remove</span><span id="lstnumberx27.15" class="ltx_text ltx_font_typewriter">)</span>
</div>
<div id="lstnumberx28" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">28</span><span id="lstnumberx28.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx28.2" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scores_processed</span><span id="lstnumberx28.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx28.4" class="ltx_text ltx_font_typewriter">=</span><span id="lstnumberx28.5" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx28.6" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scores</span><span id="lstnumberx28.7" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx28.8" class="ltx_text ltx_lst_identifier ltx_font_typewriter">masked_fill</span><span id="lstnumberx28.9" class="ltx_text ltx_font_typewriter">(</span><span id="lstnumberx28.10" class="ltx_text ltx_lst_identifier ltx_font_typewriter">indices_to_remove</span><span id="lstnumberx28.11" class="ltx_text ltx_font_typewriter">,</span><span id="lstnumberx28.12" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx28.13" class="ltx_text ltx_lst_identifier ltx_font_typewriter">self</span><span id="lstnumberx28.14" class="ltx_text ltx_font_typewriter">.</span><span id="lstnumberx28.15" class="ltx_text ltx_lst_identifier ltx_font_typewriter">filter_value</span><span id="lstnumberx28.16" class="ltx_text ltx_font_typewriter">)</span>
</div>
<div id="lstnumberx29" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">29</span><span id="lstnumberx29.1" class="ltx_text ltx_lst_space ltx_font_typewriter">        </span><span id="lstnumberx29.2" class="ltx_text ltx_lst_keyword ltx_font_typewriter" style="color:#0000FF;">return</span><span id="lstnumberx29.3" class="ltx_text ltx_lst_space ltx_font_typewriter"> </span><span id="lstnumberx29.4" class="ltx_text ltx_lst_identifier ltx_font_typewriter">scores_processed</span>
</div>
</div>
</div>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Full Plots for GSM8K/GPQA results</h2>

<figure id="A2.F4" class="ltx_figure"><img src="/html/2407.01082/assets/x13.png" id="A2.F4.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="322" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 4: </span>Results for all GSM8K experiments on a single plot.</figcaption>
</figure>
<figure id="A2.F5" class="ltx_figure"><img src="/html/2407.01082/assets/x14.png" id="A2.F5.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="461" height="224" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 5: </span>Results for all GPQA experiments on a single plot.</figcaption>
</figure>
<div class="ltx_pagination ltx_role_newpage"></div>
</section><div class="ltx_rdf" about="" property="dcterms:title" content="Overleaf Example"></div>

</article>
</div>
<div class="ar5iv-footer"><a href="/html/2407.01081" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2407.01082" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2407.01082">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2407.01082" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2407.01083" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Mon Aug  5 13:51:45 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2407.01082
Large Language Models (LLMs)  generate longform text by successively sampling the next token based on the probability distribution of the token vocabulary at each decoding step. Current popular truncation sampling methods such as top-sampling, also known as nucleus sampling, often struggle to balance coherence and creativity in generating text, particularly when using higher temperatures. To address this issue, we propose min-, a dynamic truncation sampling method, that establishes a minimum base percentage threshold for tokens, which the scales according to the probability of the top candidate token. Through experiments on several benchmarks, such as GPQA, GSM8K and AlpacaEval Creative Writing, we demonstrate that min-improves the coherence and quality of generated text even at high temperatures, while also facilitating more creative and diverse outputs compared to top-and other sampling methods. As of writing, min-has been adopted by multiple open-source LLM implementations, and have been independently assessed by members of the open-source LLM community, further validating its practical utility and potential111Code, evaluation code and results available athttps://github.com/menhguin/minp_paper/.
abstract
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
Abstract
2407.01082
1IntroductionLarge Language Models (LLMs)  have shown a remarkable ability to generate coherent and creative text across various domains. However, the quality and diversity of the generated text depends heavily on the choice of sampling methods (samplers)  and hyperparameters, such as thresholds and sampling temperature, during the decoding process, where the model autoregressively samples the next token based on the probability distribution over its vocabulary.Figure 1:Effects of top-, top-, and min-sampling on token probability distributions. Min-sampling dynamically adjusts its filtering threshold based on the model’s confidence, focusing on high-probability tokens when confident and including diverse but plausible options when uncertain. This helps min-balances coherence and diversity more effectively than top-and top-sampling.In particular, increasing the sampling temperature can enhance creativity by promoting more diverse token selection—but often at the cost of coherence, as the quality of the generated text tends to degrade quickly at higher temperature with nucleus sampling. Thiscreativity-coherence trade-offis the main reason low conservative temperatures between 0 and 1 are often chosen for sampling. In this work, we will examine whether the creativity-coherence trade-off can be relaxed, that is whether we can achieve higher creativity while preserving coherence.As of writing, top-samplingis the state-of-the-art sampling method that improves coherence by focusing on high-probability tokens and truncating the "unreliable tail" of low-probability tokens that can lead to degenerate text in top-sampling. However, top-sampling does not work well for higher temperatures: here, tokens from the "unreliable tail" can still enter the sampling pool.Inspired by this observation, our proposed truncation sampling method, min-, dynamically sets a minimum probability threshold for token selection. Min-maintains coherence at higher temperatures without sacrificing creativity by eliminating the long tail.In summary, our objectives for min-are to(1) match or outperform the state-of-the-art top-sampling across various reasoning and performance benchmarks at standard temperature settings between 0 and 1,(2) better handle the creativity-coherence trade-off at higher temperatures, and(3) provide a simple, effective sampling method without relying on additional techniques like repetition penalties to address the externalities of high-tempature sampling.To evaluate the performance of min-sampling, we conduct experiments comparing it with top-and other popular sampling methods on several LLM benchmarks, spanning open-ended question answering, grade school math, and creative writing. The results demonstrate min-’s ability to maintain coherence at higher temperatures while outperforming top-in terms of output diversity. The practical utility of min-is further validated by its adoption in multiple open-source LLM frameworks.Our key contributions can be summarized as follows:1.We introduce min-, a dynamic truncation sampling method that balances quality and diversity, leading to more creative and coherent outputs at higher temperature settings.2.We experimentally demonstrate min-’s advantages over top-and other samplers on LLM benchmarks, emphasizing its reduced tradeoffs between creativity and coherence at temperature settings beyond 1.By achieving these objectives, we establish min-sampling as a viable and user-friendly alternative to top-sampling, unlocking the benefits of high-temperature settings for enhanced creativity in LLM-generated text without sacrificing coherence.
introduction
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
1Introduction
2407.01082
2Background2.1Language ModellingLanguage models are probabilistic models that assign probabilities to sequences of words or tokens. An autoregressive language model predicts the probability distribution over the vocabulary for the next token, conditioned on the previous tokens, and iteratively samples tokens to generate a sequence. Formally, given a sequence of tokensand a finite vocabulary, where each token, an autoregressive language model models the probability distribution, for each tokenin the sequence. This probability distribution represents the likelihood of each token occurring at positiongiven the preceding tokens.Greedy decoding.A deterministic way to sample tokens isgreedy decoding, which selects the token with the highest probability at each step. Given the probability distributionover the vocabularyat position, it chooses the tokenthat maximizes this probability:(1) By repeatedly selecting the most probable token at each step, we can generate a sequence of tokensof arbitrary length. However, this greedy approach often leads to degenerate output, as it always chooses the most likely continuation without considering other plausible alternatives. This deterministic sampling has found to often result in repetitive or generic generated text that lacks diversity and creativity.Stochastic Sampling.The stochastic alternative to greedy decoding is sampling of tokens according to the probability distribution:(2) In this case, there are several ways to modify and constrain the sampling distribution to improve the quality of the generated text. We will look at the most commonly used approaches next.2.2Temperature ScalingTemperature scaling modifies the sharpness of the token distribution. The term temperature is inspired by thermodynamics and the Boltzmann distribution: higher temperatures correspond to more uniform mixing that more uniform sampling while lower temperatures converge to the argmax.Formally, given a probability distributionover the vocabularyat position, temperature sampling first divides the logits (unnormalized log probabilities)  by a temperature parameterbefore passing them through the softmax function to obtain a modified probability distribution:(3) whereis the (unnormalized)  log probability of token.
The next tokenis then randomly sampled from the modified probability distribution:(4) The temperature parametercontrols the randomness of the sampling process. Higher values of(e.g.,)  result in a more uniform probability distribution, increasing the chances of sampling lower-probability tokens and generating more diverse output. Lower values of(e.g.,)  make the distribution sharper, favoring higher-probability tokens andmore conservative and deterministic outputs. Settingleaves the original probability distribution unchanged.2.3Top-SamplingTop-sampling, also known as nucleus sampling, is a stochastic truncation decoding method that aims to address the limitations of greedy decoding. Instead of always selecting the most probable token like in greedy decoding, top-sampling introduces stochasticity while still focusing on high-probability tokens.Givenover the vocabularyat position, top-sampling first sorts the tokens in descending order of their probabilities. It then selects the smallest set of tokens whose cumulative probability exceeds a predefined threshold, where. Formally, letbe the smallest set such that:(5) The next tokenis then randomly sampled from this reduced setaccording to the renormalized probabilities:By restricting the sampling to the nucleus of high-probability tokens, top-sampling allows for more diverse and coherent generated text compared to greedy decoding.However, combining truncation methods like top-sampling with temperature scaling can lead to suboptimal results. When using top-sampling, it is recommended to adjust either the value ofor the temperature, but not both simultaneouslyas they can lead to conflicting effects.
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
2Background
2407.01082
3Min-SamplingWe introduce min-sampling, a stochastic truncation sampling method that maintains a minimum level of token diversity while still focusing on high-probability tokens. The method uses a relative probability thresholdto scales the maximum token probabilityto determine the absolute probability threshold. Sampling is then performed on tokens with probability greater than or equal to.Formally, given the maximum probability over the token distribution, the absolute probability thresholdis calculated as:(6) The sampling poolis then defined as the set of tokens whose probability is greater than or equal to:(7) Finally, the next tokenis randomly sampled from the setaccording to the normalized probabilities:(8) 
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
3Min-Sampling
2407.01082
4Case StudyIn this section, we inspect the behavior of min-with top-when both methods are used in conjunction with temperature. Briefly:•High-confidence predictions:When the model assigns high probability to a particular next token, min-filters out low-probability alternatives. This preserves coherence by avoiding sampling from the "unreliable tail" that could derail generation.•Low-confidence predictions:When there is no clear front-runner, min-relaxes its filter. This allows the model to sample from a more diverse set of plausible continuations, enabling creativity in open-ended settings.We illustrate these differences in the following two examples222Interactive demo available athttps://artefact2.github.io/llm-sampling/index.xhtml. Demo not affliated with authors.. Table1shows a case where the next token probabilities are relatively low, while Table2shows a case with a clear high probability next token.4.1Case 1: Low-Certainty Next TokenTable1shows the token probabilities for a creative writing prompt where there are many plausible ways to continue the story. At temperature, the highest probability token "was" only has a 70.3% probability. When the temperature is increased to, this drops to 11.9% as probability mass shifts to other candidate tokens. In this low-certainty case, applying top-and min-to the post-temperature distribution yields similar results, as there is no clear top token for min-to prioritize.4.2Case 2: High-Certainty Next TokenIn contrast, Table2shows a case where using min-has a clear impact. The prompt is a factual statement about rainbows, with "light"being the obvious next token with 98.25% probability when temperature.Raising the temperature toflattens out the probability distribution, but "light" remains dominant at 34.41%. When applying top-, several additional tokens like "water", "sunshine", "a" still remain in the sampling pool at ~2-3% probability each. While these are vaguely plausible, they are much lower quality continuations compared to "light".Min-, on the other hand, truncates much more aggressively. Setting a 10% base probability threshold when "light" has 34.41% probability leads min-to only include "light" and "sunlight" in its sampling pool, as all other tokens have less than the scaled 3.441 percent probability threshold. This showcases min-’s ability to maintain coherence at high temperatures by calibrating its cutoff to the strength of the top candidate.Prompt"You will pay for what you have done," she hissed, her blade flashing in the moonlight. The battle that ensued _____SamplingTemperature+top-+min-Methodwas70.311.913.118.5lasted9.56.16.79.5between6.25.35.98.2left4.54.85.37.4would3.24.34.76.6could1.23.13.44.8seemed0.52.32.53.5Table 1:Comparison between temperature sampling, top-, min-sampling at higher temperatures, for a low-certainty case. In cases where the tokens are relatively low-certainty, min-does not differ significantly from top-.PromptA rainbow is an optically brilliant meteorological event, resulting from the refraction, reflection, and dispersion of _____SamplingTemperature+top-+min-Methodlight98.2534.4138.1880.90sunlight1.298.129.0119.10water0.103.443.820.00sunshine0.062.893.210.00a0.052.713.010.00moisture0.052.703.000.00Table 2:Comparison between temperature sampling, top-, min-sampling at higher temperatures, for a high-certainty case. In cases where the tokens are high-certainty, min-filters for the high probability token more aggressively than top-. In this case, when, top-lets initially low probability tokens (<0.10% probability)  into its nucleus (~3% probability) , while min-filters them out entirely.4.3DiscussionOur case studies demonstrate that min-sampling’s token choices differ most notably from top-sampling when selecting high-probability tokens at elevated temperatures. This has important implications for improving the performance of language models in high-temperature settings:Maintaining accuracy in reasoning tasks.For tasks that require factual and logical reasoning, such as mathematics, science, programming, and recalling specific facts, min-significantly reduces the accuracy tradeoff at high temperatures. In the rainbow example (Table2) , 100% of min-’s probability distribution ("light" and "sunlight")  can be considered accurate, with the most relevant answer "light" comprising 80.90%. In contrast, only 47.19% of top-’s distribution is relevant to light. This suggests that min-can enable language models to tackle reasoning tasks at temperature settings previously considered unsuitable. We further evaluate min-’s reasoning capabilities in Section5.Mitigating cascading errors.Language models construct longform outputs by sequentially selecting tokens, with previous choices influencing future decisions. Consequently, erroneous token selections can compound, increasing the likelihood of further inaccurate or incoherent choices. The rainbow prompt illustrates this, as it contains multiple technical terms ("meteorological", "refraction", "dispersion")  that require high-accuracy token selections to maintain semantic coherence and factual correctness. Min-’s ability to robustly select high-confidence tokens, even at high temperatures, helps prevent such cascading failures.
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
4Case Study
2407.01082
5ExperimentsTo establish min-sampling as a viable alternative to the state-of-the-art top-(nucleus)  sampling, we conduct experiments to demonstrate that:1.Min-achieves equal or better benchmark performance compared to top-at standard temperature settings () .2.Min-handles the creativity-coherence tradeoff well at higher temperatures ()  by achieving more diverse outputs while maintaining coherence, as demonstrated by improved performance on reasoning benchmarks at high temperatures.5.1Experimental SetupWe evaluate the performance of min-sampling against top-sampling on three generative language model tasks using a Mistral 7Bmodel:•Graduate-level reasoning:Google-Proof Q&A Benchmark - Main, Generative (GPQA Main)  - 5-shot•Grade school math:GSM8K Chain-of-Thought (GSM8K COT)  - 8-shot•Creative writing:AlpacaEval Creative WritingWhile 7B parameter models typically achieve less than 50% accuracy on these challenging benchmarks, this is sufficient for comparing the relative performance between truncation sampling methods.We primarily focus on comparing min-values in the range of 0.05 to 0.1 with top-values between 0.9 and 0.95. These are common settings preferred by users, although they are not directly comparable. In general, a higher min-value or a lower top-value results in more selective token choices and a bias towards more likely outputs.For GPQA and GSM8K, we used the EleutherAI Evaluation Harness.5.2Graduate-Level Reasoning (GPQA Main) To evaluate min-’s impact on graduate-level reasoning, we employ the Google-Proof Q&A Benchmark (GPQA) , a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. GPQA is designed to test difficult, graduate-level domain expertise with minimal dataset leakage, making it particularly suitable for our experiments, as Mistral 7B was released in September 2023, two months before GPQA’s public release.5-shot prompting.Initial experiments revealed that the baseline Mistral 7B model at temperature 1 with zero-shot prompting was unable to answer any question correctly (0% accuracy) . Few-shot prompting proved necessary to obtain statistically significant and meaningful comparisons. After discussions with GPQA’s lead author regarding standard few-shot settings for a 7B parameter model, we opted for 5-shot prompting in this evaluation.5.3Grade School Math (GSM8K COT) To evaluate min-’s impact on mathematical reasoning, we employ the GS8MK Chain-of-Thought (GSM8K COT)  dataset, which consists of 8.5K high-quality, linguistically diverse grade school math word problems. We consider this a rigorous, falsifiable, quantitative test of reasoning that 7B parameter models can somewhat manage, albeit with significant performance degradations at higher temperatures. Our prompting set-up is as follows:8-shot prompting.Initial experiments revealed that the baseline Mistral 7B model at temperature 1 with zero-shot prompting was unable to answer any question correctly (0% accuracy) . Few-shot prompting proved necessary to obtain statistically significant and meaningful comparisons. We chose 8-shot prompting, as it is the recommended baseline on the EleutherAI Evaluations Harness and widely used in reporting GSM8K COT results.Chain-of-Thought (COT) .We utilize Chain-of-Thought prompting for two reasons. First, baseline Mistral 7B scored too low on GSM8K 8-shot for meaningful comparisons ( 25%) , while GSM8K COT 8-shot yielded a more helpful baseline of  38% at temperature 1. Second, COT more accurately simulates real-world environments that require multi-step reasoning.Exact Match.We primarily report Exact Match scores, in line with standard reporting practices, although Flexible Match scores generally favor min-even more than Exact Match scores.5.4Creative Writing (AlpacaEval Creative Writing) To evaluate min-’s impact on creative writing tasks, we employ the AlpacaEval Creative Writing Benchmark. AlpacaEval is an LLM-based automatic evaluation framework that is fast, cost-effective, replicable, and validated against 20K human annotations. The Creative Writing Benchmark utilizes a frontier LLM evaluator (in this case, Anthropic’s Claude)  to select between two outputs generated from creative writing prompts.We chose this benchmark due to the significant number of users within the open-source community who have qualitatively reported a preference for min-sampling on high-temperature creative writing tasks. This preference has contributed to min-’s rapid adoption within the open-source language model community. At the time of writing, we do not have access to other creativity or creative writing evaluation datasets, whether automated or human-annotated.We note that these AlpacaEval Creative Writing results are primarily reproduced with permission from another independent evaluator within the open-source community. This independent evaluator is one of many open-source community members who have taken interest in min-within the months after its release, conducting their own evaluation which we then reproduced via the same procedure.
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
5Experiments
2407.01082
6ResultsWe find that min-sampling outperforms top-across various temperature settings and benchmarks, maintaining coherence while allowing for increased diversity at higher temperatures (>1) .6.1Graduate-Level Reasoning (GPQA Main) Figure3presents the results of the GPQA Main Generative evaluation. We observe that min-sampling slightly outperforms top-sampling at a temperature of 1.As the temperature increases, both truncation sampling methods experience a degradation in benchmark performance. However, min-’s performance degradation is noticeably lower compared to top-, indicating that it more reliably answers graduate-level reasoning questions at higher temperatures.It is notable that min-sampling exhibits a slight improvement in reasoning performance between temperatures 1 and 3. This suggests that the increased diversity in token selection introduced by min-at these temperature levels may be beneficial for complex reasoning tasks, as it allows the model to explore a broader range of plausible solutions without significantly sacrificing coherence.6.2Grade School Math (GSM8K COT) Figure2shows the results of the GSM8K Chain-of-Thought evaluation. Similar to the GPQA results, min-sampling slightly outperforms top-sampling at a temperature of 1. As the temperature increases, both truncation sampling methods experience a decline in benchmark performance. However, min-’s performance degradation is noticeably lower compared to top-, suggesting that it more reliably derives the correct answer to mathematical word problems at higher temperatures.Figure 2:Results of running min-vs top-on GSM8K. The control method used is pure sampling. A single plot with all results is available in Figure4.Figure 3:Results of running min-vs top-on GPQA. The control method used is pure sampling. A single plot with all results is available in Figure5.These results demonstrate min-’s ability to maintain higher levels of coherence and accuracy in multi-step reasoning tasks, even when the diversity of token selection is increased through higher temperature settings. This finding aligns with our hypothesis that min-sampling can better navigate the creativity-coherence tradeoff in complex reasoning scenarios.6.3AlpacaEvalTable3shows that min-displays strong performance on the AlpacaEval Creative Writing Benchmark, especially atat, a temperature where top-struggles to produce coherent creative writing.TempConstraintsWinrateWinrate(LC) 1.5min_p=0.158.1256.541.0min_p=0.0555.0752.010.8top_p=0.9854.6551.291.0min_p=0.153.2450.141.0top_p=0.9853.0050.431.0top_p=0.952.0750.070.8top_p=0.9551.8050.221.0min_p=0.0251.6250.431.0min_p=0.0251.4648.850.8min_p=0.0550.9947.840.8top_p=0.9550.7648.781.050.050.00.049.9046.641.5min_p=0.0548.1348.571.5min_p=0.0242.0944.831.0mirostat_tau=4.016.0416.691.0mirostat_tau=5.016.0316.401.5FAILFAIL1.5top_p=0.98FAILFAIL1.5top_p=0.95FAILFAIL1.5top_p=0.9FAILFAIL1.5top_p=0.8FAILFAILTable 3:AlpacaEval on creative writing prompts with different sampling parameters. LC-winrate is winrate with length control, which alleviates the length biases of LLM-based judging. Min-results are inbold.
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
6Results
2407.01082
7Different Min-ConfigurationsAs of June 2024, users of open-source LLM inference engines generally recommend min-settings of 0.05-0.1 and temperature  1 for normal usage, temperature 1-1.5 for varied text generation while balancing factual and logical tasks, and min-0.05-0.1 with temperature 2-5 for roleplay and creative writing. Higher values of min-base percentage (e.g., min-= 0.9)  result in significantly reduced degradation for temperature scaling values >20.Sampling Order.Min-can be used with any other sampling method, in any order. Applying min-after temperature scaling is recommended for steerability. Applying min-before temperature scaling results in the same token pool as temperature 1, with temperature only shifting relative probabilities, leading to higher accuracy for high-certainty tokens but constrained diversity.Combining Sampling Methods.Combining multiple sampling methods is possible, it has not been thoroughly tested, and no particular benefits have been observed or reported, likely because sampling methods are designed to be used in isolation.
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
7Different Min-Configurations
2407.01082
8Related WorkIn addition to temperature sampling and top-sampling, several alternative sampling methods have been proposed, whichclassify as deterministic and stochastic. This section focuses on stochastic sampling methods, as min-sampling falls under this category.Top-samplingis a truncation-based method that restricts the sampling pool to themost probable tokens at each step.Mirostat samplingbuilds upon top-sampling by dynamically adjusting the value ofat each generation step to maintain a target perplexity level, enabling the generation of coherent and diverse text without requiring manual tuning of the sampling parameters.Locally typical samplingis a sampling method that limits the sampling pool to words with negative log-probability within a certain range from the conditional entropy of the distribution at that time step, which helps to avoid both highly unlikely and overly generic tokens.Contrastive decodinguses an auxiliary language model to penalize tokens that are unlikely under the main language model, which guides the selection of higher-quality text while ensuring the generated output remains plausible.-samplingis an entropy-dependent truncation method that sets a probability threshold based on the entropy of the conditional distribution at each step, truncating words below this threshold.
body
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
8Related Work
2407.01082
9ConclusionIn this work, we introduced min-sampling, a novel dynamic truncation sampling method that aims to balance creativity and coherence in text generation from large language models, particularly at high temperatures. Through experiments on multiple LLM benchmarks, including graduate-level reasoning (GPQA) , grade school math (GSM8K) , and creative writing (AlpacaEval) , we demonstrated that min-outperforms the popular top-sampling method in maintaining coherence while enabling more diverse outputs at temperatures above 1. The rapid adoption of min-by the open-source LLM community further validates its practical utility and potential. By effectively navigating the trade-off between creativity and coherence, min-sampling opens up new possibilities for high-quality open-ended text generation.Further Work.Future research directions for min-sampling include investigating its performance on a wider range of NLP tasks, such as dialogue generation, story completion, and language translation. A more rigorous theoretical analysis of min-’s behavior may provide valuable insights for developing even more effective sampling strategies.9.1LimitationsAlthough our study of min-has shown promising results, it is not without limitations. In particular, we focus on a single model architecture (Mistral 7B)  and a limited set of benchmarks. To establish the generality of min-’s benefits, evaluating its performance across a diverse range of model sizes, architectures, and tasks is important. Furthermore, the AlpacaEval Creative Writing benchmark, while cost-effective and validated against human judgments, may not fully capture all aspects of creativity in text generation. Human evaluation studies could provide additional insights into the quality and creativity of text generated using min-compared to other sampling methods using the same models. At the same time, the results position min-as a computationally cheap alternative to nucleus sampling, especially when creativity and higher diversity of the generated text are important.
conclusion
Min P Sampling: Balancing Creativity and Coherence at High
Temperature
9Conclusion