<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2310.10134] CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization</title><meta property="og:description" content="Language agents have shown some ability to interact with an external environment, e.g., a virtual
world such as ScienceWorld, to perform complex tasks, e.g., growing a plant, without the startup
costs of reinforcement …">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2310.10134">

<!--Generated on Wed Feb 28 00:40:41 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document ltx_authors_1line">
<h1 class="ltx_title ltx_title_document">CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Bodhisattwa Prasad Majumder<sup id="id1.1.1" class="ltx_sup"><math id="id1.1.1.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id1.1.1.m1.1a"><mn id="id1.1.1.m1.1.1" xref="id1.1.1.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id1.1.1.m1.1b"><cn type="integer" id="id1.1.1.m1.1.1.cmml" xref="id1.1.1.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id1.1.1.m1.1c">1</annotation></semantics></math></sup>, Bhavana Dalvi Mishra<sup id="id2.2.2" class="ltx_sup"><math id="id2.2.2.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id2.2.2.m1.1a"><mn id="id2.2.2.m1.1.1" xref="id2.2.2.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id2.2.2.m1.1b"><cn type="integer" id="id2.2.2.m1.1.1.cmml" xref="id2.2.2.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id2.2.2.m1.1c">1</annotation></semantics></math></sup>,
<br class="ltx_break"><span id="id8.8.8" class="ltx_text ltx_font_bold">Peter Jansen<sup id="id3.3.3.1" class="ltx_sup"><math id="id3.3.3.1.m1.2" class="ltx_Math" alttext="1,2" display="inline"><semantics id="id3.3.3.1.m1.2a"><mrow id="id3.3.3.1.m1.2.3.2" xref="id3.3.3.1.m1.2.3.1.cmml"><mn id="id3.3.3.1.m1.1.1" xref="id3.3.3.1.m1.1.1.cmml">1</mn><mo id="id3.3.3.1.m1.2.3.2.1" xref="id3.3.3.1.m1.2.3.1.cmml">,</mo><mn id="id3.3.3.1.m1.2.2" xref="id3.3.3.1.m1.2.2.cmml">2</mn></mrow><annotation-xml encoding="MathML-Content" id="id3.3.3.1.m1.2b"><list id="id3.3.3.1.m1.2.3.1.cmml" xref="id3.3.3.1.m1.2.3.2"><cn type="integer" id="id3.3.3.1.m1.1.1.cmml" xref="id3.3.3.1.m1.1.1">1</cn><cn type="integer" id="id3.3.3.1.m1.2.2.cmml" xref="id3.3.3.1.m1.2.2">2</cn></list></annotation-xml><annotation encoding="application/x-tex" id="id3.3.3.1.m1.2c">1,2</annotation></semantics></math></sup>, Oyvind Tafjord<sup id="id4.4.4.2" class="ltx_sup"><math id="id4.4.4.2.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id4.4.4.2.m1.1a"><mn id="id4.4.4.2.m1.1.1" xref="id4.4.4.2.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id4.4.4.2.m1.1b"><cn type="integer" id="id4.4.4.2.m1.1.1.cmml" xref="id4.4.4.2.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id4.4.4.2.m1.1c">1</annotation></semantics></math></sup>, Niket Tandon<sup id="id5.5.5.3" class="ltx_sup"><math id="id5.5.5.3.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id5.5.5.3.m1.1a"><mn id="id5.5.5.3.m1.1.1" xref="id5.5.5.3.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id5.5.5.3.m1.1b"><cn type="integer" id="id5.5.5.3.m1.1.1.cmml" xref="id5.5.5.3.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id5.5.5.3.m1.1c">1</annotation></semantics></math></sup>, Li Zhang<sup id="id6.6.6.4" class="ltx_sup"><math id="id6.6.6.4.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id6.6.6.4.m1.1a"><mn id="id6.6.6.4.m1.1.1" xref="id6.6.6.4.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id6.6.6.4.m1.1b"><cn type="integer" id="id6.6.6.4.m1.1.1.cmml" xref="id6.6.6.4.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id6.6.6.4.m1.1c">3</annotation></semantics></math></sup>,
<br class="ltx_break">Chris-Callison Burch<sup id="id7.7.7.5" class="ltx_sup"><math id="id7.7.7.5.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id7.7.7.5.m1.1a"><mn id="id7.7.7.5.m1.1.1" xref="id7.7.7.5.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id7.7.7.5.m1.1b"><cn type="integer" id="id7.7.7.5.m1.1.1.cmml" xref="id7.7.7.5.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id7.7.7.5.m1.1c">3</annotation></semantics></math></sup>, Peter Clark<sup id="id8.8.8.6" class="ltx_sup"><math id="id8.8.8.6.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id8.8.8.6.m1.1a"><mn id="id8.8.8.6.m1.1.1" xref="id8.8.8.6.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id8.8.8.6.m1.1b"><cn type="integer" id="id8.8.8.6.m1.1.1.cmml" xref="id8.8.8.6.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id8.8.8.6.m1.1c">1</annotation></semantics></math></sup></span>

<br class="ltx_break"><sup id="id9.9.9" class="ltx_sup"><math id="id9.9.9.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id9.9.9.m1.1a"><mn id="id9.9.9.m1.1.1" xref="id9.9.9.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id9.9.9.m1.1b"><cn type="integer" id="id9.9.9.m1.1.1.cmml" xref="id9.9.9.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id9.9.9.m1.1c">1</annotation></semantics></math></sup>Allen Institute of AI
<br class="ltx_break"><sup id="id10.10.10" class="ltx_sup"><math id="id10.10.10.m1.1" class="ltx_Math" alttext="2" display="inline"><semantics id="id10.10.10.m1.1a"><mn id="id10.10.10.m1.1.1" xref="id10.10.10.m1.1.1.cmml">2</mn><annotation-xml encoding="MathML-Content" id="id10.10.10.m1.1b"><cn type="integer" id="id10.10.10.m1.1.1.cmml" xref="id10.10.10.m1.1.1">2</cn></annotation-xml><annotation encoding="application/x-tex" id="id10.10.10.m1.1c">2</annotation></semantics></math></sup>University of Arizona
<br class="ltx_break"><sup id="id11.11.11" class="ltx_sup"><math id="id11.11.11.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id11.11.11.m1.1a"><mn id="id11.11.11.m1.1.1" xref="id11.11.11.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id11.11.11.m1.1b"><cn type="integer" id="id11.11.11.m1.1.1.cmml" xref="id11.11.11.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id11.11.11.m1.1c">3</annotation></semantics></math></sup>University of Pennsylvania
<br class="ltx_break">
<br class="ltx_break">Contact: <span id="id12.12.id1" class="ltx_text ltx_font_typewriter">{bodhisattwam, bhavanad}@allenai.org
<br class="ltx_break"></span>Project page: <a target="_blank" href="https://allenai.github.io/clin/" title="" class="ltx_ref ltx_href ltx_font_typewriter">https://allenai.github.io/clin/</a>
</span></span>
</div>

<h1 class="ltx_title ltx_title_document">CLIN: A goal-driven agent that continually improves by learning causal abstractions</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Bodhisattwa Prasad Majumder<sup id="id1.1.1" class="ltx_sup"><math id="id1.1.1.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id1.1.1.m1.1a"><mn id="id1.1.1.m1.1.1" xref="id1.1.1.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id1.1.1.m1.1b"><cn type="integer" id="id1.1.1.m1.1.1.cmml" xref="id1.1.1.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id1.1.1.m1.1c">1</annotation></semantics></math></sup>, Bhavana Dalvi Mishra<sup id="id2.2.2" class="ltx_sup"><math id="id2.2.2.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id2.2.2.m1.1a"><mn id="id2.2.2.m1.1.1" xref="id2.2.2.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id2.2.2.m1.1b"><cn type="integer" id="id2.2.2.m1.1.1.cmml" xref="id2.2.2.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id2.2.2.m1.1c">1</annotation></semantics></math></sup>,
<br class="ltx_break"><span id="id8.8.8" class="ltx_text ltx_font_bold">Peter Jansen<sup id="id3.3.3.1" class="ltx_sup"><math id="id3.3.3.1.m1.2" class="ltx_Math" alttext="1,2" display="inline"><semantics id="id3.3.3.1.m1.2a"><mrow id="id3.3.3.1.m1.2.3.2" xref="id3.3.3.1.m1.2.3.1.cmml"><mn id="id3.3.3.1.m1.1.1" xref="id3.3.3.1.m1.1.1.cmml">1</mn><mo id="id3.3.3.1.m1.2.3.2.1" xref="id3.3.3.1.m1.2.3.1.cmml">,</mo><mn id="id3.3.3.1.m1.2.2" xref="id3.3.3.1.m1.2.2.cmml">2</mn></mrow><annotation-xml encoding="MathML-Content" id="id3.3.3.1.m1.2b"><list id="id3.3.3.1.m1.2.3.1.cmml" xref="id3.3.3.1.m1.2.3.2"><cn type="integer" id="id3.3.3.1.m1.1.1.cmml" xref="id3.3.3.1.m1.1.1">1</cn><cn type="integer" id="id3.3.3.1.m1.2.2.cmml" xref="id3.3.3.1.m1.2.2">2</cn></list></annotation-xml><annotation encoding="application/x-tex" id="id3.3.3.1.m1.2c">1,2</annotation></semantics></math></sup>, Oyvind Tafjord<sup id="id4.4.4.2" class="ltx_sup"><math id="id4.4.4.2.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id4.4.4.2.m1.1a"><mn id="id4.4.4.2.m1.1.1" xref="id4.4.4.2.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id4.4.4.2.m1.1b"><cn type="integer" id="id4.4.4.2.m1.1.1.cmml" xref="id4.4.4.2.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id4.4.4.2.m1.1c">1</annotation></semantics></math></sup>, Niket Tandon<sup id="id5.5.5.3" class="ltx_sup"><math id="id5.5.5.3.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id5.5.5.3.m1.1a"><mn id="id5.5.5.3.m1.1.1" xref="id5.5.5.3.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id5.5.5.3.m1.1b"><cn type="integer" id="id5.5.5.3.m1.1.1.cmml" xref="id5.5.5.3.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id5.5.5.3.m1.1c">1</annotation></semantics></math></sup>, Li Zhang<sup id="id6.6.6.4" class="ltx_sup"><math id="id6.6.6.4.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id6.6.6.4.m1.1a"><mn id="id6.6.6.4.m1.1.1" xref="id6.6.6.4.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id6.6.6.4.m1.1b"><cn type="integer" id="id6.6.6.4.m1.1.1.cmml" xref="id6.6.6.4.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id6.6.6.4.m1.1c">3</annotation></semantics></math></sup>,
<br class="ltx_break">Chris-Callison Burch<sup id="id7.7.7.5" class="ltx_sup"><math id="id7.7.7.5.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id7.7.7.5.m1.1a"><mn id="id7.7.7.5.m1.1.1" xref="id7.7.7.5.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id7.7.7.5.m1.1b"><cn type="integer" id="id7.7.7.5.m1.1.1.cmml" xref="id7.7.7.5.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id7.7.7.5.m1.1c">3</annotation></semantics></math></sup>, Peter Clark<sup id="id8.8.8.6" class="ltx_sup"><math id="id8.8.8.6.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id8.8.8.6.m1.1a"><mn id="id8.8.8.6.m1.1.1" xref="id8.8.8.6.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id8.8.8.6.m1.1b"><cn type="integer" id="id8.8.8.6.m1.1.1.cmml" xref="id8.8.8.6.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id8.8.8.6.m1.1c">1</annotation></semantics></math></sup></span>

<br class="ltx_break"><sup id="id9.9.9" class="ltx_sup"><math id="id9.9.9.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id9.9.9.m1.1a"><mn id="id9.9.9.m1.1.1" xref="id9.9.9.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id9.9.9.m1.1b"><cn type="integer" id="id9.9.9.m1.1.1.cmml" xref="id9.9.9.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id9.9.9.m1.1c">1</annotation></semantics></math></sup>Allen Institute of AI
<br class="ltx_break"><sup id="id10.10.10" class="ltx_sup"><math id="id10.10.10.m1.1" class="ltx_Math" alttext="2" display="inline"><semantics id="id10.10.10.m1.1a"><mn id="id10.10.10.m1.1.1" xref="id10.10.10.m1.1.1.cmml">2</mn><annotation-xml encoding="MathML-Content" id="id10.10.10.m1.1b"><cn type="integer" id="id10.10.10.m1.1.1.cmml" xref="id10.10.10.m1.1.1">2</cn></annotation-xml><annotation encoding="application/x-tex" id="id10.10.10.m1.1c">2</annotation></semantics></math></sup>University of Arizona
<br class="ltx_break"><sup id="id11.11.11" class="ltx_sup"><math id="id11.11.11.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id11.11.11.m1.1a"><mn id="id11.11.11.m1.1.1" xref="id11.11.11.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id11.11.11.m1.1b"><cn type="integer" id="id11.11.11.m1.1.1.cmml" xref="id11.11.11.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id11.11.11.m1.1c">3</annotation></semantics></math></sup>University of Pennsylvania
<br class="ltx_break">
<br class="ltx_break">Contact: <span id="id12.12.id1" class="ltx_text ltx_font_typewriter">{bodhisattwam, bhavanad}@allenai.org
<br class="ltx_break"></span>Project page: <a target="_blank" href="https://allenai.github.io/clin/" title="" class="ltx_ref ltx_href ltx_font_typewriter">https://allenai.github.io/clin/</a>
</span></span>
</div>

<h1 class="ltx_title ltx_title_document">CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Bodhisattwa Prasad Majumder<sup id="id1.1.1" class="ltx_sup"><math id="id1.1.1.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id1.1.1.m1.1a"><mn id="id1.1.1.m1.1.1" xref="id1.1.1.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id1.1.1.m1.1b"><cn type="integer" id="id1.1.1.m1.1.1.cmml" xref="id1.1.1.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id1.1.1.m1.1c">1</annotation></semantics></math></sup>, Bhavana Dalvi Mishra<sup id="id2.2.2" class="ltx_sup"><math id="id2.2.2.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id2.2.2.m1.1a"><mn id="id2.2.2.m1.1.1" xref="id2.2.2.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id2.2.2.m1.1b"><cn type="integer" id="id2.2.2.m1.1.1.cmml" xref="id2.2.2.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id2.2.2.m1.1c">1</annotation></semantics></math></sup>,
<br class="ltx_break"><span id="id8.8.8" class="ltx_text ltx_font_bold">Peter Jansen<sup id="id3.3.3.1" class="ltx_sup"><math id="id3.3.3.1.m1.2" class="ltx_Math" alttext="1,2" display="inline"><semantics id="id3.3.3.1.m1.2a"><mrow id="id3.3.3.1.m1.2.3.2" xref="id3.3.3.1.m1.2.3.1.cmml"><mn id="id3.3.3.1.m1.1.1" xref="id3.3.3.1.m1.1.1.cmml">1</mn><mo id="id3.3.3.1.m1.2.3.2.1" xref="id3.3.3.1.m1.2.3.1.cmml">,</mo><mn id="id3.3.3.1.m1.2.2" xref="id3.3.3.1.m1.2.2.cmml">2</mn></mrow><annotation-xml encoding="MathML-Content" id="id3.3.3.1.m1.2b"><list id="id3.3.3.1.m1.2.3.1.cmml" xref="id3.3.3.1.m1.2.3.2"><cn type="integer" id="id3.3.3.1.m1.1.1.cmml" xref="id3.3.3.1.m1.1.1">1</cn><cn type="integer" id="id3.3.3.1.m1.2.2.cmml" xref="id3.3.3.1.m1.2.2">2</cn></list></annotation-xml><annotation encoding="application/x-tex" id="id3.3.3.1.m1.2c">1,2</annotation></semantics></math></sup>, Oyvind Tafjord<sup id="id4.4.4.2" class="ltx_sup"><math id="id4.4.4.2.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id4.4.4.2.m1.1a"><mn id="id4.4.4.2.m1.1.1" xref="id4.4.4.2.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id4.4.4.2.m1.1b"><cn type="integer" id="id4.4.4.2.m1.1.1.cmml" xref="id4.4.4.2.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id4.4.4.2.m1.1c">1</annotation></semantics></math></sup>, Niket Tandon<sup id="id5.5.5.3" class="ltx_sup"><math id="id5.5.5.3.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id5.5.5.3.m1.1a"><mn id="id5.5.5.3.m1.1.1" xref="id5.5.5.3.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id5.5.5.3.m1.1b"><cn type="integer" id="id5.5.5.3.m1.1.1.cmml" xref="id5.5.5.3.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id5.5.5.3.m1.1c">1</annotation></semantics></math></sup>, Li Zhang<sup id="id6.6.6.4" class="ltx_sup"><math id="id6.6.6.4.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id6.6.6.4.m1.1a"><mn id="id6.6.6.4.m1.1.1" xref="id6.6.6.4.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id6.6.6.4.m1.1b"><cn type="integer" id="id6.6.6.4.m1.1.1.cmml" xref="id6.6.6.4.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id6.6.6.4.m1.1c">3</annotation></semantics></math></sup>,
<br class="ltx_break">Chris-Callison Burch<sup id="id7.7.7.5" class="ltx_sup"><math id="id7.7.7.5.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id7.7.7.5.m1.1a"><mn id="id7.7.7.5.m1.1.1" xref="id7.7.7.5.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id7.7.7.5.m1.1b"><cn type="integer" id="id7.7.7.5.m1.1.1.cmml" xref="id7.7.7.5.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id7.7.7.5.m1.1c">3</annotation></semantics></math></sup>, Peter Clark<sup id="id8.8.8.6" class="ltx_sup"><math id="id8.8.8.6.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id8.8.8.6.m1.1a"><mn id="id8.8.8.6.m1.1.1" xref="id8.8.8.6.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id8.8.8.6.m1.1b"><cn type="integer" id="id8.8.8.6.m1.1.1.cmml" xref="id8.8.8.6.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id8.8.8.6.m1.1c">1</annotation></semantics></math></sup></span>

<br class="ltx_break"><sup id="id9.9.9" class="ltx_sup"><math id="id9.9.9.m1.1" class="ltx_Math" alttext="1" display="inline"><semantics id="id9.9.9.m1.1a"><mn id="id9.9.9.m1.1.1" xref="id9.9.9.m1.1.1.cmml">1</mn><annotation-xml encoding="MathML-Content" id="id9.9.9.m1.1b"><cn type="integer" id="id9.9.9.m1.1.1.cmml" xref="id9.9.9.m1.1.1">1</cn></annotation-xml><annotation encoding="application/x-tex" id="id9.9.9.m1.1c">1</annotation></semantics></math></sup>Allen Institute of AI
<br class="ltx_break"><sup id="id10.10.10" class="ltx_sup"><math id="id10.10.10.m1.1" class="ltx_Math" alttext="2" display="inline"><semantics id="id10.10.10.m1.1a"><mn id="id10.10.10.m1.1.1" xref="id10.10.10.m1.1.1.cmml">2</mn><annotation-xml encoding="MathML-Content" id="id10.10.10.m1.1b"><cn type="integer" id="id10.10.10.m1.1.1.cmml" xref="id10.10.10.m1.1.1">2</cn></annotation-xml><annotation encoding="application/x-tex" id="id10.10.10.m1.1c">2</annotation></semantics></math></sup>University of Arizona
<br class="ltx_break"><sup id="id11.11.11" class="ltx_sup"><math id="id11.11.11.m1.1" class="ltx_Math" alttext="3" display="inline"><semantics id="id11.11.11.m1.1a"><mn id="id11.11.11.m1.1.1" xref="id11.11.11.m1.1.1.cmml">3</mn><annotation-xml encoding="MathML-Content" id="id11.11.11.m1.1b"><cn type="integer" id="id11.11.11.m1.1.1.cmml" xref="id11.11.11.m1.1.1">3</cn></annotation-xml><annotation encoding="application/x-tex" id="id11.11.11.m1.1c">3</annotation></semantics></math></sup>University of Pennsylvania
<br class="ltx_break">
<br class="ltx_break">Contact: <span id="id12.12.id1" class="ltx_text ltx_font_typewriter">{bodhisattwam, bhavanad}@allenai.org
<br class="ltx_break"></span>Project page: <a target="_blank" href="https://allenai.github.io/clin/" title="" class="ltx_ref ltx_href ltx_font_typewriter">https://allenai.github.io/clin/</a>
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id13.id1" class="ltx_p">Language agents have shown some ability to interact with an external environment, e.g., a virtual
world such as ScienceWorld, to perform complex tasks, e.g., growing a plant, without the startup
costs of reinforcement learning. However, despite their zero-shot capabilities, these agents
to date do not continually improve over time, beyond performance refinement on a specific task.
Here we present CLIN, the first language-based agent to achieve this,
so that it continually improves over multiple trials, including when both the
environment and task are varied, and without requiring parameter updates.
Our approach is to use a persistent, dynamic, textual memory,
centered on <span id="id13.id1.1" class="ltx_text ltx_font_italic">causal abstractions</span> (rather than general “helpful hints”),
that is regularly updated after each trial so that the agent gradually learns useful
knowledge for new trials. In the ScienceWorld benchmark, CLIN is able to continually improve on repeated trials on the same task and environment,
outperforming state-of-the-art reflective language agents like Reflexion by 23 absolute points.
CLIN can also transfer its learning to new environments (or new tasks), improving its zero-shot performance
by 4 points (13 for new tasks) and can further improve performance there through continual memory updates, enhancing
performance by an additional 17 points (7 for new tasks).
This suggests a new architecture for agents built on frozen models that can still
continually and rapidly improve over time.</p>
</div>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para ltx_noindent">
<p id="S1.p1.1" class="ltx_p">Large language models (LLMs) have been increasingly used to interact with external
environments (e.g., simulated worlds) as goal-driven agents <cite class="ltx_cite ltx_citemacro_cite">Reed et al. (<a href="#bib.bib18" title="" class="ltx_ref">2022</a>)</cite>. However, it has been
challenging for these language agents to efficiently learn from trial-and-error
as traditional reinforcement learning methods require extensive training samples and
expensive model fine-tuning <cite class="ltx_cite ltx_citemacro_cite">Chen et al. (<a href="#bib.bib8" title="" class="ltx_ref">2021</a>); Ammanabrolu et al. (<a href="#bib.bib5" title="" class="ltx_ref">2020</a>)</cite>. More recently, new techniques have appeared in which
an agent reflects on its own past experience solving a task in a particular environment,
and generates language-based insights to help it retry the task, e.g., Reflexion <cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>)</cite>.
Such methods have the advantage of not requiring parameter updates (particularly useful given
the growing popularity of frozen language models). However, the style of such insights plays
a crucial role in performance, and not all insights improve generalization performance. For example,
a specific insight such as "In the next trial, I will go to desk 1 and find the lamp"
<cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>)</cite> may have limited value (or even hurt) for a different environment
or task.</p>
</div>
<figure id="S1.F1" class="ltx_figure"><img src="/html/2310.10134/assets/x1.png" id="S1.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="289" height="117" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>CLIN creates (Trial1) or adapts (Trial2+) a <span id="S1.F1.2.1" class="ltx_text ltx_font_bold">memory of causal abstractions</span> to help in future trials by reflecting on the last trial and current memory. It does this using a suitably prompted LLM to generate the updated memory (Section <a href="#S3.SS2" title="3.2 Memory: A collection of Adaptive Causal Abstractions ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.2</span></a>). Here, reflecting on Trial1, CLIN notes in memory that going to the kitchen helped with finding seeds, enabling it to find the seeds faster in Trial2. From there, it also learns that moving the seeds to the pot helped plant the seeds. To further generalize across episodes (sequences of trials, right figure) for use in new environments, CLIN generates a summary (“meta-memory”) of the best (starred) memories from each prior episode, here generating the generalization that moving to different rooms helps finding objects (Section <a href="#S3.SS3" title="3.3 Meta-Memory for Better Generalization ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.3</span></a>)
</figcaption>
</figure>
<div id="S1.p2" class="ltx_para ltx_noindent">
<p id="S1.p2.1" class="ltx_p">Our goal is a system that will continually improve over time, both while attempting
the same task in the same environment, and across different tasks and environments.
Our approach builds on prior work on reflection in two ways: First, we conjecture that a
specific <span id="S1.p2.1.1" class="ltx_text ltx_font_italic">style</span> of insight will be useful, namely one that captures <span id="S1.p2.1.2" class="ltx_text ltx_font_bold">causal abstractions</span>
about agent’s actions, e.g., “opening doors may be necessary for
movement between rooms”. Causal abstractions can potentially help the agent decide
which action to take in the future, and can be viewed as a kind of action model
learning <cite class="ltx_cite ltx_citemacro_cite">Arora et al. (<a href="#bib.bib7" title="" class="ltx_ref">2018</a>)</cite>, but placed in the modern context of language models.
Second, we maintain these abstractions in a <span id="S1.p2.1.3" class="ltx_text ltx_font_bold">continually
evolving, dynamic memory</span>, which is regularly updated as the agent gains experience,
allowing useful causal knowledge to persist (and unhelpful knowledge to be dropped)
over time and between tasks and environments, as illustrated in Figure <a href="#S1.F1" title="Figure 1 ‣ 1 Introduction ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>.</p>
</div>
<div id="S1.p3" class="ltx_para ltx_noindent">
<p id="S1.p3.1" class="ltx_p">We operationalize and evaluate this approach in a memory-augmented language agent called
CLIN (<span id="S1.p3.1.1" class="ltx_text ltx_font_bold">c</span>ontinual <span id="S1.p3.1.2" class="ltx_text ltx_font_bold">l</span>earning from <span id="S1.p3.1.3" class="ltx_text ltx_font_bold">in</span>teractions). CLIN is an agent that operates in
ScienceWorld <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib23" title="" class="ltx_ref">2022</a>)</cite>, a virtual, text-based environment in which an agent is
tasked with science-based goals, e.g., boiling a liquid, growing a plant. We find that
CLIN is able to rapidly learn about the environment and its action vocabulary and continually improve on repeated trials on the same task and environment,
outperforming state-of-the-art (SOTA) reflective language agents like Reflexion by 23 points.
CLIN can also transfer its learning to new environments (or tasks), improving its zero-shot performance
by 4 (13 for new tasks) points and can further improve performance through continual memory updates, enhancing
performance by an additional 17 (7 for new tasks) points. Our contributions are as follows:</p>
<ul id="S1.I1" class="ltx_itemize">
<li id="S1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i1.p1" class="ltx_para">
<p id="S1.I1.i1.p1.1" class="ltx_p">For memory-based language agents, we show that memory of
causal abstractions is effective at helping the agents learn over an extended
period and in varying conditions.</p>
</div>
</li>
<li id="S1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i2.p1" class="ltx_para ltx_noindent">
<p id="S1.I1.i2.p1.1" class="ltx_p">We describe and evaluate CLIN, an architecture for a novel nonparametric learning paradigm. We
show that CLIN learns faster than prior systems and generalizes better
to new tasks and new environments, achieving state-of-the-art.</p>
</div>
</li>
</ul>
<p id="S1.p3.2" class="ltx_p">Overall, this work suggests that a dynamically maintained memory, centered
around causal knowledge, is a promising way forward for agents built
on frozen models to continually improve over time.</p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Related Work </h2>

<div id="S2.p1" class="ltx_para ltx_noindent">
<p id="S2.p1.1" class="ltx_p">There is a long literature of work on agents that can navigate complex environments.
A common approach is to use reinforcement learning (RL), e.g., DRRN <cite class="ltx_cite ltx_citemacro_cite">He et al. (<a href="#bib.bib11" title="" class="ltx_ref">2015</a>)</cite>,
KG-A2C <cite class="ltx_cite ltx_citemacro_cite">Ammanabrolu &amp; Hausknecht (<a href="#bib.bib4" title="" class="ltx_ref">2020</a>)</cite>,
CALM <cite class="ltx_cite ltx_citemacro_cite">Yao et al. (<a href="#bib.bib24" title="" class="ltx_ref">2020</a>)</cite>,
where agents learn a task over repeated trials. However, while effective,
such agents typically require a large number of trials to learn and have trouble
adapting to unexpected changes in the test environment. More recently,
<cite class="ltx_cite ltx_citemacro_cite">Adaptive-Agent-Team et al. (<a href="#bib.bib1" title="" class="ltx_ref">2023</a>)</cite> demonstrated AdA, an agent that could rapidly adapt to open-ended
novel 3D problems, using meta-reinforcement learning, essentially being able
to change its policy on the fly. However, AdA required vast amounts of
pretraining, and this skill was still limited to the style of environments and
problems seen in pretraining.</p>
</div>
<div id="S2.p2" class="ltx_para ltx_noindent">
<p id="S2.p2.1" class="ltx_p">Recently, LLMs have provided a new tool for building
goal-directed agents <cite class="ltx_cite ltx_citemacro_cite">Huang et al. (<a href="#bib.bib12" title="" class="ltx_ref">2022</a>)</cite>. Given a linguistic description of the world state,
a task, and a history, the LLM can be prompted to suggest next actions
to take to achieve a goal, exploiting their wealth of semantic knowledge
about the world and requiring little training, e.g., SayCan <cite class="ltx_cite ltx_citemacro_cite">Ahn et al. (<a href="#bib.bib2" title="" class="ltx_ref">2022</a>)</cite>,
ReAct <cite class="ltx_cite ltx_citemacro_cite">Yao et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>)</cite>, and more recently SwiftSage <cite class="ltx_cite ltx_citemacro_cite">Lin et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite>, which
combines a supervised agent and a deliberative agent together. However, while
performing reasonably with little training data, such agents are unable
to learn and adapt from experience.</p>
</div>
<div id="S2.p3" class="ltx_para ltx_noindent">
<p id="S2.p3.1" class="ltx_p">Two recent systems have demonstrated how a frozen-model-based agent could
improve at a task. Voyager <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib22" title="" class="ltx_ref">2023</a>)</cite> operates in the world of Minecraft,
growing a (code-based) skill library from rich feedback of its failures.
Reflexion <cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>)</cite> improves at a task by <span id="S2.p3.1.1" class="ltx_text ltx_font_italic">reflecting</span>
on a failed attempt at that task and devise a new plan that accounted for
that mistake, used in the subsequent prompt to retry the task. While Reflexion did not have a long-term memory,
and its reflections were task- and environment-specific,
e.g., “In the next trial, I will go to desk 1 and find the lamp.”, we take
inspiration from it to build an agent, <span id="S2.p3.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span>, which continually maintains and
adapts a long-term, persistent memory of reflections, useful across
different trials, tasks, and environments.</p>
</div>
<div id="S2.p4" class="ltx_para ltx_noindent">
<p id="S2.p4.1" class="ltx_p">More generally, others have found that a memory of useful learnings can be
used to improve frozen LLM behavior, e.g., in QA
<cite class="ltx_cite ltx_citemacro_cite">Dalvi et al. (<a href="#bib.bib10" title="" class="ltx_ref">2022</a>); Tandon et al. (<a href="#bib.bib21" title="" class="ltx_ref">2022</a>); Madaan et al. (<a href="#bib.bib15" title="" class="ltx_ref">2023</a>)</cite>, or for modeling social behavior
<cite class="ltx_cite ltx_citemacro_cite">Park et al. (<a href="#bib.bib17" title="" class="ltx_ref">2023</a>)</cite>. We apply this finding to
goal-directed agents.</p>
</div>
<div id="S2.p5" class="ltx_para ltx_noindent">
<p id="S2.p5.1" class="ltx_p">Finally, we note that the <span id="S2.p5.1.1" class="ltx_text ltx_font_italic">content</span> of experiential memory is also important. Specifically, <span id="S2.p5.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> learns a memory of <span id="S2.p5.1.3" class="ltx_text ltx_font_italic">causal abstractions</span>, which
can be seen as learning a linguistic form of action model, describing the causal
effects of actions. While there has been substantial work in the
planning community of learning action models in a fully formalized context
<cite class="ltx_cite ltx_citemacro_cite">Arora et al. (<a href="#bib.bib7" title="" class="ltx_ref">2018</a>); Aineto et al. (<a href="#bib.bib3" title="" class="ltx_ref">2018</a>)</cite>,
<span id="S2.p5.1.4" class="ltx_text ltx_font_smallcaps">CLIN</span> loosely applies this idea in the linguistic world of LLM agents.</p>
</div>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Approach</h2>

<div id="S3.p1" class="ltx_para ltx_noindent">
<p id="S3.p1.1" class="ltx_p"><span id="S3.p1.1.1" class="ltx_text ltx_font_bold">Problem Formulation.</span>   
Sequential decision-making tasks require agents to repeatedly observe and then act in an environment until they accomplish a specific goal. At a high level, this can be accomplished by developing beliefs about the world, acting on the environment based on those beliefs, and then updating one’s beliefs based on the observed outcome. Here, we investigate constructing an agent that can continually update its beliefs through interaction and observation while exploiting its past experience toward solving unseen parametric variations of tasks.</p>
</div>
<div id="S3.p2" class="ltx_para ltx_noindent">
<p id="S3.p2.4" class="ltx_p"><span id="S3.p2.4.1" class="ltx_text ltx_font_bold">Setup.</span>    We investigate our continual learning agents in simulated environments. Our environments are modeled in a high-fidelity text-based simulator <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib23" title="" class="ltx_ref">2022</a>)</cite>, where both actions and observations are expressed in natural language. Let’s define the task space to be <math id="S3.p2.1.m1.1" class="ltx_Math" alttext="\mathcal{M}" display="inline"><semantics id="S3.p2.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.p2.1.m1.1.1" xref="S3.p2.1.m1.1.1.cmml">ℳ</mi><annotation-xml encoding="MathML-Content" id="S3.p2.1.m1.1b"><ci id="S3.p2.1.m1.1.1.cmml" xref="S3.p2.1.m1.1.1">ℳ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.1.m1.1c">\mathcal{M}</annotation></semantics></math>, a collection of partially observable Markov Decision Processes (POMDPs) that can be executed in a set of environment configurations <math id="S3.p2.2.m2.1" class="ltx_Math" alttext="\mathcal{E}" display="inline"><semantics id="S3.p2.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S3.p2.2.m2.1.1" xref="S3.p2.2.m2.1.1.cmml">ℰ</mi><annotation-xml encoding="MathML-Content" id="S3.p2.2.m2.1b"><ci id="S3.p2.2.m2.1.1.cmml" xref="S3.p2.2.m2.1.1">ℰ</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.2.m2.1c">\mathcal{E}</annotation></semantics></math>. Each task <math id="S3.p2.3.m3.1" class="ltx_Math" alttext="m\in\mathcal{M}" display="inline"><semantics id="S3.p2.3.m3.1a"><mrow id="S3.p2.3.m3.1.1" xref="S3.p2.3.m3.1.1.cmml"><mi id="S3.p2.3.m3.1.1.2" xref="S3.p2.3.m3.1.1.2.cmml">m</mi><mo id="S3.p2.3.m3.1.1.1" xref="S3.p2.3.m3.1.1.1.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S3.p2.3.m3.1.1.3" xref="S3.p2.3.m3.1.1.3.cmml">ℳ</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p2.3.m3.1b"><apply id="S3.p2.3.m3.1.1.cmml" xref="S3.p2.3.m3.1.1"><in id="S3.p2.3.m3.1.1.1.cmml" xref="S3.p2.3.m3.1.1.1"></in><ci id="S3.p2.3.m3.1.1.2.cmml" xref="S3.p2.3.m3.1.1.2">𝑚</ci><ci id="S3.p2.3.m3.1.1.3.cmml" xref="S3.p2.3.m3.1.1.3">ℳ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.3.m3.1c">m\in\mathcal{M}</annotation></semantics></math> has an initial state and a desired winning state, which vary depending on the environment <math id="S3.p2.4.m4.1" class="ltx_Math" alttext="e\in\mathcal{E}" display="inline"><semantics id="S3.p2.4.m4.1a"><mrow id="S3.p2.4.m4.1.1" xref="S3.p2.4.m4.1.1.cmml"><mi id="S3.p2.4.m4.1.1.2" xref="S3.p2.4.m4.1.1.2.cmml">e</mi><mo id="S3.p2.4.m4.1.1.1" xref="S3.p2.4.m4.1.1.1.cmml">∈</mo><mi class="ltx_font_mathcaligraphic" id="S3.p2.4.m4.1.1.3" xref="S3.p2.4.m4.1.1.3.cmml">ℰ</mi></mrow><annotation-xml encoding="MathML-Content" id="S3.p2.4.m4.1b"><apply id="S3.p2.4.m4.1.1.cmml" xref="S3.p2.4.m4.1.1"><in id="S3.p2.4.m4.1.1.1.cmml" xref="S3.p2.4.m4.1.1.1"></in><ci id="S3.p2.4.m4.1.1.2.cmml" xref="S3.p2.4.m4.1.1.2">𝑒</ci><ci id="S3.p2.4.m4.1.1.3.cmml" xref="S3.p2.4.m4.1.1.3">ℰ</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.p2.4.m4.1c">e\in\mathcal{E}</annotation></semantics></math>.</p>
</div>
<div id="S3.p3" class="ltx_para ltx_noindent">
<p id="S3.p3.6" class="ltx_p">Our setup allows an agent to attempt a task several times; each time is denoted as a <span id="S3.p3.6.1" class="ltx_text ltx_font_italic">trial</span>, <math id="S3.p3.1.m1.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="S3.p3.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.p3.1.m1.1.1" xref="S3.p3.1.m1.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="S3.p3.1.m1.1b"><ci id="S3.p3.1.m1.1.1.cmml" xref="S3.p3.1.m1.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.1.m1.1c">\mathcal{T}</annotation></semantics></math>, which consists of a total of <math id="S3.p3.2.m2.1" class="ltx_Math" alttext="\tau" display="inline"><semantics id="S3.p3.2.m2.1a"><mi id="S3.p3.2.m2.1.1" xref="S3.p3.2.m2.1.1.cmml">τ</mi><annotation-xml encoding="MathML-Content" id="S3.p3.2.m2.1b"><ci id="S3.p3.2.m2.1.1.cmml" xref="S3.p3.2.m2.1.1">𝜏</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.2.m2.1c">\tau</annotation></semantics></math> steps. Each step comprises an action by the agent (<math id="S3.p3.3.m3.1" class="ltx_Math" alttext="a" display="inline"><semantics id="S3.p3.3.m3.1a"><mi id="S3.p3.3.m3.1.1" xref="S3.p3.3.m3.1.1.cmml">a</mi><annotation-xml encoding="MathML-Content" id="S3.p3.3.m3.1b"><ci id="S3.p3.3.m3.1.1.cmml" xref="S3.p3.3.m3.1.1">𝑎</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.3.m3.1c">a</annotation></semantics></math>), and in response, the simulator returns the result of that action in the form of an observation (<math id="S3.p3.4.m4.1" class="ltx_Math" alttext="o" display="inline"><semantics id="S3.p3.4.m4.1a"><mi id="S3.p3.4.m4.1.1" xref="S3.p3.4.m4.1.1.cmml">o</mi><annotation-xml encoding="MathML-Content" id="S3.p3.4.m4.1b"><ci id="S3.p3.4.m4.1.1.cmml" xref="S3.p3.4.m4.1.1">𝑜</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.4.m4.1c">o</annotation></semantics></math>) and a reward (<math id="S3.p3.5.m5.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S3.p3.5.m5.1a"><mi id="S3.p3.5.m5.1.1" xref="S3.p3.5.m5.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S3.p3.5.m5.1b"><ci id="S3.p3.5.m5.1.1.cmml" xref="S3.p3.5.m5.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.5.m5.1c">r</annotation></semantics></math>). A collection of <math id="S3.p3.6.m6.1" class="ltx_Math" alttext="K" display="inline"><semantics id="S3.p3.6.m6.1a"><mi id="S3.p3.6.m6.1.1" xref="S3.p3.6.m6.1.1.cmml">K</mi><annotation-xml encoding="MathML-Content" id="S3.p3.6.m6.1b"><ci id="S3.p3.6.m6.1.1.cmml" xref="S3.p3.6.m6.1.1">𝐾</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.p3.6.m6.1c">K</annotation></semantics></math> trials is called an <span id="S3.p3.6.2" class="ltx_text ltx_font_italic">episode</span>. The environment gets reset when the task reaches an end state (such as completing, failing, or timing out). In our continual learning setup, the agent retains its memory across trials/episodes, reaping the benefits of continued interaction with the environment.</p>
</div>
<section id="S3.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.1 </span><span id="S3.SS1.1.1" class="ltx_text ltx_font_bold ltx_font_smallcaps">CLIN</span>: A Generative Agent <span id="S3.SS1.2.2" class="ltx_text ltx_font_bold">C</span>ontinually <span id="S3.SS1.3.3" class="ltx_text ltx_font_bold">L</span>earning from <span id="S3.SS1.4.4" class="ltx_text ltx_font_bold">IN</span>teractions</h3>

<div id="S3.SS1.p1" class="ltx_para ltx_noindent">
<p id="S3.SS1.p1.1" class="ltx_p">To act in the world, CLIN uses three modules: a <span id="S3.SS1.p1.1.1" class="ltx_text ltx_font_bold">memory</span>, a <span id="S3.SS1.p1.1.2" class="ltx_text ltx_font_bold">controller</span>, and an <span id="S3.SS1.p1.1.3" class="ltx_text ltx_font_bold">executor</span>, illustrated in Figure <a href="#S3.F2" title="Figure 2 ‣ 3.1 CLIN: A Generative Agent Continually Learning from INteractions ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> and which we now describe.
Learning then occurs using a fourth module, a <span id="S3.SS1.p1.1.4" class="ltx_text ltx_font_bold">memory generator</span>, to generate an updated memory after each trial.</p>
</div>
<figure id="S3.F2" class="ltx_figure"><img src="/html/2310.10134/assets/x2.png" id="S3.F2.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="289" height="105" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 2: </span>The architecture of CLIN. A <span id="S3.F2.4.1" class="ltx_text ltx_font_bold">controller</span> takes the current task, retrievals from memory, and the trial so far, to generate the next goal to achieve. The <span id="S3.F2.5.2" class="ltx_text ltx_font_bold">executor</span> then converts this to a valid action to perform towards that goal. The simulator then performs the action and returns an observation of that action’s effect. Memory is updated at the end of each trial by the <span id="S3.F2.6.3" class="ltx_text ltx_font_bold">memory generator</span> (Section <a href="#S3.SS2" title="3.2 Memory: A collection of Adaptive Causal Abstractions ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.2</span></a>).
</figcaption>
</figure>
<div id="S3.SS1.p2" class="ltx_para ltx_noindent">
<p id="S3.SS1.p2.1" class="ltx_p"><span id="S3.SS1.p2.1.1" class="ltx_text ltx_font_bold">Memory.</span>    CLIN’s memory (<math id="S3.SS1.p2.1.m1.1" class="ltx_Math" alttext="\mathcal{S}" display="inline"><semantics id="S3.SS1.p2.1.m1.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.p2.1.m1.1.1" xref="S3.SS1.p2.1.m1.1.1.cmml">𝒮</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p2.1.m1.1b"><ci id="S3.SS1.p2.1.m1.1.1.cmml" xref="S3.SS1.p2.1.m1.1.1">𝒮</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p2.1.m1.1c">\mathcal{S}</annotation></semantics></math>) is a persistent, dynamic collection of NL sentences that express causal abstractions, generated by reflecting on past experiences in order to help the agent perform better in the future. This generation process is described shortly in Section <a href="#S3.SS2" title="3.2 Memory: A collection of Adaptive Causal Abstractions ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3.2</span></a>.
For example, having an explicit causal insight, “opening the fridge is <span id="S3.SS1.p2.1.2" class="ltx_text ltx_font_typewriter">necessary</span> to access apple juice”, learned from past experiences, can reduce the action search space for <span id="S3.SS1.p2.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> while looking for “apple juice” in the same environment in future trials. To aid in continual learning, the memory also captures <span id="S3.SS1.p2.1.4" class="ltx_text ltx_font_italic">mistakes</span> made by the agent in previous trials, similar to reflective agents explored in recent work <cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>); Park et al. (<a href="#bib.bib17" title="" class="ltx_ref">2023</a>)</cite>, noting actions that failed to contribute to a task.</p>
</div>
<div id="S3.SS1.p3" class="ltx_para ltx_noindent">
<p id="S3.SS1.p3.4" class="ltx_p"><span id="S3.SS1.p3.4.1" class="ltx_text ltx_font_bold">Controller.</span>    The role of the controller is to generate the next goal to pursue in service of the task.
In CLIN it is a frozen LLM, whose prompt includes the current <span id="S3.SS1.p3.4.2" class="ltx_text ltx_font_bold">task</span> <math id="S3.SS1.p3.1.m1.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S3.SS1.p3.1.m1.1a"><mi id="S3.SS1.p3.1.m1.1.1" xref="S3.SS1.p3.1.m1.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.1.m1.1b"><ci id="S3.SS1.p3.1.m1.1.1.cmml" xref="S3.SS1.p3.1.m1.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.1.m1.1c">m</annotation></semantics></math>, e.g., “convert water into steam”, retrievals from the <span id="S3.SS1.p3.4.3" class="ltx_text ltx_font_bold">current memory</span> <math id="S3.SS1.p3.2.m2.1" class="ltx_Math" alttext="\mathcal{S}" display="inline"><semantics id="S3.SS1.p3.2.m2.1a"><mi class="ltx_font_mathcaligraphic" id="S3.SS1.p3.2.m2.1.1" xref="S3.SS1.p3.2.m2.1.1.cmml">𝒮</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.2.m2.1b"><ci id="S3.SS1.p3.2.m2.1.1.cmml" xref="S3.SS1.p3.2.m2.1.1">𝒮</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.2.m2.1c">\mathcal{S}</annotation></semantics></math>, and the <span id="S3.SS1.p3.4.4" class="ltx_text ltx_font_bold">trial so far</span> (the sequence of goal-action-observation triples {<math id="S3.SS1.p3.3.m3.7" class="ltx_Math" alttext="g_{1},a_{1},o_{1},...,g_{t},a_{t},o_{t}" display="inline"><semantics id="S3.SS1.p3.3.m3.7a"><mrow id="S3.SS1.p3.3.m3.7.7.6" xref="S3.SS1.p3.3.m3.7.7.7.cmml"><msub id="S3.SS1.p3.3.m3.2.2.1.1" xref="S3.SS1.p3.3.m3.2.2.1.1.cmml"><mi id="S3.SS1.p3.3.m3.2.2.1.1.2" xref="S3.SS1.p3.3.m3.2.2.1.1.2.cmml">g</mi><mn id="S3.SS1.p3.3.m3.2.2.1.1.3" xref="S3.SS1.p3.3.m3.2.2.1.1.3.cmml">1</mn></msub><mo id="S3.SS1.p3.3.m3.7.7.6.7" xref="S3.SS1.p3.3.m3.7.7.7.cmml">,</mo><msub id="S3.SS1.p3.3.m3.3.3.2.2" xref="S3.SS1.p3.3.m3.3.3.2.2.cmml"><mi id="S3.SS1.p3.3.m3.3.3.2.2.2" xref="S3.SS1.p3.3.m3.3.3.2.2.2.cmml">a</mi><mn id="S3.SS1.p3.3.m3.3.3.2.2.3" xref="S3.SS1.p3.3.m3.3.3.2.2.3.cmml">1</mn></msub><mo id="S3.SS1.p3.3.m3.7.7.6.8" xref="S3.SS1.p3.3.m3.7.7.7.cmml">,</mo><msub id="S3.SS1.p3.3.m3.4.4.3.3" xref="S3.SS1.p3.3.m3.4.4.3.3.cmml"><mi id="S3.SS1.p3.3.m3.4.4.3.3.2" xref="S3.SS1.p3.3.m3.4.4.3.3.2.cmml">o</mi><mn id="S3.SS1.p3.3.m3.4.4.3.3.3" xref="S3.SS1.p3.3.m3.4.4.3.3.3.cmml">1</mn></msub><mo id="S3.SS1.p3.3.m3.7.7.6.9" xref="S3.SS1.p3.3.m3.7.7.7.cmml">,</mo><mi mathvariant="normal" id="S3.SS1.p3.3.m3.1.1" xref="S3.SS1.p3.3.m3.1.1.cmml">…</mi><mo id="S3.SS1.p3.3.m3.7.7.6.10" xref="S3.SS1.p3.3.m3.7.7.7.cmml">,</mo><msub id="S3.SS1.p3.3.m3.5.5.4.4" xref="S3.SS1.p3.3.m3.5.5.4.4.cmml"><mi id="S3.SS1.p3.3.m3.5.5.4.4.2" xref="S3.SS1.p3.3.m3.5.5.4.4.2.cmml">g</mi><mi id="S3.SS1.p3.3.m3.5.5.4.4.3" xref="S3.SS1.p3.3.m3.5.5.4.4.3.cmml">t</mi></msub><mo id="S3.SS1.p3.3.m3.7.7.6.11" xref="S3.SS1.p3.3.m3.7.7.7.cmml">,</mo><msub id="S3.SS1.p3.3.m3.6.6.5.5" xref="S3.SS1.p3.3.m3.6.6.5.5.cmml"><mi id="S3.SS1.p3.3.m3.6.6.5.5.2" xref="S3.SS1.p3.3.m3.6.6.5.5.2.cmml">a</mi><mi id="S3.SS1.p3.3.m3.6.6.5.5.3" xref="S3.SS1.p3.3.m3.6.6.5.5.3.cmml">t</mi></msub><mo id="S3.SS1.p3.3.m3.7.7.6.12" xref="S3.SS1.p3.3.m3.7.7.7.cmml">,</mo><msub id="S3.SS1.p3.3.m3.7.7.6.6" xref="S3.SS1.p3.3.m3.7.7.6.6.cmml"><mi id="S3.SS1.p3.3.m3.7.7.6.6.2" xref="S3.SS1.p3.3.m3.7.7.6.6.2.cmml">o</mi><mi id="S3.SS1.p3.3.m3.7.7.6.6.3" xref="S3.SS1.p3.3.m3.7.7.6.6.3.cmml">t</mi></msub></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.3.m3.7b"><list id="S3.SS1.p3.3.m3.7.7.7.cmml" xref="S3.SS1.p3.3.m3.7.7.6"><apply id="S3.SS1.p3.3.m3.2.2.1.1.cmml" xref="S3.SS1.p3.3.m3.2.2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.2.2.1.1.1.cmml" xref="S3.SS1.p3.3.m3.2.2.1.1">subscript</csymbol><ci id="S3.SS1.p3.3.m3.2.2.1.1.2.cmml" xref="S3.SS1.p3.3.m3.2.2.1.1.2">𝑔</ci><cn type="integer" id="S3.SS1.p3.3.m3.2.2.1.1.3.cmml" xref="S3.SS1.p3.3.m3.2.2.1.1.3">1</cn></apply><apply id="S3.SS1.p3.3.m3.3.3.2.2.cmml" xref="S3.SS1.p3.3.m3.3.3.2.2"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.3.3.2.2.1.cmml" xref="S3.SS1.p3.3.m3.3.3.2.2">subscript</csymbol><ci id="S3.SS1.p3.3.m3.3.3.2.2.2.cmml" xref="S3.SS1.p3.3.m3.3.3.2.2.2">𝑎</ci><cn type="integer" id="S3.SS1.p3.3.m3.3.3.2.2.3.cmml" xref="S3.SS1.p3.3.m3.3.3.2.2.3">1</cn></apply><apply id="S3.SS1.p3.3.m3.4.4.3.3.cmml" xref="S3.SS1.p3.3.m3.4.4.3.3"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.4.4.3.3.1.cmml" xref="S3.SS1.p3.3.m3.4.4.3.3">subscript</csymbol><ci id="S3.SS1.p3.3.m3.4.4.3.3.2.cmml" xref="S3.SS1.p3.3.m3.4.4.3.3.2">𝑜</ci><cn type="integer" id="S3.SS1.p3.3.m3.4.4.3.3.3.cmml" xref="S3.SS1.p3.3.m3.4.4.3.3.3">1</cn></apply><ci id="S3.SS1.p3.3.m3.1.1.cmml" xref="S3.SS1.p3.3.m3.1.1">…</ci><apply id="S3.SS1.p3.3.m3.5.5.4.4.cmml" xref="S3.SS1.p3.3.m3.5.5.4.4"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.5.5.4.4.1.cmml" xref="S3.SS1.p3.3.m3.5.5.4.4">subscript</csymbol><ci id="S3.SS1.p3.3.m3.5.5.4.4.2.cmml" xref="S3.SS1.p3.3.m3.5.5.4.4.2">𝑔</ci><ci id="S3.SS1.p3.3.m3.5.5.4.4.3.cmml" xref="S3.SS1.p3.3.m3.5.5.4.4.3">𝑡</ci></apply><apply id="S3.SS1.p3.3.m3.6.6.5.5.cmml" xref="S3.SS1.p3.3.m3.6.6.5.5"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.6.6.5.5.1.cmml" xref="S3.SS1.p3.3.m3.6.6.5.5">subscript</csymbol><ci id="S3.SS1.p3.3.m3.6.6.5.5.2.cmml" xref="S3.SS1.p3.3.m3.6.6.5.5.2">𝑎</ci><ci id="S3.SS1.p3.3.m3.6.6.5.5.3.cmml" xref="S3.SS1.p3.3.m3.6.6.5.5.3">𝑡</ci></apply><apply id="S3.SS1.p3.3.m3.7.7.6.6.cmml" xref="S3.SS1.p3.3.m3.7.7.6.6"><csymbol cd="ambiguous" id="S3.SS1.p3.3.m3.7.7.6.6.1.cmml" xref="S3.SS1.p3.3.m3.7.7.6.6">subscript</csymbol><ci id="S3.SS1.p3.3.m3.7.7.6.6.2.cmml" xref="S3.SS1.p3.3.m3.7.7.6.6.2">𝑜</ci><ci id="S3.SS1.p3.3.m3.7.7.6.6.3.cmml" xref="S3.SS1.p3.3.m3.7.7.6.6.3">𝑡</ci></apply></list></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.3.m3.7c">g_{1},a_{1},o_{1},...,g_{t},a_{t},o_{t}</annotation></semantics></math>}), and is prompted to output the next
<span id="S3.SS1.p3.4.5" class="ltx_text ltx_font_bold">goal</span> <math id="S3.SS1.p3.4.m4.1" class="ltx_Math" alttext="g_{t+1}" display="inline"><semantics id="S3.SS1.p3.4.m4.1a"><msub id="S3.SS1.p3.4.m4.1.1" xref="S3.SS1.p3.4.m4.1.1.cmml"><mi id="S3.SS1.p3.4.m4.1.1.2" xref="S3.SS1.p3.4.m4.1.1.2.cmml">g</mi><mrow id="S3.SS1.p3.4.m4.1.1.3" xref="S3.SS1.p3.4.m4.1.1.3.cmml"><mi id="S3.SS1.p3.4.m4.1.1.3.2" xref="S3.SS1.p3.4.m4.1.1.3.2.cmml">t</mi><mo id="S3.SS1.p3.4.m4.1.1.3.1" xref="S3.SS1.p3.4.m4.1.1.3.1.cmml">+</mo><mn id="S3.SS1.p3.4.m4.1.1.3.3" xref="S3.SS1.p3.4.m4.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p3.4.m4.1b"><apply id="S3.SS1.p3.4.m4.1.1.cmml" xref="S3.SS1.p3.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS1.p3.4.m4.1.1.1.cmml" xref="S3.SS1.p3.4.m4.1.1">subscript</csymbol><ci id="S3.SS1.p3.4.m4.1.1.2.cmml" xref="S3.SS1.p3.4.m4.1.1.2">𝑔</ci><apply id="S3.SS1.p3.4.m4.1.1.3.cmml" xref="S3.SS1.p3.4.m4.1.1.3"><plus id="S3.SS1.p3.4.m4.1.1.3.1.cmml" xref="S3.SS1.p3.4.m4.1.1.3.1"></plus><ci id="S3.SS1.p3.4.m4.1.1.3.2.cmml" xref="S3.SS1.p3.4.m4.1.1.3.2">𝑡</ci><cn type="integer" id="S3.SS1.p3.4.m4.1.1.3.3.cmml" xref="S3.SS1.p3.4.m4.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p3.4.m4.1c">g_{t+1}</annotation></semantics></math> to pursue, e.g., “find water”. Memory items are retrieved using both the task instruction and the trial history. The controller first selects one or more memory items given the current state and if they are useful for the next action to progress in the task. After that, it appends the learning, if selected, in context to generate a goal, otherwise the goal is generated based on the trial history (see full prompt in <a href="#A1.F6" title="In Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">6</span></a>).</p>
</div>
<div id="S3.SS1.p4" class="ltx_para ltx_noindent">
<p id="S3.SS1.p4.3" class="ltx_p"><span id="S3.SS1.p4.3.1" class="ltx_text ltx_font_bold">Executor.</span>    The role of the executor is to convert the generated goal <math id="S3.SS1.p4.1.m1.1" class="ltx_Math" alttext="g_{t+1}" display="inline"><semantics id="S3.SS1.p4.1.m1.1a"><msub id="S3.SS1.p4.1.m1.1.1" xref="S3.SS1.p4.1.m1.1.1.cmml"><mi id="S3.SS1.p4.1.m1.1.1.2" xref="S3.SS1.p4.1.m1.1.1.2.cmml">g</mi><mrow id="S3.SS1.p4.1.m1.1.1.3" xref="S3.SS1.p4.1.m1.1.1.3.cmml"><mi id="S3.SS1.p4.1.m1.1.1.3.2" xref="S3.SS1.p4.1.m1.1.1.3.2.cmml">t</mi><mo id="S3.SS1.p4.1.m1.1.1.3.1" xref="S3.SS1.p4.1.m1.1.1.3.1.cmml">+</mo><mn id="S3.SS1.p4.1.m1.1.1.3.3" xref="S3.SS1.p4.1.m1.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.1.m1.1b"><apply id="S3.SS1.p4.1.m1.1.1.cmml" xref="S3.SS1.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.1.m1.1.1.1.cmml" xref="S3.SS1.p4.1.m1.1.1">subscript</csymbol><ci id="S3.SS1.p4.1.m1.1.1.2.cmml" xref="S3.SS1.p4.1.m1.1.1.2">𝑔</ci><apply id="S3.SS1.p4.1.m1.1.1.3.cmml" xref="S3.SS1.p4.1.m1.1.1.3"><plus id="S3.SS1.p4.1.m1.1.1.3.1.cmml" xref="S3.SS1.p4.1.m1.1.1.3.1"></plus><ci id="S3.SS1.p4.1.m1.1.1.3.2.cmml" xref="S3.SS1.p4.1.m1.1.1.3.2">𝑡</ci><cn type="integer" id="S3.SS1.p4.1.m1.1.1.3.3.cmml" xref="S3.SS1.p4.1.m1.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.1.m1.1c">g_{t+1}</annotation></semantics></math> into a valid <span id="S3.SS1.p4.3.2" class="ltx_text ltx_font_bold">action</span> <math id="S3.SS1.p4.2.m2.1" class="ltx_Math" alttext="a_{t+1}" display="inline"><semantics id="S3.SS1.p4.2.m2.1a"><msub id="S3.SS1.p4.2.m2.1.1" xref="S3.SS1.p4.2.m2.1.1.cmml"><mi id="S3.SS1.p4.2.m2.1.1.2" xref="S3.SS1.p4.2.m2.1.1.2.cmml">a</mi><mrow id="S3.SS1.p4.2.m2.1.1.3" xref="S3.SS1.p4.2.m2.1.1.3.cmml"><mi id="S3.SS1.p4.2.m2.1.1.3.2" xref="S3.SS1.p4.2.m2.1.1.3.2.cmml">t</mi><mo id="S3.SS1.p4.2.m2.1.1.3.1" xref="S3.SS1.p4.2.m2.1.1.3.1.cmml">+</mo><mn id="S3.SS1.p4.2.m2.1.1.3.3" xref="S3.SS1.p4.2.m2.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.2.m2.1b"><apply id="S3.SS1.p4.2.m2.1.1.cmml" xref="S3.SS1.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.2.m2.1.1.1.cmml" xref="S3.SS1.p4.2.m2.1.1">subscript</csymbol><ci id="S3.SS1.p4.2.m2.1.1.2.cmml" xref="S3.SS1.p4.2.m2.1.1.2">𝑎</ci><apply id="S3.SS1.p4.2.m2.1.1.3.cmml" xref="S3.SS1.p4.2.m2.1.1.3"><plus id="S3.SS1.p4.2.m2.1.1.3.1.cmml" xref="S3.SS1.p4.2.m2.1.1.3.1"></plus><ci id="S3.SS1.p4.2.m2.1.1.3.2.cmml" xref="S3.SS1.p4.2.m2.1.1.3.2">𝑡</ci><cn type="integer" id="S3.SS1.p4.2.m2.1.1.3.3.cmml" xref="S3.SS1.p4.2.m2.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.2.m2.1c">a_{t+1}</annotation></semantics></math> that can be executed in the environment in pursuit of that goal.
Again a (frozen) LLM is used, whose prompt includes the goal <math id="S3.SS1.p4.3.m3.1" class="ltx_Math" alttext="g_{t+1}" display="inline"><semantics id="S3.SS1.p4.3.m3.1a"><msub id="S3.SS1.p4.3.m3.1.1" xref="S3.SS1.p4.3.m3.1.1.cmml"><mi id="S3.SS1.p4.3.m3.1.1.2" xref="S3.SS1.p4.3.m3.1.1.2.cmml">g</mi><mrow id="S3.SS1.p4.3.m3.1.1.3" xref="S3.SS1.p4.3.m3.1.1.3.cmml"><mi id="S3.SS1.p4.3.m3.1.1.3.2" xref="S3.SS1.p4.3.m3.1.1.3.2.cmml">t</mi><mo id="S3.SS1.p4.3.m3.1.1.3.1" xref="S3.SS1.p4.3.m3.1.1.3.1.cmml">+</mo><mn id="S3.SS1.p4.3.m3.1.1.3.3" xref="S3.SS1.p4.3.m3.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p4.3.m3.1b"><apply id="S3.SS1.p4.3.m3.1.1.cmml" xref="S3.SS1.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS1.p4.3.m3.1.1.1.cmml" xref="S3.SS1.p4.3.m3.1.1">subscript</csymbol><ci id="S3.SS1.p4.3.m3.1.1.2.cmml" xref="S3.SS1.p4.3.m3.1.1.2">𝑔</ci><apply id="S3.SS1.p4.3.m3.1.1.3.cmml" xref="S3.SS1.p4.3.m3.1.1.3"><plus id="S3.SS1.p4.3.m3.1.1.3.1.cmml" xref="S3.SS1.p4.3.m3.1.1.3.1"></plus><ci id="S3.SS1.p4.3.m3.1.1.3.2.cmml" xref="S3.SS1.p4.3.m3.1.1.3.2">𝑡</ci><cn type="integer" id="S3.SS1.p4.3.m3.1.1.3.3.cmml" xref="S3.SS1.p4.3.m3.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p4.3.m3.1c">g_{t+1}</annotation></semantics></math>, the trial so far, and all the possible actions that can be performed in the
current state (provided by the simulator, as is standard practice in current generative agent research <cite class="ltx_cite ltx_citemacro_cite">Ahn et al. (<a href="#bib.bib2" title="" class="ltx_ref">2022</a>); Yao et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>); Lin et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>); Park et al. (<a href="#bib.bib17" title="" class="ltx_ref">2023</a>)</cite>).
The list of possible actions is expressed as possible action templates and available objects that can instantiate them, rather than
a combinatorially large enumeration of possible actions. The model is then prompted to generate a candidate action to perform (see prompt in <a href="#A1.F6" title="In Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">6</span></a>). Finally, CLIN checks this
candidate action is one of the valid actions. If it is not, it finds the most similar valid action using the pre-trained embeddings from the <span id="S3.SS1.p4.3.3" class="ltx_text ltx_font_typewriter">sentence-transformer</span> model <cite class="ltx_cite ltx_citemacro_cite">Reimers &amp; Gurevych (<a href="#bib.bib19" title="" class="ltx_ref">2019</a>)</cite>. If the top-ranked valid action has a similarity score greater than a threshold (here, 0.9, chosen as a hyperparameter), the action is selected. Otherwise, we perform iterative refinement <cite class="ltx_cite ltx_citemacro_cite">Madaan et al. (<a href="#bib.bib15" title="" class="ltx_ref">2023</a>)</cite> by suffixing the context with feedback that the generated candidate action is not executable.
This allows the executor to retry the generation for up to a maximum number of tries (here, 5).</p>
</div>
<div id="S3.SS1.p5" class="ltx_para ltx_noindent">
<p id="S3.SS1.p5.3" class="ltx_p">Finally, upon executing the action <math id="S3.SS1.p5.1.m1.1" class="ltx_Math" alttext="a_{t+1}" display="inline"><semantics id="S3.SS1.p5.1.m1.1a"><msub id="S3.SS1.p5.1.m1.1.1" xref="S3.SS1.p5.1.m1.1.1.cmml"><mi id="S3.SS1.p5.1.m1.1.1.2" xref="S3.SS1.p5.1.m1.1.1.2.cmml">a</mi><mrow id="S3.SS1.p5.1.m1.1.1.3" xref="S3.SS1.p5.1.m1.1.1.3.cmml"><mi id="S3.SS1.p5.1.m1.1.1.3.2" xref="S3.SS1.p5.1.m1.1.1.3.2.cmml">t</mi><mo id="S3.SS1.p5.1.m1.1.1.3.1" xref="S3.SS1.p5.1.m1.1.1.3.1.cmml">+</mo><mn id="S3.SS1.p5.1.m1.1.1.3.3" xref="S3.SS1.p5.1.m1.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.SS1.p5.1.m1.1b"><apply id="S3.SS1.p5.1.m1.1.1.cmml" xref="S3.SS1.p5.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS1.p5.1.m1.1.1.1.cmml" xref="S3.SS1.p5.1.m1.1.1">subscript</csymbol><ci id="S3.SS1.p5.1.m1.1.1.2.cmml" xref="S3.SS1.p5.1.m1.1.1.2">𝑎</ci><apply id="S3.SS1.p5.1.m1.1.1.3.cmml" xref="S3.SS1.p5.1.m1.1.1.3"><plus id="S3.SS1.p5.1.m1.1.1.3.1.cmml" xref="S3.SS1.p5.1.m1.1.1.3.1"></plus><ci id="S3.SS1.p5.1.m1.1.1.3.2.cmml" xref="S3.SS1.p5.1.m1.1.1.3.2">𝑡</ci><cn type="integer" id="S3.SS1.p5.1.m1.1.1.3.3.cmml" xref="S3.SS1.p5.1.m1.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p5.1.m1.1c">a_{t+1}</annotation></semantics></math>, <span id="S3.SS1.p5.3.1" class="ltx_text ltx_font_smallcaps">CLIN</span> receives a partial next state, as an <span id="S3.SS1.p5.3.2" class="ltx_text ltx_font_bold">observation</span>, from the simulator and the reward (<math id="S3.SS1.p5.2.m2.1" class="ltx_Math" alttext="r" display="inline"><semantics id="S3.SS1.p5.2.m2.1a"><mi id="S3.SS1.p5.2.m2.1.1" xref="S3.SS1.p5.2.m2.1.1.cmml">r</mi><annotation-xml encoding="MathML-Content" id="S3.SS1.p5.2.m2.1b"><ci id="S3.SS1.p5.2.m2.1.1.cmml" xref="S3.SS1.p5.2.m2.1.1">𝑟</ci></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p5.2.m2.1c">r</annotation></semantics></math>) <math id="S3.SS1.p5.3.m3.2" class="ltx_Math" alttext="\in[0,1]" display="inline"><semantics id="S3.SS1.p5.3.m3.2a"><mrow id="S3.SS1.p5.3.m3.2.3" xref="S3.SS1.p5.3.m3.2.3.cmml"><mi id="S3.SS1.p5.3.m3.2.3.2" xref="S3.SS1.p5.3.m3.2.3.2.cmml"></mi><mo id="S3.SS1.p5.3.m3.2.3.1" xref="S3.SS1.p5.3.m3.2.3.1.cmml">∈</mo><mrow id="S3.SS1.p5.3.m3.2.3.3.2" xref="S3.SS1.p5.3.m3.2.3.3.1.cmml"><mo stretchy="false" id="S3.SS1.p5.3.m3.2.3.3.2.1" xref="S3.SS1.p5.3.m3.2.3.3.1.cmml">[</mo><mn id="S3.SS1.p5.3.m3.1.1" xref="S3.SS1.p5.3.m3.1.1.cmml">0</mn><mo id="S3.SS1.p5.3.m3.2.3.3.2.2" xref="S3.SS1.p5.3.m3.2.3.3.1.cmml">,</mo><mn id="S3.SS1.p5.3.m3.2.2" xref="S3.SS1.p5.3.m3.2.2.cmml">1</mn><mo stretchy="false" id="S3.SS1.p5.3.m3.2.3.3.2.3" xref="S3.SS1.p5.3.m3.2.3.3.1.cmml">]</mo></mrow></mrow><annotation-xml encoding="MathML-Content" id="S3.SS1.p5.3.m3.2b"><apply id="S3.SS1.p5.3.m3.2.3.cmml" xref="S3.SS1.p5.3.m3.2.3"><in id="S3.SS1.p5.3.m3.2.3.1.cmml" xref="S3.SS1.p5.3.m3.2.3.1"></in><csymbol cd="latexml" id="S3.SS1.p5.3.m3.2.3.2.cmml" xref="S3.SS1.p5.3.m3.2.3.2">absent</csymbol><interval closure="closed" id="S3.SS1.p5.3.m3.2.3.3.1.cmml" xref="S3.SS1.p5.3.m3.2.3.3.2"><cn type="integer" id="S3.SS1.p5.3.m3.1.1.cmml" xref="S3.SS1.p5.3.m3.1.1">0</cn><cn type="integer" id="S3.SS1.p5.3.m3.2.2.cmml" xref="S3.SS1.p5.3.m3.2.2">1</cn></interval></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS1.p5.3.m3.2c">\in[0,1]</annotation></semantics></math>. Rewards are nominally given by the simulator for achieving either major task subgoals (e.g., finding water, for the boiling task), or minor and optional subgoals (e.g., being in the kitchen, for the boiling task). Rewards are sparse and generally only supplied after the completion of a task subgoal. A snapshot of a full trial is given in lines 4-10 in Algorithm <a href="#alg1" title="Algorithm 1 ‣ Figure 3 ‣ 3.1 CLIN: A Generative Agent Continually Learning from INteractions ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>.</p>
</div>
<div id="S3.SS1.p6" class="ltx_para ltx_noindent">
<p id="S3.SS1.p6.1" class="ltx_p">Note that CLIN does not make use of any gold data to identify goals and memories. Rather, we expect <span id="S3.SS1.p6.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> to perform a balanced act of exploration-exploitation by interacting, learning, and adapting to unseen tasks or environment configurations—a key difference from few-shot generative agents by previous work <cite class="ltx_cite ltx_citemacro_cite">Ahn et al. (<a href="#bib.bib2" title="" class="ltx_ref">2022</a>); Yao et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>); Lin et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>); Park et al. (<a href="#bib.bib17" title="" class="ltx_ref">2023</a>)</cite>.</p>
</div>
<figure id="S3.F3" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_1">
<figure id="alg1" class="ltx_float ltx_figure_panel ltx_minipage ltx_align_middle ltx_framed ltx_framed_top" style="width:198.7pt;">
<div id="alg1.2" class="ltx_listing ltx_listing">
<div id="alg0.l1" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">1:</span><span id="alg0.l1.1" class="ltx_text ltx_font_bold">procedure</span> <span id="alg0.l1.2" class="ltx_text ltx_font_smallcaps">Adaptation</span>(Task: <math id="alg0.l1.m1.1" class="ltx_Math" alttext="m" display="inline"><semantics id="alg0.l1.m1.1a"><mi id="alg0.l1.m1.1.1" xref="alg0.l1.m1.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="alg0.l1.m1.1b"><ci id="alg0.l1.m1.1.1.cmml" xref="alg0.l1.m1.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m1.1c">m</annotation></semantics></math>, Env: <math id="alg0.l1.m2.1" class="ltx_Math" alttext="e" display="inline"><semantics id="alg0.l1.m2.1a"><mi id="alg0.l1.m2.1.1" xref="alg0.l1.m2.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="alg0.l1.m2.1b"><ci id="alg0.l1.m2.1.1.cmml" xref="alg0.l1.m2.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m2.1c">e</annotation></semantics></math>, Memory: <math id="alg0.l1.m3.1" class="ltx_Math" alttext="\mathcal{S}" display="inline"><semantics id="alg0.l1.m3.1a"><mi class="ltx_font_mathcaligraphic" id="alg0.l1.m3.1.1" xref="alg0.l1.m3.1.1.cmml">𝒮</mi><annotation-xml encoding="MathML-Content" id="alg0.l1.m3.1b"><ci id="alg0.l1.m3.1.1.cmml" xref="alg0.l1.m3.1.1">𝒮</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l1.m3.1c">\mathcal{S}</annotation></semantics></math>):

</div>
<div id="alg0.l2" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">2:</span>   Initialize Memory: <math id="alg0.l2.m1.1" class="ltx_Math" alttext="\mathcal{S}_{0}" display="inline"><semantics id="alg0.l2.m1.1a"><msub id="alg0.l2.m1.1.1" xref="alg0.l2.m1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l2.m1.1.1.2" xref="alg0.l2.m1.1.1.2.cmml">𝒮</mi><mn id="alg0.l2.m1.1.1.3" xref="alg0.l2.m1.1.1.3.cmml">0</mn></msub><annotation-xml encoding="MathML-Content" id="alg0.l2.m1.1b"><apply id="alg0.l2.m1.1.1.cmml" xref="alg0.l2.m1.1.1"><csymbol cd="ambiguous" id="alg0.l2.m1.1.1.1.cmml" xref="alg0.l2.m1.1.1">subscript</csymbol><ci id="alg0.l2.m1.1.1.2.cmml" xref="alg0.l2.m1.1.1.2">𝒮</ci><cn type="integer" id="alg0.l2.m1.1.1.3.cmml" xref="alg0.l2.m1.1.1.3">0</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l2.m1.1c">\mathcal{S}_{0}</annotation></semantics></math>

</div>
<div id="alg0.l3" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">3:</span>   <span id="alg0.l3.1" class="ltx_text ltx_font_bold">for</span> <math id="alg0.l3.m1.3" class="ltx_Math" alttext="k\in 1,\cdots,K" display="inline"><semantics id="alg0.l3.m1.3a"><mrow id="alg0.l3.m1.3.4" xref="alg0.l3.m1.3.4.cmml"><mi id="alg0.l3.m1.3.4.2" xref="alg0.l3.m1.3.4.2.cmml">k</mi><mo id="alg0.l3.m1.3.4.1" xref="alg0.l3.m1.3.4.1.cmml">∈</mo><mrow id="alg0.l3.m1.3.4.3.2" xref="alg0.l3.m1.3.4.3.1.cmml"><mn id="alg0.l3.m1.1.1" xref="alg0.l3.m1.1.1.cmml">1</mn><mo id="alg0.l3.m1.3.4.3.2.1" xref="alg0.l3.m1.3.4.3.1.cmml">,</mo><mi mathvariant="normal" id="alg0.l3.m1.2.2" xref="alg0.l3.m1.2.2.cmml">⋯</mi><mo id="alg0.l3.m1.3.4.3.2.2" xref="alg0.l3.m1.3.4.3.1.cmml">,</mo><mi id="alg0.l3.m1.3.3" xref="alg0.l3.m1.3.3.cmml">K</mi></mrow></mrow><annotation-xml encoding="MathML-Content" id="alg0.l3.m1.3b"><apply id="alg0.l3.m1.3.4.cmml" xref="alg0.l3.m1.3.4"><in id="alg0.l3.m1.3.4.1.cmml" xref="alg0.l3.m1.3.4.1"></in><ci id="alg0.l3.m1.3.4.2.cmml" xref="alg0.l3.m1.3.4.2">𝑘</ci><list id="alg0.l3.m1.3.4.3.1.cmml" xref="alg0.l3.m1.3.4.3.2"><cn type="integer" id="alg0.l3.m1.1.1.cmml" xref="alg0.l3.m1.1.1">1</cn><ci id="alg0.l3.m1.2.2.cmml" xref="alg0.l3.m1.2.2">⋯</ci><ci id="alg0.l3.m1.3.3.cmml" xref="alg0.l3.m1.3.3">𝐾</ci></list></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l3.m1.3c">k\in 1,\cdots,K</annotation></semantics></math> <span id="alg0.l3.2" class="ltx_text ltx_font_bold">do</span>:

</div>
<div id="alg0.l4" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">4:</span>     Intialize Trial <math id="alg0.l4.m1.1" class="ltx_Math" alttext="\mathcal{T}" display="inline"><semantics id="alg0.l4.m1.1a"><mi class="ltx_font_mathcaligraphic" id="alg0.l4.m1.1.1" xref="alg0.l4.m1.1.1.cmml">𝒯</mi><annotation-xml encoding="MathML-Content" id="alg0.l4.m1.1b"><ci id="alg0.l4.m1.1.1.cmml" xref="alg0.l4.m1.1.1">𝒯</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l4.m1.1c">\mathcal{T}</annotation></semantics></math>, <math id="alg0.l4.m2.1" class="ltx_Math" alttext="t" display="inline"><semantics id="alg0.l4.m2.1a"><mi id="alg0.l4.m2.1.1" xref="alg0.l4.m2.1.1.cmml">t</mi><annotation-xml encoding="MathML-Content" id="alg0.l4.m2.1b"><ci id="alg0.l4.m2.1.1.cmml" xref="alg0.l4.m2.1.1">𝑡</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l4.m2.1c">t</annotation></semantics></math>

</div>
<div id="alg0.l5" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">5:</span>     <span id="alg0.l5.1" class="ltx_text ltx_font_bold">while</span> <math id="alg0.l5.m1.1" class="ltx_Math" alttext="t&lt;" display="inline"><semantics id="alg0.l5.m1.1a"><mrow id="alg0.l5.m1.1.1" xref="alg0.l5.m1.1.1.cmml"><mi id="alg0.l5.m1.1.1.2" xref="alg0.l5.m1.1.1.2.cmml">t</mi><mo id="alg0.l5.m1.1.1.1" xref="alg0.l5.m1.1.1.1.cmml">&lt;</mo><mi id="alg0.l5.m1.1.1.3" xref="alg0.l5.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l5.m1.1b"><apply id="alg0.l5.m1.1.1.cmml" xref="alg0.l5.m1.1.1"><lt id="alg0.l5.m1.1.1.1.cmml" xref="alg0.l5.m1.1.1.1"></lt><ci id="alg0.l5.m1.1.1.2.cmml" xref="alg0.l5.m1.1.1.2">𝑡</ci><csymbol cd="latexml" id="alg0.l5.m1.1.1.3.cmml" xref="alg0.l5.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l5.m1.1c">t&lt;</annotation></semantics></math> max. steps or task not complete <span id="alg0.l5.2" class="ltx_text ltx_font_bold">do</span>:

</div>
<div id="alg0.l6" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">6:</span>       <math id="alg0.l6.m1.1" class="ltx_Math" alttext="g_{t}=" display="inline"><semantics id="alg0.l6.m1.1a"><mrow id="alg0.l6.m1.1.1" xref="alg0.l6.m1.1.1.cmml"><msub id="alg0.l6.m1.1.1.2" xref="alg0.l6.m1.1.1.2.cmml"><mi id="alg0.l6.m1.1.1.2.2" xref="alg0.l6.m1.1.1.2.2.cmml">g</mi><mi id="alg0.l6.m1.1.1.2.3" xref="alg0.l6.m1.1.1.2.3.cmml">t</mi></msub><mo id="alg0.l6.m1.1.1.1" xref="alg0.l6.m1.1.1.1.cmml">=</mo><mi id="alg0.l6.m1.1.1.3" xref="alg0.l6.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l6.m1.1b"><apply id="alg0.l6.m1.1.1.cmml" xref="alg0.l6.m1.1.1"><eq id="alg0.l6.m1.1.1.1.cmml" xref="alg0.l6.m1.1.1.1"></eq><apply id="alg0.l6.m1.1.1.2.cmml" xref="alg0.l6.m1.1.1.2"><csymbol cd="ambiguous" id="alg0.l6.m1.1.1.2.1.cmml" xref="alg0.l6.m1.1.1.2">subscript</csymbol><ci id="alg0.l6.m1.1.1.2.2.cmml" xref="alg0.l6.m1.1.1.2.2">𝑔</ci><ci id="alg0.l6.m1.1.1.2.3.cmml" xref="alg0.l6.m1.1.1.2.3">𝑡</ci></apply><csymbol cd="latexml" id="alg0.l6.m1.1.1.3.cmml" xref="alg0.l6.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l6.m1.1c">g_{t}=</annotation></semantics></math> <span id="alg0.l6.1" class="ltx_text ltx_font_typewriter">Controller</span> (<math id="alg0.l6.m2.1" class="ltx_Math" alttext="m" display="inline"><semantics id="alg0.l6.m2.1a"><mi id="alg0.l6.m2.1.1" xref="alg0.l6.m2.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="alg0.l6.m2.1b"><ci id="alg0.l6.m2.1.1.cmml" xref="alg0.l6.m2.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l6.m2.1c">m</annotation></semantics></math>, <math id="alg0.l6.m3.1" class="ltx_Math" alttext="e" display="inline"><semantics id="alg0.l6.m3.1a"><mi id="alg0.l6.m3.1.1" xref="alg0.l6.m3.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="alg0.l6.m3.1b"><ci id="alg0.l6.m3.1.1.cmml" xref="alg0.l6.m3.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l6.m3.1c">e</annotation></semantics></math>, <math id="alg0.l6.m4.1" class="ltx_Math" alttext="\mathcal{T}_{&lt;t}" display="inline"><semantics id="alg0.l6.m4.1a"><msub id="alg0.l6.m4.1.1" xref="alg0.l6.m4.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l6.m4.1.1.2" xref="alg0.l6.m4.1.1.2.cmml">𝒯</mi><mrow id="alg0.l6.m4.1.1.3" xref="alg0.l6.m4.1.1.3.cmml"><mi id="alg0.l6.m4.1.1.3.2" xref="alg0.l6.m4.1.1.3.2.cmml"></mi><mo id="alg0.l6.m4.1.1.3.1" xref="alg0.l6.m4.1.1.3.1.cmml">&lt;</mo><mi id="alg0.l6.m4.1.1.3.3" xref="alg0.l6.m4.1.1.3.3.cmml">t</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="alg0.l6.m4.1b"><apply id="alg0.l6.m4.1.1.cmml" xref="alg0.l6.m4.1.1"><csymbol cd="ambiguous" id="alg0.l6.m4.1.1.1.cmml" xref="alg0.l6.m4.1.1">subscript</csymbol><ci id="alg0.l6.m4.1.1.2.cmml" xref="alg0.l6.m4.1.1.2">𝒯</ci><apply id="alg0.l6.m4.1.1.3.cmml" xref="alg0.l6.m4.1.1.3"><lt id="alg0.l6.m4.1.1.3.1.cmml" xref="alg0.l6.m4.1.1.3.1"></lt><csymbol cd="latexml" id="alg0.l6.m4.1.1.3.2.cmml" xref="alg0.l6.m4.1.1.3.2">absent</csymbol><ci id="alg0.l6.m4.1.1.3.3.cmml" xref="alg0.l6.m4.1.1.3.3">𝑡</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l6.m4.1c">\mathcal{T}_{&lt;t}</annotation></semantics></math>, <math id="alg0.l6.m5.1" class="ltx_Math" alttext="\mathcal{S}_{k-1}" display="inline"><semantics id="alg0.l6.m5.1a"><msub id="alg0.l6.m5.1.1" xref="alg0.l6.m5.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l6.m5.1.1.2" xref="alg0.l6.m5.1.1.2.cmml">𝒮</mi><mrow id="alg0.l6.m5.1.1.3" xref="alg0.l6.m5.1.1.3.cmml"><mi id="alg0.l6.m5.1.1.3.2" xref="alg0.l6.m5.1.1.3.2.cmml">k</mi><mo id="alg0.l6.m5.1.1.3.1" xref="alg0.l6.m5.1.1.3.1.cmml">−</mo><mn id="alg0.l6.m5.1.1.3.3" xref="alg0.l6.m5.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="alg0.l6.m5.1b"><apply id="alg0.l6.m5.1.1.cmml" xref="alg0.l6.m5.1.1"><csymbol cd="ambiguous" id="alg0.l6.m5.1.1.1.cmml" xref="alg0.l6.m5.1.1">subscript</csymbol><ci id="alg0.l6.m5.1.1.2.cmml" xref="alg0.l6.m5.1.1.2">𝒮</ci><apply id="alg0.l6.m5.1.1.3.cmml" xref="alg0.l6.m5.1.1.3"><minus id="alg0.l6.m5.1.1.3.1.cmml" xref="alg0.l6.m5.1.1.3.1"></minus><ci id="alg0.l6.m5.1.1.3.2.cmml" xref="alg0.l6.m5.1.1.3.2">𝑘</ci><cn type="integer" id="alg0.l6.m5.1.1.3.3.cmml" xref="alg0.l6.m5.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l6.m5.1c">\mathcal{S}_{k-1}</annotation></semantics></math>)

</div>
<div id="alg0.l7" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">7:</span>       <math id="alg0.l7.m1.1" class="ltx_Math" alttext="a_{t}=" display="inline"><semantics id="alg0.l7.m1.1a"><mrow id="alg0.l7.m1.1.1" xref="alg0.l7.m1.1.1.cmml"><msub id="alg0.l7.m1.1.1.2" xref="alg0.l7.m1.1.1.2.cmml"><mi id="alg0.l7.m1.1.1.2.2" xref="alg0.l7.m1.1.1.2.2.cmml">a</mi><mi id="alg0.l7.m1.1.1.2.3" xref="alg0.l7.m1.1.1.2.3.cmml">t</mi></msub><mo id="alg0.l7.m1.1.1.1" xref="alg0.l7.m1.1.1.1.cmml">=</mo><mi id="alg0.l7.m1.1.1.3" xref="alg0.l7.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l7.m1.1b"><apply id="alg0.l7.m1.1.1.cmml" xref="alg0.l7.m1.1.1"><eq id="alg0.l7.m1.1.1.1.cmml" xref="alg0.l7.m1.1.1.1"></eq><apply id="alg0.l7.m1.1.1.2.cmml" xref="alg0.l7.m1.1.1.2"><csymbol cd="ambiguous" id="alg0.l7.m1.1.1.2.1.cmml" xref="alg0.l7.m1.1.1.2">subscript</csymbol><ci id="alg0.l7.m1.1.1.2.2.cmml" xref="alg0.l7.m1.1.1.2.2">𝑎</ci><ci id="alg0.l7.m1.1.1.2.3.cmml" xref="alg0.l7.m1.1.1.2.3">𝑡</ci></apply><csymbol cd="latexml" id="alg0.l7.m1.1.1.3.cmml" xref="alg0.l7.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l7.m1.1c">a_{t}=</annotation></semantics></math> <span id="alg0.l7.1" class="ltx_text ltx_font_typewriter">Executor</span> (<math id="alg0.l7.m2.1" class="ltx_Math" alttext="g_{t}" display="inline"><semantics id="alg0.l7.m2.1a"><msub id="alg0.l7.m2.1.1" xref="alg0.l7.m2.1.1.cmml"><mi id="alg0.l7.m2.1.1.2" xref="alg0.l7.m2.1.1.2.cmml">g</mi><mi id="alg0.l7.m2.1.1.3" xref="alg0.l7.m2.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l7.m2.1b"><apply id="alg0.l7.m2.1.1.cmml" xref="alg0.l7.m2.1.1"><csymbol cd="ambiguous" id="alg0.l7.m2.1.1.1.cmml" xref="alg0.l7.m2.1.1">subscript</csymbol><ci id="alg0.l7.m2.1.1.2.cmml" xref="alg0.l7.m2.1.1.2">𝑔</ci><ci id="alg0.l7.m2.1.1.3.cmml" xref="alg0.l7.m2.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l7.m2.1c">g_{t}</annotation></semantics></math>, admissible actions)

</div>
<div id="alg0.l8" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">8:</span>       <math id="alg0.l8.m1.2" class="ltx_Math" alttext="r_{t},o_{t}=" display="inline"><semantics id="alg0.l8.m1.2a"><mrow id="alg0.l8.m1.2.2" xref="alg0.l8.m1.2.2.cmml"><mrow id="alg0.l8.m1.2.2.2.2" xref="alg0.l8.m1.2.2.2.3.cmml"><msub id="alg0.l8.m1.1.1.1.1.1" xref="alg0.l8.m1.1.1.1.1.1.cmml"><mi id="alg0.l8.m1.1.1.1.1.1.2" xref="alg0.l8.m1.1.1.1.1.1.2.cmml">r</mi><mi id="alg0.l8.m1.1.1.1.1.1.3" xref="alg0.l8.m1.1.1.1.1.1.3.cmml">t</mi></msub><mo id="alg0.l8.m1.2.2.2.2.3" xref="alg0.l8.m1.2.2.2.3.cmml">,</mo><msub id="alg0.l8.m1.2.2.2.2.2" xref="alg0.l8.m1.2.2.2.2.2.cmml"><mi id="alg0.l8.m1.2.2.2.2.2.2" xref="alg0.l8.m1.2.2.2.2.2.2.cmml">o</mi><mi id="alg0.l8.m1.2.2.2.2.2.3" xref="alg0.l8.m1.2.2.2.2.2.3.cmml">t</mi></msub></mrow><mo id="alg0.l8.m1.2.2.3" xref="alg0.l8.m1.2.2.3.cmml">=</mo><mi id="alg0.l8.m1.2.2.4" xref="alg0.l8.m1.2.2.4.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l8.m1.2b"><apply id="alg0.l8.m1.2.2.cmml" xref="alg0.l8.m1.2.2"><eq id="alg0.l8.m1.2.2.3.cmml" xref="alg0.l8.m1.2.2.3"></eq><list id="alg0.l8.m1.2.2.2.3.cmml" xref="alg0.l8.m1.2.2.2.2"><apply id="alg0.l8.m1.1.1.1.1.1.cmml" xref="alg0.l8.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l8.m1.1.1.1.1.1.1.cmml" xref="alg0.l8.m1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l8.m1.1.1.1.1.1.2.cmml" xref="alg0.l8.m1.1.1.1.1.1.2">𝑟</ci><ci id="alg0.l8.m1.1.1.1.1.1.3.cmml" xref="alg0.l8.m1.1.1.1.1.1.3">𝑡</ci></apply><apply id="alg0.l8.m1.2.2.2.2.2.cmml" xref="alg0.l8.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l8.m1.2.2.2.2.2.1.cmml" xref="alg0.l8.m1.2.2.2.2.2">subscript</csymbol><ci id="alg0.l8.m1.2.2.2.2.2.2.cmml" xref="alg0.l8.m1.2.2.2.2.2.2">𝑜</ci><ci id="alg0.l8.m1.2.2.2.2.2.3.cmml" xref="alg0.l8.m1.2.2.2.2.2.3">𝑡</ci></apply></list><csymbol cd="latexml" id="alg0.l8.m1.2.2.4.cmml" xref="alg0.l8.m1.2.2.4">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l8.m1.2c">r_{t},o_{t}=</annotation></semantics></math> <span id="alg0.l8.1" class="ltx_text ltx_font_typewriter">Simulator</span> (<math id="alg0.l8.m2.1" class="ltx_Math" alttext="\mathcal{T}_{&lt;t}" display="inline"><semantics id="alg0.l8.m2.1a"><msub id="alg0.l8.m2.1.1" xref="alg0.l8.m2.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l8.m2.1.1.2" xref="alg0.l8.m2.1.1.2.cmml">𝒯</mi><mrow id="alg0.l8.m2.1.1.3" xref="alg0.l8.m2.1.1.3.cmml"><mi id="alg0.l8.m2.1.1.3.2" xref="alg0.l8.m2.1.1.3.2.cmml"></mi><mo id="alg0.l8.m2.1.1.3.1" xref="alg0.l8.m2.1.1.3.1.cmml">&lt;</mo><mi id="alg0.l8.m2.1.1.3.3" xref="alg0.l8.m2.1.1.3.3.cmml">t</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="alg0.l8.m2.1b"><apply id="alg0.l8.m2.1.1.cmml" xref="alg0.l8.m2.1.1"><csymbol cd="ambiguous" id="alg0.l8.m2.1.1.1.cmml" xref="alg0.l8.m2.1.1">subscript</csymbol><ci id="alg0.l8.m2.1.1.2.cmml" xref="alg0.l8.m2.1.1.2">𝒯</ci><apply id="alg0.l8.m2.1.1.3.cmml" xref="alg0.l8.m2.1.1.3"><lt id="alg0.l8.m2.1.1.3.1.cmml" xref="alg0.l8.m2.1.1.3.1"></lt><csymbol cd="latexml" id="alg0.l8.m2.1.1.3.2.cmml" xref="alg0.l8.m2.1.1.3.2">absent</csymbol><ci id="alg0.l8.m2.1.1.3.3.cmml" xref="alg0.l8.m2.1.1.3.3">𝑡</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l8.m2.1c">\mathcal{T}_{&lt;t}</annotation></semantics></math>, <math id="alg0.l8.m3.1" class="ltx_Math" alttext="a_{t}" display="inline"><semantics id="alg0.l8.m3.1a"><msub id="alg0.l8.m3.1.1" xref="alg0.l8.m3.1.1.cmml"><mi id="alg0.l8.m3.1.1.2" xref="alg0.l8.m3.1.1.2.cmml">a</mi><mi id="alg0.l8.m3.1.1.3" xref="alg0.l8.m3.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l8.m3.1b"><apply id="alg0.l8.m3.1.1.cmml" xref="alg0.l8.m3.1.1"><csymbol cd="ambiguous" id="alg0.l8.m3.1.1.1.cmml" xref="alg0.l8.m3.1.1">subscript</csymbol><ci id="alg0.l8.m3.1.1.2.cmml" xref="alg0.l8.m3.1.1.2">𝑎</ci><ci id="alg0.l8.m3.1.1.3.cmml" xref="alg0.l8.m3.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l8.m3.1c">a_{t}</annotation></semantics></math>)

</div>
<div id="alg0.l9" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">9:</span>       <math id="alg0.l9.m1.1" class="ltx_Math" alttext="\mathcal{T}_{&lt;t+1}=" display="inline"><semantics id="alg0.l9.m1.1a"><mrow id="alg0.l9.m1.1.1" xref="alg0.l9.m1.1.1.cmml"><msub id="alg0.l9.m1.1.1.2" xref="alg0.l9.m1.1.1.2.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l9.m1.1.1.2.2" xref="alg0.l9.m1.1.1.2.2.cmml">𝒯</mi><mrow id="alg0.l9.m1.1.1.2.3" xref="alg0.l9.m1.1.1.2.3.cmml"><mi id="alg0.l9.m1.1.1.2.3.2" xref="alg0.l9.m1.1.1.2.3.2.cmml"></mi><mo id="alg0.l9.m1.1.1.2.3.1" xref="alg0.l9.m1.1.1.2.3.1.cmml">&lt;</mo><mrow id="alg0.l9.m1.1.1.2.3.3" xref="alg0.l9.m1.1.1.2.3.3.cmml"><mi id="alg0.l9.m1.1.1.2.3.3.2" xref="alg0.l9.m1.1.1.2.3.3.2.cmml">t</mi><mo id="alg0.l9.m1.1.1.2.3.3.1" xref="alg0.l9.m1.1.1.2.3.3.1.cmml">+</mo><mn id="alg0.l9.m1.1.1.2.3.3.3" xref="alg0.l9.m1.1.1.2.3.3.3.cmml">1</mn></mrow></mrow></msub><mo id="alg0.l9.m1.1.1.1" xref="alg0.l9.m1.1.1.1.cmml">=</mo><mi id="alg0.l9.m1.1.1.3" xref="alg0.l9.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l9.m1.1b"><apply id="alg0.l9.m1.1.1.cmml" xref="alg0.l9.m1.1.1"><eq id="alg0.l9.m1.1.1.1.cmml" xref="alg0.l9.m1.1.1.1"></eq><apply id="alg0.l9.m1.1.1.2.cmml" xref="alg0.l9.m1.1.1.2"><csymbol cd="ambiguous" id="alg0.l9.m1.1.1.2.1.cmml" xref="alg0.l9.m1.1.1.2">subscript</csymbol><ci id="alg0.l9.m1.1.1.2.2.cmml" xref="alg0.l9.m1.1.1.2.2">𝒯</ci><apply id="alg0.l9.m1.1.1.2.3.cmml" xref="alg0.l9.m1.1.1.2.3"><lt id="alg0.l9.m1.1.1.2.3.1.cmml" xref="alg0.l9.m1.1.1.2.3.1"></lt><csymbol cd="latexml" id="alg0.l9.m1.1.1.2.3.2.cmml" xref="alg0.l9.m1.1.1.2.3.2">absent</csymbol><apply id="alg0.l9.m1.1.1.2.3.3.cmml" xref="alg0.l9.m1.1.1.2.3.3"><plus id="alg0.l9.m1.1.1.2.3.3.1.cmml" xref="alg0.l9.m1.1.1.2.3.3.1"></plus><ci id="alg0.l9.m1.1.1.2.3.3.2.cmml" xref="alg0.l9.m1.1.1.2.3.3.2">𝑡</ci><cn type="integer" id="alg0.l9.m1.1.1.2.3.3.3.cmml" xref="alg0.l9.m1.1.1.2.3.3.3">1</cn></apply></apply></apply><csymbol cd="latexml" id="alg0.l9.m1.1.1.3.cmml" xref="alg0.l9.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m1.1c">\mathcal{T}_{&lt;t+1}=</annotation></semantics></math> <math id="alg0.l9.m2.1" class="ltx_Math" alttext="\mathcal{T}_{&lt;t}" display="inline"><semantics id="alg0.l9.m2.1a"><msub id="alg0.l9.m2.1.1" xref="alg0.l9.m2.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l9.m2.1.1.2" xref="alg0.l9.m2.1.1.2.cmml">𝒯</mi><mrow id="alg0.l9.m2.1.1.3" xref="alg0.l9.m2.1.1.3.cmml"><mi id="alg0.l9.m2.1.1.3.2" xref="alg0.l9.m2.1.1.3.2.cmml"></mi><mo id="alg0.l9.m2.1.1.3.1" xref="alg0.l9.m2.1.1.3.1.cmml">&lt;</mo><mi id="alg0.l9.m2.1.1.3.3" xref="alg0.l9.m2.1.1.3.3.cmml">t</mi></mrow></msub><annotation-xml encoding="MathML-Content" id="alg0.l9.m2.1b"><apply id="alg0.l9.m2.1.1.cmml" xref="alg0.l9.m2.1.1"><csymbol cd="ambiguous" id="alg0.l9.m2.1.1.1.cmml" xref="alg0.l9.m2.1.1">subscript</csymbol><ci id="alg0.l9.m2.1.1.2.cmml" xref="alg0.l9.m2.1.1.2">𝒯</ci><apply id="alg0.l9.m2.1.1.3.cmml" xref="alg0.l9.m2.1.1.3"><lt id="alg0.l9.m2.1.1.3.1.cmml" xref="alg0.l9.m2.1.1.3.1"></lt><csymbol cd="latexml" id="alg0.l9.m2.1.1.3.2.cmml" xref="alg0.l9.m2.1.1.3.2">absent</csymbol><ci id="alg0.l9.m2.1.1.3.3.cmml" xref="alg0.l9.m2.1.1.3.3">𝑡</ci></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m2.1c">\mathcal{T}_{&lt;t}</annotation></semantics></math> + (<math id="alg0.l9.m3.1" class="ltx_Math" alttext="g_{t}" display="inline"><semantics id="alg0.l9.m3.1a"><msub id="alg0.l9.m3.1.1" xref="alg0.l9.m3.1.1.cmml"><mi id="alg0.l9.m3.1.1.2" xref="alg0.l9.m3.1.1.2.cmml">g</mi><mi id="alg0.l9.m3.1.1.3" xref="alg0.l9.m3.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l9.m3.1b"><apply id="alg0.l9.m3.1.1.cmml" xref="alg0.l9.m3.1.1"><csymbol cd="ambiguous" id="alg0.l9.m3.1.1.1.cmml" xref="alg0.l9.m3.1.1">subscript</csymbol><ci id="alg0.l9.m3.1.1.2.cmml" xref="alg0.l9.m3.1.1.2">𝑔</ci><ci id="alg0.l9.m3.1.1.3.cmml" xref="alg0.l9.m3.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m3.1c">g_{t}</annotation></semantics></math>, <math id="alg0.l9.m4.1" class="ltx_Math" alttext="a_{t}" display="inline"><semantics id="alg0.l9.m4.1a"><msub id="alg0.l9.m4.1.1" xref="alg0.l9.m4.1.1.cmml"><mi id="alg0.l9.m4.1.1.2" xref="alg0.l9.m4.1.1.2.cmml">a</mi><mi id="alg0.l9.m4.1.1.3" xref="alg0.l9.m4.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l9.m4.1b"><apply id="alg0.l9.m4.1.1.cmml" xref="alg0.l9.m4.1.1"><csymbol cd="ambiguous" id="alg0.l9.m4.1.1.1.cmml" xref="alg0.l9.m4.1.1">subscript</csymbol><ci id="alg0.l9.m4.1.1.2.cmml" xref="alg0.l9.m4.1.1.2">𝑎</ci><ci id="alg0.l9.m4.1.1.3.cmml" xref="alg0.l9.m4.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m4.1c">a_{t}</annotation></semantics></math>, <math id="alg0.l9.m5.1" class="ltx_Math" alttext="o_{t}" display="inline"><semantics id="alg0.l9.m5.1a"><msub id="alg0.l9.m5.1.1" xref="alg0.l9.m5.1.1.cmml"><mi id="alg0.l9.m5.1.1.2" xref="alg0.l9.m5.1.1.2.cmml">o</mi><mi id="alg0.l9.m5.1.1.3" xref="alg0.l9.m5.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l9.m5.1b"><apply id="alg0.l9.m5.1.1.cmml" xref="alg0.l9.m5.1.1"><csymbol cd="ambiguous" id="alg0.l9.m5.1.1.1.cmml" xref="alg0.l9.m5.1.1">subscript</csymbol><ci id="alg0.l9.m5.1.1.2.cmml" xref="alg0.l9.m5.1.1.2">𝑜</ci><ci id="alg0.l9.m5.1.1.3.cmml" xref="alg0.l9.m5.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m5.1c">o_{t}</annotation></semantics></math>, <math id="alg0.l9.m6.1" class="ltx_Math" alttext="r_{t}" display="inline"><semantics id="alg0.l9.m6.1a"><msub id="alg0.l9.m6.1.1" xref="alg0.l9.m6.1.1.cmml"><mi id="alg0.l9.m6.1.1.2" xref="alg0.l9.m6.1.1.2.cmml">r</mi><mi id="alg0.l9.m6.1.1.3" xref="alg0.l9.m6.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l9.m6.1b"><apply id="alg0.l9.m6.1.1.cmml" xref="alg0.l9.m6.1.1"><csymbol cd="ambiguous" id="alg0.l9.m6.1.1.1.cmml" xref="alg0.l9.m6.1.1">subscript</csymbol><ci id="alg0.l9.m6.1.1.2.cmml" xref="alg0.l9.m6.1.1.2">𝑟</ci><ci id="alg0.l9.m6.1.1.3.cmml" xref="alg0.l9.m6.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l9.m6.1c">r_{t}</annotation></semantics></math>)

</div>
<div id="alg0.l10" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">10:</span>     Final reward <math id="alg0.l10.m1.1" class="ltx_Math" alttext="r_{\text{k}}" display="inline"><semantics id="alg0.l10.m1.1a"><msub id="alg0.l10.m1.1.1" xref="alg0.l10.m1.1.1.cmml"><mi id="alg0.l10.m1.1.1.2" xref="alg0.l10.m1.1.1.2.cmml">r</mi><mtext id="alg0.l10.m1.1.1.3" xref="alg0.l10.m1.1.1.3a.cmml">k</mtext></msub><annotation-xml encoding="MathML-Content" id="alg0.l10.m1.1b"><apply id="alg0.l10.m1.1.1.cmml" xref="alg0.l10.m1.1.1"><csymbol cd="ambiguous" id="alg0.l10.m1.1.1.1.cmml" xref="alg0.l10.m1.1.1">subscript</csymbol><ci id="alg0.l10.m1.1.1.2.cmml" xref="alg0.l10.m1.1.1.2">𝑟</ci><ci id="alg0.l10.m1.1.1.3a.cmml" xref="alg0.l10.m1.1.1.3"><mtext mathsize="70%" id="alg0.l10.m1.1.1.3.cmml" xref="alg0.l10.m1.1.1.3">k</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l10.m1.1c">r_{\text{k}}</annotation></semantics></math> = <math id="alg0.l10.m2.1" class="ltx_Math" alttext="r_{t}" display="inline"><semantics id="alg0.l10.m2.1a"><msub id="alg0.l10.m2.1.1" xref="alg0.l10.m2.1.1.cmml"><mi id="alg0.l10.m2.1.1.2" xref="alg0.l10.m2.1.1.2.cmml">r</mi><mi id="alg0.l10.m2.1.1.3" xref="alg0.l10.m2.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l10.m2.1b"><apply id="alg0.l10.m2.1.1.cmml" xref="alg0.l10.m2.1.1"><csymbol cd="ambiguous" id="alg0.l10.m2.1.1.1.cmml" xref="alg0.l10.m2.1.1">subscript</csymbol><ci id="alg0.l10.m2.1.1.2.cmml" xref="alg0.l10.m2.1.1.2">𝑟</ci><ci id="alg0.l10.m2.1.1.3.cmml" xref="alg0.l10.m2.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l10.m2.1c">r_{t}</annotation></semantics></math>

</div>
<div id="alg0.l11" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">11:</span>     <math id="alg0.l11.m1.1" class="ltx_Math" alttext="\mathcal{S}_{k}=" display="inline"><semantics id="alg0.l11.m1.1a"><mrow id="alg0.l11.m1.1.1" xref="alg0.l11.m1.1.1.cmml"><msub id="alg0.l11.m1.1.1.2" xref="alg0.l11.m1.1.1.2.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l11.m1.1.1.2.2" xref="alg0.l11.m1.1.1.2.2.cmml">𝒮</mi><mi id="alg0.l11.m1.1.1.2.3" xref="alg0.l11.m1.1.1.2.3.cmml">k</mi></msub><mo id="alg0.l11.m1.1.1.1" xref="alg0.l11.m1.1.1.1.cmml">=</mo><mi id="alg0.l11.m1.1.1.3" xref="alg0.l11.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l11.m1.1b"><apply id="alg0.l11.m1.1.1.cmml" xref="alg0.l11.m1.1.1"><eq id="alg0.l11.m1.1.1.1.cmml" xref="alg0.l11.m1.1.1.1"></eq><apply id="alg0.l11.m1.1.1.2.cmml" xref="alg0.l11.m1.1.1.2"><csymbol cd="ambiguous" id="alg0.l11.m1.1.1.2.1.cmml" xref="alg0.l11.m1.1.1.2">subscript</csymbol><ci id="alg0.l11.m1.1.1.2.2.cmml" xref="alg0.l11.m1.1.1.2.2">𝒮</ci><ci id="alg0.l11.m1.1.1.2.3.cmml" xref="alg0.l11.m1.1.1.2.3">𝑘</ci></apply><csymbol cd="latexml" id="alg0.l11.m1.1.1.3.cmml" xref="alg0.l11.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l11.m1.1c">\mathcal{S}_{k}=</annotation></semantics></math> <span id="alg0.l11.1" class="ltx_text ltx_font_typewriter">memory-generator</span> (<math id="alg0.l11.m2.1" class="ltx_Math" alttext="\{\mathcal{S}_{&lt;k}\}" display="inline"><semantics id="alg0.l11.m2.1a"><mrow id="alg0.l11.m2.1.1.1" xref="alg0.l11.m2.1.1.2.cmml"><mo stretchy="false" id="alg0.l11.m2.1.1.1.2" xref="alg0.l11.m2.1.1.2.cmml">{</mo><msub id="alg0.l11.m2.1.1.1.1" xref="alg0.l11.m2.1.1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l11.m2.1.1.1.1.2" xref="alg0.l11.m2.1.1.1.1.2.cmml">𝒮</mi><mrow id="alg0.l11.m2.1.1.1.1.3" xref="alg0.l11.m2.1.1.1.1.3.cmml"><mi id="alg0.l11.m2.1.1.1.1.3.2" xref="alg0.l11.m2.1.1.1.1.3.2.cmml"></mi><mo id="alg0.l11.m2.1.1.1.1.3.1" xref="alg0.l11.m2.1.1.1.1.3.1.cmml">&lt;</mo><mi id="alg0.l11.m2.1.1.1.1.3.3" xref="alg0.l11.m2.1.1.1.1.3.3.cmml">k</mi></mrow></msub><mo stretchy="false" id="alg0.l11.m2.1.1.1.3" xref="alg0.l11.m2.1.1.2.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="alg0.l11.m2.1b"><set id="alg0.l11.m2.1.1.2.cmml" xref="alg0.l11.m2.1.1.1"><apply id="alg0.l11.m2.1.1.1.1.cmml" xref="alg0.l11.m2.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l11.m2.1.1.1.1.1.cmml" xref="alg0.l11.m2.1.1.1.1">subscript</csymbol><ci id="alg0.l11.m2.1.1.1.1.2.cmml" xref="alg0.l11.m2.1.1.1.1.2">𝒮</ci><apply id="alg0.l11.m2.1.1.1.1.3.cmml" xref="alg0.l11.m2.1.1.1.1.3"><lt id="alg0.l11.m2.1.1.1.1.3.1.cmml" xref="alg0.l11.m2.1.1.1.1.3.1"></lt><csymbol cd="latexml" id="alg0.l11.m2.1.1.1.1.3.2.cmml" xref="alg0.l11.m2.1.1.1.1.3.2">absent</csymbol><ci id="alg0.l11.m2.1.1.1.1.3.3.cmml" xref="alg0.l11.m2.1.1.1.1.3.3">𝑘</ci></apply></apply></set></annotation-xml><annotation encoding="application/x-tex" id="alg0.l11.m2.1c">\{\mathcal{S}_{&lt;k}\}</annotation></semantics></math>, <math id="alg0.l11.m3.1" class="ltx_Math" alttext="\mathcal{T}_{k}" display="inline"><semantics id="alg0.l11.m3.1a"><msub id="alg0.l11.m3.1.1" xref="alg0.l11.m3.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l11.m3.1.1.2" xref="alg0.l11.m3.1.1.2.cmml">𝒯</mi><mi id="alg0.l11.m3.1.1.3" xref="alg0.l11.m3.1.1.3.cmml">k</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l11.m3.1b"><apply id="alg0.l11.m3.1.1.cmml" xref="alg0.l11.m3.1.1"><csymbol cd="ambiguous" id="alg0.l11.m3.1.1.1.cmml" xref="alg0.l11.m3.1.1">subscript</csymbol><ci id="alg0.l11.m3.1.1.2.cmml" xref="alg0.l11.m3.1.1.2">𝒯</ci><ci id="alg0.l11.m3.1.1.3.cmml" xref="alg0.l11.m3.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l11.m3.1c">\mathcal{T}_{k}</annotation></semantics></math>, <math id="alg0.l11.m4.1" class="ltx_Math" alttext="r_{k}" display="inline"><semantics id="alg0.l11.m4.1a"><msub id="alg0.l11.m4.1.1" xref="alg0.l11.m4.1.1.cmml"><mi id="alg0.l11.m4.1.1.2" xref="alg0.l11.m4.1.1.2.cmml">r</mi><mi id="alg0.l11.m4.1.1.3" xref="alg0.l11.m4.1.1.3.cmml">k</mi></msub><annotation-xml encoding="MathML-Content" id="alg0.l11.m4.1b"><apply id="alg0.l11.m4.1.1.cmml" xref="alg0.l11.m4.1.1"><csymbol cd="ambiguous" id="alg0.l11.m4.1.1.1.cmml" xref="alg0.l11.m4.1.1">subscript</csymbol><ci id="alg0.l11.m4.1.1.2.cmml" xref="alg0.l11.m4.1.1.2">𝑟</ci><ci id="alg0.l11.m4.1.1.3.cmml" xref="alg0.l11.m4.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l11.m4.1c">r_{k}</annotation></semantics></math>)

</div>
<div id="alg0.l12" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">12:</span>
</div>
<div id="alg0.l13" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">13:</span><span id="alg0.l13.1" class="ltx_text ltx_font_bold">procedure</span> <span id="alg0.l13.2" class="ltx_text ltx_font_smallcaps">Generalization</span>(Task: <math id="alg0.l13.m1.1" class="ltx_Math" alttext="m" display="inline"><semantics id="alg0.l13.m1.1a"><mi id="alg0.l13.m1.1.1" xref="alg0.l13.m1.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="alg0.l13.m1.1b"><ci id="alg0.l13.m1.1.1.cmml" xref="alg0.l13.m1.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l13.m1.1c">m</annotation></semantics></math>, Env: <math id="alg0.l13.m2.1" class="ltx_Math" alttext="e" display="inline"><semantics id="alg0.l13.m2.1a"><mi id="alg0.l13.m2.1.1" xref="alg0.l13.m2.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="alg0.l13.m2.1b"><ci id="alg0.l13.m2.1.1.cmml" xref="alg0.l13.m2.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l13.m2.1c">e</annotation></semantics></math>, past <math id="alg0.l13.m3.1" class="ltx_Math" alttext="m^{\prime}" display="inline"><semantics id="alg0.l13.m3.1a"><msup id="alg0.l13.m3.1.1" xref="alg0.l13.m3.1.1.cmml"><mi id="alg0.l13.m3.1.1.2" xref="alg0.l13.m3.1.1.2.cmml">m</mi><mo id="alg0.l13.m3.1.1.3" xref="alg0.l13.m3.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="alg0.l13.m3.1b"><apply id="alg0.l13.m3.1.1.cmml" xref="alg0.l13.m3.1.1"><csymbol cd="ambiguous" id="alg0.l13.m3.1.1.1.cmml" xref="alg0.l13.m3.1.1">superscript</csymbol><ci id="alg0.l13.m3.1.1.2.cmml" xref="alg0.l13.m3.1.1.2">𝑚</ci><ci id="alg0.l13.m3.1.1.3.cmml" xref="alg0.l13.m3.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l13.m3.1c">m^{\prime}</annotation></semantics></math>/<math id="alg0.l13.m4.1" class="ltx_Math" alttext="e^{\prime}" display="inline"><semantics id="alg0.l13.m4.1a"><msup id="alg0.l13.m4.1.1" xref="alg0.l13.m4.1.1.cmml"><mi id="alg0.l13.m4.1.1.2" xref="alg0.l13.m4.1.1.2.cmml">e</mi><mo id="alg0.l13.m4.1.1.3" xref="alg0.l13.m4.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="alg0.l13.m4.1b"><apply id="alg0.l13.m4.1.1.cmml" xref="alg0.l13.m4.1.1"><csymbol cd="ambiguous" id="alg0.l13.m4.1.1.1.cmml" xref="alg0.l13.m4.1.1">superscript</csymbol><ci id="alg0.l13.m4.1.1.2.cmml" xref="alg0.l13.m4.1.1.2">𝑒</ci><ci id="alg0.l13.m4.1.1.3.cmml" xref="alg0.l13.m4.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l13.m4.1c">e^{\prime}</annotation></semantics></math>)

</div>
<div id="alg0.l14" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">14:</span>   <math id="alg0.l14.m1.2" class="ltx_Math" alttext="\{\mathcal{S}_{\text{crucial}},r_{k}\}=" display="inline"><semantics id="alg0.l14.m1.2a"><mrow id="alg0.l14.m1.2.2" xref="alg0.l14.m1.2.2.cmml"><mrow id="alg0.l14.m1.2.2.2.2" xref="alg0.l14.m1.2.2.2.3.cmml"><mo stretchy="false" id="alg0.l14.m1.2.2.2.2.3" xref="alg0.l14.m1.2.2.2.3.cmml">{</mo><msub id="alg0.l14.m1.1.1.1.1.1" xref="alg0.l14.m1.1.1.1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l14.m1.1.1.1.1.1.2" xref="alg0.l14.m1.1.1.1.1.1.2.cmml">𝒮</mi><mtext id="alg0.l14.m1.1.1.1.1.1.3" xref="alg0.l14.m1.1.1.1.1.1.3a.cmml">crucial</mtext></msub><mo id="alg0.l14.m1.2.2.2.2.4" xref="alg0.l14.m1.2.2.2.3.cmml">,</mo><msub id="alg0.l14.m1.2.2.2.2.2" xref="alg0.l14.m1.2.2.2.2.2.cmml"><mi id="alg0.l14.m1.2.2.2.2.2.2" xref="alg0.l14.m1.2.2.2.2.2.2.cmml">r</mi><mi id="alg0.l14.m1.2.2.2.2.2.3" xref="alg0.l14.m1.2.2.2.2.2.3.cmml">k</mi></msub><mo stretchy="false" id="alg0.l14.m1.2.2.2.2.5" xref="alg0.l14.m1.2.2.2.3.cmml">}</mo></mrow><mo id="alg0.l14.m1.2.2.3" xref="alg0.l14.m1.2.2.3.cmml">=</mo><mi id="alg0.l14.m1.2.2.4" xref="alg0.l14.m1.2.2.4.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l14.m1.2b"><apply id="alg0.l14.m1.2.2.cmml" xref="alg0.l14.m1.2.2"><eq id="alg0.l14.m1.2.2.3.cmml" xref="alg0.l14.m1.2.2.3"></eq><set id="alg0.l14.m1.2.2.2.3.cmml" xref="alg0.l14.m1.2.2.2.2"><apply id="alg0.l14.m1.1.1.1.1.1.cmml" xref="alg0.l14.m1.1.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l14.m1.1.1.1.1.1.1.cmml" xref="alg0.l14.m1.1.1.1.1.1">subscript</csymbol><ci id="alg0.l14.m1.1.1.1.1.1.2.cmml" xref="alg0.l14.m1.1.1.1.1.1.2">𝒮</ci><ci id="alg0.l14.m1.1.1.1.1.1.3a.cmml" xref="alg0.l14.m1.1.1.1.1.1.3"><mtext mathsize="70%" id="alg0.l14.m1.1.1.1.1.1.3.cmml" xref="alg0.l14.m1.1.1.1.1.1.3">crucial</mtext></ci></apply><apply id="alg0.l14.m1.2.2.2.2.2.cmml" xref="alg0.l14.m1.2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l14.m1.2.2.2.2.2.1.cmml" xref="alg0.l14.m1.2.2.2.2.2">subscript</csymbol><ci id="alg0.l14.m1.2.2.2.2.2.2.cmml" xref="alg0.l14.m1.2.2.2.2.2.2">𝑟</ci><ci id="alg0.l14.m1.2.2.2.2.2.3.cmml" xref="alg0.l14.m1.2.2.2.2.2.3">𝑘</ci></apply></set><csymbol cd="latexml" id="alg0.l14.m1.2.2.4.cmml" xref="alg0.l14.m1.2.2.4">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l14.m1.2c">\{\mathcal{S}_{\text{crucial}},r_{k}\}=</annotation></semantics></math> <span id="alg0.l14.1" class="ltx_text ltx_font_typewriter">crucial-memories</span> (past <math id="alg0.l14.m2.1" class="ltx_Math" alttext="m^{\prime}" display="inline"><semantics id="alg0.l14.m2.1a"><msup id="alg0.l14.m2.1.1" xref="alg0.l14.m2.1.1.cmml"><mi id="alg0.l14.m2.1.1.2" xref="alg0.l14.m2.1.1.2.cmml">m</mi><mo id="alg0.l14.m2.1.1.3" xref="alg0.l14.m2.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="alg0.l14.m2.1b"><apply id="alg0.l14.m2.1.1.cmml" xref="alg0.l14.m2.1.1"><csymbol cd="ambiguous" id="alg0.l14.m2.1.1.1.cmml" xref="alg0.l14.m2.1.1">superscript</csymbol><ci id="alg0.l14.m2.1.1.2.cmml" xref="alg0.l14.m2.1.1.2">𝑚</ci><ci id="alg0.l14.m2.1.1.3.cmml" xref="alg0.l14.m2.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l14.m2.1c">m^{\prime}</annotation></semantics></math>/<math id="alg0.l14.m3.1" class="ltx_Math" alttext="e^{\prime}" display="inline"><semantics id="alg0.l14.m3.1a"><msup id="alg0.l14.m3.1.1" xref="alg0.l14.m3.1.1.cmml"><mi id="alg0.l14.m3.1.1.2" xref="alg0.l14.m3.1.1.2.cmml">e</mi><mo id="alg0.l14.m3.1.1.3" xref="alg0.l14.m3.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="alg0.l14.m3.1b"><apply id="alg0.l14.m3.1.1.cmml" xref="alg0.l14.m3.1.1"><csymbol cd="ambiguous" id="alg0.l14.m3.1.1.1.cmml" xref="alg0.l14.m3.1.1">superscript</csymbol><ci id="alg0.l14.m3.1.1.2.cmml" xref="alg0.l14.m3.1.1.2">𝑒</ci><ci id="alg0.l14.m3.1.1.3.cmml" xref="alg0.l14.m3.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l14.m3.1c">e^{\prime}</annotation></semantics></math>)

</div>
<div id="alg0.l15" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">15:</span>   <math id="alg0.l15.m1.1" class="ltx_Math" alttext="\mathcal{S}_{\text{meta}}=" display="inline"><semantics id="alg0.l15.m1.1a"><mrow id="alg0.l15.m1.1.1" xref="alg0.l15.m1.1.1.cmml"><msub id="alg0.l15.m1.1.1.2" xref="alg0.l15.m1.1.1.2.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l15.m1.1.1.2.2" xref="alg0.l15.m1.1.1.2.2.cmml">𝒮</mi><mtext id="alg0.l15.m1.1.1.2.3" xref="alg0.l15.m1.1.1.2.3a.cmml">meta</mtext></msub><mo id="alg0.l15.m1.1.1.1" xref="alg0.l15.m1.1.1.1.cmml">=</mo><mi id="alg0.l15.m1.1.1.3" xref="alg0.l15.m1.1.1.3.cmml"></mi></mrow><annotation-xml encoding="MathML-Content" id="alg0.l15.m1.1b"><apply id="alg0.l15.m1.1.1.cmml" xref="alg0.l15.m1.1.1"><eq id="alg0.l15.m1.1.1.1.cmml" xref="alg0.l15.m1.1.1.1"></eq><apply id="alg0.l15.m1.1.1.2.cmml" xref="alg0.l15.m1.1.1.2"><csymbol cd="ambiguous" id="alg0.l15.m1.1.1.2.1.cmml" xref="alg0.l15.m1.1.1.2">subscript</csymbol><ci id="alg0.l15.m1.1.1.2.2.cmml" xref="alg0.l15.m1.1.1.2.2">𝒮</ci><ci id="alg0.l15.m1.1.1.2.3a.cmml" xref="alg0.l15.m1.1.1.2.3"><mtext mathsize="70%" id="alg0.l15.m1.1.1.2.3.cmml" xref="alg0.l15.m1.1.1.2.3">meta</mtext></ci></apply><csymbol cd="latexml" id="alg0.l15.m1.1.1.3.cmml" xref="alg0.l15.m1.1.1.3">absent</csymbol></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l15.m1.1c">\mathcal{S}_{\text{meta}}=</annotation></semantics></math> <span id="alg0.l15.1" class="ltx_text ltx_font_typewriter">meta-memory</span> (<math id="alg0.l15.m2.2" class="ltx_Math" alttext="\{\mathcal{S}_{\text{crucial}},r_{k}\}" display="inline"><semantics id="alg0.l15.m2.2a"><mrow id="alg0.l15.m2.2.2.2" xref="alg0.l15.m2.2.2.3.cmml"><mo stretchy="false" id="alg0.l15.m2.2.2.2.3" xref="alg0.l15.m2.2.2.3.cmml">{</mo><msub id="alg0.l15.m2.1.1.1.1" xref="alg0.l15.m2.1.1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l15.m2.1.1.1.1.2" xref="alg0.l15.m2.1.1.1.1.2.cmml">𝒮</mi><mtext id="alg0.l15.m2.1.1.1.1.3" xref="alg0.l15.m2.1.1.1.1.3a.cmml">crucial</mtext></msub><mo id="alg0.l15.m2.2.2.2.4" xref="alg0.l15.m2.2.2.3.cmml">,</mo><msub id="alg0.l15.m2.2.2.2.2" xref="alg0.l15.m2.2.2.2.2.cmml"><mi id="alg0.l15.m2.2.2.2.2.2" xref="alg0.l15.m2.2.2.2.2.2.cmml">r</mi><mi id="alg0.l15.m2.2.2.2.2.3" xref="alg0.l15.m2.2.2.2.2.3.cmml">k</mi></msub><mo stretchy="false" id="alg0.l15.m2.2.2.2.5" xref="alg0.l15.m2.2.2.3.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="alg0.l15.m2.2b"><set id="alg0.l15.m2.2.2.3.cmml" xref="alg0.l15.m2.2.2.2"><apply id="alg0.l15.m2.1.1.1.1.cmml" xref="alg0.l15.m2.1.1.1.1"><csymbol cd="ambiguous" id="alg0.l15.m2.1.1.1.1.1.cmml" xref="alg0.l15.m2.1.1.1.1">subscript</csymbol><ci id="alg0.l15.m2.1.1.1.1.2.cmml" xref="alg0.l15.m2.1.1.1.1.2">𝒮</ci><ci id="alg0.l15.m2.1.1.1.1.3a.cmml" xref="alg0.l15.m2.1.1.1.1.3"><mtext mathsize="70%" id="alg0.l15.m2.1.1.1.1.3.cmml" xref="alg0.l15.m2.1.1.1.1.3">crucial</mtext></ci></apply><apply id="alg0.l15.m2.2.2.2.2.cmml" xref="alg0.l15.m2.2.2.2.2"><csymbol cd="ambiguous" id="alg0.l15.m2.2.2.2.2.1.cmml" xref="alg0.l15.m2.2.2.2.2">subscript</csymbol><ci id="alg0.l15.m2.2.2.2.2.2.cmml" xref="alg0.l15.m2.2.2.2.2.2">𝑟</ci><ci id="alg0.l15.m2.2.2.2.2.3.cmml" xref="alg0.l15.m2.2.2.2.2.3">𝑘</ci></apply></set></annotation-xml><annotation encoding="application/x-tex" id="alg0.l15.m2.2c">\{\mathcal{S}_{\text{crucial}},r_{k}\}</annotation></semantics></math>, <math id="alg0.l15.m3.1" class="ltx_Math" alttext="m" display="inline"><semantics id="alg0.l15.m3.1a"><mi id="alg0.l15.m3.1.1" xref="alg0.l15.m3.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="alg0.l15.m3.1b"><ci id="alg0.l15.m3.1.1.cmml" xref="alg0.l15.m3.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l15.m3.1c">m</annotation></semantics></math>)

</div>
<div id="alg0.l16" class="ltx_listingline">
<span class="ltx_tag ltx_tag_listingline">16:</span>   <span id="alg0.l16.1" class="ltx_text ltx_font_smallcaps">Adaptation</span>(<math id="alg0.l16.m1.1" class="ltx_Math" alttext="m" display="inline"><semantics id="alg0.l16.m1.1a"><mi id="alg0.l16.m1.1.1" xref="alg0.l16.m1.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="alg0.l16.m1.1b"><ci id="alg0.l16.m1.1.1.cmml" xref="alg0.l16.m1.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l16.m1.1c">m</annotation></semantics></math>, <math id="alg0.l16.m2.1" class="ltx_Math" alttext="e" display="inline"><semantics id="alg0.l16.m2.1a"><mi id="alg0.l16.m2.1.1" xref="alg0.l16.m2.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="alg0.l16.m2.1b"><ci id="alg0.l16.m2.1.1.cmml" xref="alg0.l16.m2.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="alg0.l16.m2.1c">e</annotation></semantics></math>, <math id="alg0.l16.m3.1" class="ltx_Math" alttext="\mathcal{S}_{\text{meta}}" display="inline"><semantics id="alg0.l16.m3.1a"><msub id="alg0.l16.m3.1.1" xref="alg0.l16.m3.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="alg0.l16.m3.1.1.2" xref="alg0.l16.m3.1.1.2.cmml">𝒮</mi><mtext id="alg0.l16.m3.1.1.3" xref="alg0.l16.m3.1.1.3a.cmml">meta</mtext></msub><annotation-xml encoding="MathML-Content" id="alg0.l16.m3.1b"><apply id="alg0.l16.m3.1.1.cmml" xref="alg0.l16.m3.1.1"><csymbol cd="ambiguous" id="alg0.l16.m3.1.1.1.cmml" xref="alg0.l16.m3.1.1">subscript</csymbol><ci id="alg0.l16.m3.1.1.2.cmml" xref="alg0.l16.m3.1.1.2">𝒮</ci><ci id="alg0.l16.m3.1.1.3a.cmml" xref="alg0.l16.m3.1.1.3"><mtext mathsize="70%" id="alg0.l16.m3.1.1.3.cmml" xref="alg0.l16.m3.1.1.3">meta</mtext></ci></apply></annotation-xml><annotation encoding="application/x-tex" id="alg0.l16.m3.1c">\mathcal{S}_{\text{meta}}</annotation></semantics></math>)


</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_float"><span id="alg1.3.1.1" class="ltx_text ltx_font_bold">Algorithm 1</span> </span> Continual Learning with CLIN </figcaption>
</figure>
</div>
<div class="ltx_flex_break"></div>
<div class="ltx_flex_cell ltx_flex_size_1">
<div id="S3.F3.1" class="ltx_block ltx_figure_panel ltx_minipage ltx_align_middle" style="width:190.8pt;">
<img src="/html/2310.10134/assets/x3.png" id="S3.F3.1.g1" class="ltx_graphics ltx_img_square" width="304" height="311" alt="Refer to caption">
</div>
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 3: </span>(<span id="S3.F3.5.1" class="ltx_text ltx_font_smallcaps">left</span>) <span id="S3.F3.6.2" class="ltx_text ltx_font_smallcaps">CLIN</span>’s continual learning algorithm. (<span id="S3.F3.7.3" class="ltx_text ltx_font_smallcaps">right</span>) Example causal abstractions. </figcaption>
</figure>
</section>
<section id="S3.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.2 </span>Memory: A collection of Adaptive Causal Abstractions</h3>

<div id="S3.SS2.p1" class="ltx_para ltx_noindent">
<p id="S3.SS2.p1.1" class="ltx_p">At the end of each trial (completion or failure), CLIN uses a <span id="S3.SS2.p1.1.1" class="ltx_text ltx_font_bold">memory generator</span> to create or update its memory.
The memory generator is a (frozen) LLM prompted to reflect on the current trial and memory, and generate a new memory
of insights in the form of (English sentences expressing) useful <span id="S3.SS2.p1.1.2" class="ltx_text ltx_font_bold">causal abstractions</span>, as we now describe.</p>
</div>
<div id="S3.SS2.p2" class="ltx_para ltx_noindent">
<p id="S3.SS2.p2.1" class="ltx_p">Learning about state transitions is essential for sequential decision-making tasks <cite class="ltx_cite ltx_citemacro_cite">Mnih et al. (<a href="#bib.bib16" title="" class="ltx_ref">2013</a>)</cite>, which can be manifested by knowing 1) actions enabling desired state transitions, 2) actions producing undesired or no change in states, and 3) state transitions that contribute to the task progress. To generate these kinds of knowledge, the generator is prompted to generate insights in a particular syntax (see prompt in <a href="#A1.F7" title="In Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">7</span></a>). To capture good actions enabling desired changes and helpful state transitions, we use the template “<span id="S3.SS2.p2.1.1" class="ltx_text ltx_font_typewriter">X is NECESSARY to Y</span>”, and to capture contrastive examples of unsuitable actions and state transitions, we employ “<span id="S3.SS2.p2.1.2" class="ltx_text ltx_font_typewriter">X DOES NOT CONTRIBUTE to Y</span>”, as depicted in <a href="#S4.F5" title="In 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">5</span></a>, where <span id="S3.SS2.p2.1.3" class="ltx_text ltx_font_typewriter">X</span>, <span id="S3.SS2.p2.1.4" class="ltx_text ltx_font_typewriter">Y</span> are related to actions. These abstractions are functionally analogous to hindsight experience replay <cite class="ltx_cite ltx_citemacro_cite">Andrychowicz et al. (<a href="#bib.bib6" title="" class="ltx_ref">2017</a>)</cite>,
obtained from <span id="S3.SS2.p2.1.5" class="ltx_text ltx_font_smallcaps">CLIN</span>’s past self-explorations.</p>
</div>
<div id="S3.SS2.p3" class="ltx_para ltx_noindent">
<p id="S3.SS2.p3.1" class="ltx_p">While useful insights can be abstracted from the trials, <span id="S3.SS2.p3.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span>’s exploration can be limited, especially in the early trials, given an incredibly large action space. Hence, incomplete exploration can pose varying degrees of uncertainty on extracted insights. To capture this, we also include a measure of uncertainty in each abstraction by either of the two linguistic variations in their surface forms: “<span id="S3.SS2.p3.1.2" class="ltx_text ltx_font_typewriter">X may ...</span>” to denote moderate to high uncertainty, and “<span id="S3.SS2.p3.1.3" class="ltx_text ltx_font_typewriter">X should ...</span>” to indicate low uncertainty (See <a href="#S4.F5" title="In 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">5</span></a>). In the course of continual learning, as <span id="S3.SS2.p3.1.4" class="ltx_text ltx_font_smallcaps">CLIN</span> gathers experience, we expect the level of uncertainty to change depending on the frequency of their use and their fitness to the task.</p>
</div>
<div id="S3.SS2.p4" class="ltx_para ltx_noindent">
<p id="S3.SS2.p4.8" class="ltx_p"><span id="S3.SS2.p4.8.1" class="ltx_text ltx_font_bold">Updating Memory Across Trials.</span> <span id="S3.SS2.p4.8.2" class="ltx_text ltx_font_smallcaps">CLIN</span> continually attempts to solve a task in an environment for multiple trials (in sum, an episode). To update the memory after each trial within an episode, the memory generator is prompted with the most recent trial (a sequence of (<math id="S3.SS2.p4.1.m1.1" class="ltx_Math" alttext="g_{t}" display="inline"><semantics id="S3.SS2.p4.1.m1.1a"><msub id="S3.SS2.p4.1.m1.1.1" xref="S3.SS2.p4.1.m1.1.1.cmml"><mi id="S3.SS2.p4.1.m1.1.1.2" xref="S3.SS2.p4.1.m1.1.1.2.cmml">g</mi><mi id="S3.SS2.p4.1.m1.1.1.3" xref="S3.SS2.p4.1.m1.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.1.m1.1b"><apply id="S3.SS2.p4.1.m1.1.1.cmml" xref="S3.SS2.p4.1.m1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.1.m1.1.1.1.cmml" xref="S3.SS2.p4.1.m1.1.1">subscript</csymbol><ci id="S3.SS2.p4.1.m1.1.1.2.cmml" xref="S3.SS2.p4.1.m1.1.1.2">𝑔</ci><ci id="S3.SS2.p4.1.m1.1.1.3.cmml" xref="S3.SS2.p4.1.m1.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.1.m1.1c">g_{t}</annotation></semantics></math>, <math id="S3.SS2.p4.2.m2.1" class="ltx_Math" alttext="a_{t}" display="inline"><semantics id="S3.SS2.p4.2.m2.1a"><msub id="S3.SS2.p4.2.m2.1.1" xref="S3.SS2.p4.2.m2.1.1.cmml"><mi id="S3.SS2.p4.2.m2.1.1.2" xref="S3.SS2.p4.2.m2.1.1.2.cmml">a</mi><mi id="S3.SS2.p4.2.m2.1.1.3" xref="S3.SS2.p4.2.m2.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.2.m2.1b"><apply id="S3.SS2.p4.2.m2.1.1.cmml" xref="S3.SS2.p4.2.m2.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.2.m2.1.1.1.cmml" xref="S3.SS2.p4.2.m2.1.1">subscript</csymbol><ci id="S3.SS2.p4.2.m2.1.1.2.cmml" xref="S3.SS2.p4.2.m2.1.1.2">𝑎</ci><ci id="S3.SS2.p4.2.m2.1.1.3.cmml" xref="S3.SS2.p4.2.m2.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.2.m2.1c">a_{t}</annotation></semantics></math>, <math id="S3.SS2.p4.3.m3.1" class="ltx_Math" alttext="o_{t}" display="inline"><semantics id="S3.SS2.p4.3.m3.1a"><msub id="S3.SS2.p4.3.m3.1.1" xref="S3.SS2.p4.3.m3.1.1.cmml"><mi id="S3.SS2.p4.3.m3.1.1.2" xref="S3.SS2.p4.3.m3.1.1.2.cmml">o</mi><mi id="S3.SS2.p4.3.m3.1.1.3" xref="S3.SS2.p4.3.m3.1.1.3.cmml">t</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.3.m3.1b"><apply id="S3.SS2.p4.3.m3.1.1.cmml" xref="S3.SS2.p4.3.m3.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.3.m3.1.1.1.cmml" xref="S3.SS2.p4.3.m3.1.1">subscript</csymbol><ci id="S3.SS2.p4.3.m3.1.1.2.cmml" xref="S3.SS2.p4.3.m3.1.1.2">𝑜</ci><ci id="S3.SS2.p4.3.m3.1.1.3.cmml" xref="S3.SS2.p4.3.m3.1.1.3">𝑡</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.3.m3.1c">o_{t}</annotation></semantics></math>) tuples and the final reward <math id="S3.SS2.p4.4.m4.1" class="ltx_Math" alttext="r_{k}" display="inline"><semantics id="S3.SS2.p4.4.m4.1a"><msub id="S3.SS2.p4.4.m4.1.1" xref="S3.SS2.p4.4.m4.1.1.cmml"><mi id="S3.SS2.p4.4.m4.1.1.2" xref="S3.SS2.p4.4.m4.1.1.2.cmml">r</mi><mi id="S3.SS2.p4.4.m4.1.1.3" xref="S3.SS2.p4.4.m4.1.1.3.cmml">k</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.4.m4.1b"><apply id="S3.SS2.p4.4.m4.1.1.cmml" xref="S3.SS2.p4.4.m4.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.4.m4.1.1.1.cmml" xref="S3.SS2.p4.4.m4.1.1">subscript</csymbol><ci id="S3.SS2.p4.4.m4.1.1.2.cmml" xref="S3.SS2.p4.4.m4.1.1.2">𝑟</ci><ci id="S3.SS2.p4.4.m4.1.1.3.cmml" xref="S3.SS2.p4.4.m4.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.4.m4.1c">r_{k}</annotation></semantics></math><span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>
The reward is converted to NL feedback for a LLM using 7 simple rules, e.g., “if score ¿= 0 and score ¡ 20 then feedback = <span id="footnote1.1" class="ltx_text ltx_font_italic">”The agent performed poorly and made some progress but not enough to solve the task.”</span></span></span></span>
),
and the memories from the three most recent trials <math id="S3.SS2.p4.5.m5.3" class="ltx_Math" alttext="\{\mathcal{S}_{k-2},\mathcal{S}_{k-1},\mathcal{S}_{k}\}" display="inline"><semantics id="S3.SS2.p4.5.m5.3a"><mrow id="S3.SS2.p4.5.m5.3.3.3" xref="S3.SS2.p4.5.m5.3.3.4.cmml"><mo stretchy="false" id="S3.SS2.p4.5.m5.3.3.3.4" xref="S3.SS2.p4.5.m5.3.3.4.cmml">{</mo><msub id="S3.SS2.p4.5.m5.1.1.1.1" xref="S3.SS2.p4.5.m5.1.1.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.SS2.p4.5.m5.1.1.1.1.2" xref="S3.SS2.p4.5.m5.1.1.1.1.2.cmml">𝒮</mi><mrow id="S3.SS2.p4.5.m5.1.1.1.1.3" xref="S3.SS2.p4.5.m5.1.1.1.1.3.cmml"><mi id="S3.SS2.p4.5.m5.1.1.1.1.3.2" xref="S3.SS2.p4.5.m5.1.1.1.1.3.2.cmml">k</mi><mo id="S3.SS2.p4.5.m5.1.1.1.1.3.1" xref="S3.SS2.p4.5.m5.1.1.1.1.3.1.cmml">−</mo><mn id="S3.SS2.p4.5.m5.1.1.1.1.3.3" xref="S3.SS2.p4.5.m5.1.1.1.1.3.3.cmml">2</mn></mrow></msub><mo id="S3.SS2.p4.5.m5.3.3.3.5" xref="S3.SS2.p4.5.m5.3.3.4.cmml">,</mo><msub id="S3.SS2.p4.5.m5.2.2.2.2" xref="S3.SS2.p4.5.m5.2.2.2.2.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.SS2.p4.5.m5.2.2.2.2.2" xref="S3.SS2.p4.5.m5.2.2.2.2.2.cmml">𝒮</mi><mrow id="S3.SS2.p4.5.m5.2.2.2.2.3" xref="S3.SS2.p4.5.m5.2.2.2.2.3.cmml"><mi id="S3.SS2.p4.5.m5.2.2.2.2.3.2" xref="S3.SS2.p4.5.m5.2.2.2.2.3.2.cmml">k</mi><mo id="S3.SS2.p4.5.m5.2.2.2.2.3.1" xref="S3.SS2.p4.5.m5.2.2.2.2.3.1.cmml">−</mo><mn id="S3.SS2.p4.5.m5.2.2.2.2.3.3" xref="S3.SS2.p4.5.m5.2.2.2.2.3.3.cmml">1</mn></mrow></msub><mo id="S3.SS2.p4.5.m5.3.3.3.6" xref="S3.SS2.p4.5.m5.3.3.4.cmml">,</mo><msub id="S3.SS2.p4.5.m5.3.3.3.3" xref="S3.SS2.p4.5.m5.3.3.3.3.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.SS2.p4.5.m5.3.3.3.3.2" xref="S3.SS2.p4.5.m5.3.3.3.3.2.cmml">𝒮</mi><mi id="S3.SS2.p4.5.m5.3.3.3.3.3" xref="S3.SS2.p4.5.m5.3.3.3.3.3.cmml">k</mi></msub><mo stretchy="false" id="S3.SS2.p4.5.m5.3.3.3.7" xref="S3.SS2.p4.5.m5.3.3.4.cmml">}</mo></mrow><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.5.m5.3b"><set id="S3.SS2.p4.5.m5.3.3.4.cmml" xref="S3.SS2.p4.5.m5.3.3.3"><apply id="S3.SS2.p4.5.m5.1.1.1.1.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.5.m5.1.1.1.1.1.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1">subscript</csymbol><ci id="S3.SS2.p4.5.m5.1.1.1.1.2.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1.2">𝒮</ci><apply id="S3.SS2.p4.5.m5.1.1.1.1.3.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1.3"><minus id="S3.SS2.p4.5.m5.1.1.1.1.3.1.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1.3.1"></minus><ci id="S3.SS2.p4.5.m5.1.1.1.1.3.2.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1.3.2">𝑘</ci><cn type="integer" id="S3.SS2.p4.5.m5.1.1.1.1.3.3.cmml" xref="S3.SS2.p4.5.m5.1.1.1.1.3.3">2</cn></apply></apply><apply id="S3.SS2.p4.5.m5.2.2.2.2.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2"><csymbol cd="ambiguous" id="S3.SS2.p4.5.m5.2.2.2.2.1.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2">subscript</csymbol><ci id="S3.SS2.p4.5.m5.2.2.2.2.2.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2.2">𝒮</ci><apply id="S3.SS2.p4.5.m5.2.2.2.2.3.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2.3"><minus id="S3.SS2.p4.5.m5.2.2.2.2.3.1.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2.3.1"></minus><ci id="S3.SS2.p4.5.m5.2.2.2.2.3.2.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2.3.2">𝑘</ci><cn type="integer" id="S3.SS2.p4.5.m5.2.2.2.2.3.3.cmml" xref="S3.SS2.p4.5.m5.2.2.2.2.3.3">1</cn></apply></apply><apply id="S3.SS2.p4.5.m5.3.3.3.3.cmml" xref="S3.SS2.p4.5.m5.3.3.3.3"><csymbol cd="ambiguous" id="S3.SS2.p4.5.m5.3.3.3.3.1.cmml" xref="S3.SS2.p4.5.m5.3.3.3.3">subscript</csymbol><ci id="S3.SS2.p4.5.m5.3.3.3.3.2.cmml" xref="S3.SS2.p4.5.m5.3.3.3.3.2">𝒮</ci><ci id="S3.SS2.p4.5.m5.3.3.3.3.3.cmml" xref="S3.SS2.p4.5.m5.3.3.3.3.3">𝑘</ci></apply></set></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.5.m5.3c">\{\mathcal{S}_{k-2},\mathcal{S}_{k-1},\mathcal{S}_{k}\}</annotation></semantics></math>. It is then prompted to generate an updated memory <math id="S3.SS2.p4.6.m6.1" class="ltx_Math" alttext="\mathcal{S}_{k+1}" display="inline"><semantics id="S3.SS2.p4.6.m6.1a"><msub id="S3.SS2.p4.6.m6.1.1" xref="S3.SS2.p4.6.m6.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.SS2.p4.6.m6.1.1.2" xref="S3.SS2.p4.6.m6.1.1.2.cmml">𝒮</mi><mrow id="S3.SS2.p4.6.m6.1.1.3" xref="S3.SS2.p4.6.m6.1.1.3.cmml"><mi id="S3.SS2.p4.6.m6.1.1.3.2" xref="S3.SS2.p4.6.m6.1.1.3.2.cmml">k</mi><mo id="S3.SS2.p4.6.m6.1.1.3.1" xref="S3.SS2.p4.6.m6.1.1.3.1.cmml">+</mo><mn id="S3.SS2.p4.6.m6.1.1.3.3" xref="S3.SS2.p4.6.m6.1.1.3.3.cmml">1</mn></mrow></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.6.m6.1b"><apply id="S3.SS2.p4.6.m6.1.1.cmml" xref="S3.SS2.p4.6.m6.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.6.m6.1.1.1.cmml" xref="S3.SS2.p4.6.m6.1.1">subscript</csymbol><ci id="S3.SS2.p4.6.m6.1.1.2.cmml" xref="S3.SS2.p4.6.m6.1.1.2">𝒮</ci><apply id="S3.SS2.p4.6.m6.1.1.3.cmml" xref="S3.SS2.p4.6.m6.1.1.3"><plus id="S3.SS2.p4.6.m6.1.1.3.1.cmml" xref="S3.SS2.p4.6.m6.1.1.3.1"></plus><ci id="S3.SS2.p4.6.m6.1.1.3.2.cmml" xref="S3.SS2.p4.6.m6.1.1.3.2">𝑘</ci><cn type="integer" id="S3.SS2.p4.6.m6.1.1.3.3.cmml" xref="S3.SS2.p4.6.m6.1.1.3.3">1</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.6.m6.1c">\mathcal{S}_{k+1}</annotation></semantics></math>, namely a new list of semi-structured causal abstractions in the forms described above, for use in the next trial. Although we do not specify a maximum size for the memory, we observe that size of the generated memory (i.e., the number of causal abstractions generated) is far less than the number of actions executed in the trial, indicating the memory-generator additionally performs a saliency-based pruning to keep only important insights based on the success of the trial, as indicated by the final reward <math id="S3.SS2.p4.7.m7.1" class="ltx_Math" alttext="r_{k}" display="inline"><semantics id="S3.SS2.p4.7.m7.1a"><msub id="S3.SS2.p4.7.m7.1.1" xref="S3.SS2.p4.7.m7.1.1.cmml"><mi id="S3.SS2.p4.7.m7.1.1.2" xref="S3.SS2.p4.7.m7.1.1.2.cmml">r</mi><mi id="S3.SS2.p4.7.m7.1.1.3" xref="S3.SS2.p4.7.m7.1.1.3.cmml">k</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.7.m7.1b"><apply id="S3.SS2.p4.7.m7.1.1.cmml" xref="S3.SS2.p4.7.m7.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.7.m7.1.1.1.cmml" xref="S3.SS2.p4.7.m7.1.1">subscript</csymbol><ci id="S3.SS2.p4.7.m7.1.1.2.cmml" xref="S3.SS2.p4.7.m7.1.1.2">𝑟</ci><ci id="S3.SS2.p4.7.m7.1.1.3.cmml" xref="S3.SS2.p4.7.m7.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.7.m7.1c">r_{k}</annotation></semantics></math> at the end of the trial <math id="S3.SS2.p4.8.m8.1" class="ltx_Math" alttext="\mathcal{T}_{k}" display="inline"><semantics id="S3.SS2.p4.8.m8.1a"><msub id="S3.SS2.p4.8.m8.1.1" xref="S3.SS2.p4.8.m8.1.1.cmml"><mi class="ltx_font_mathcaligraphic" id="S3.SS2.p4.8.m8.1.1.2" xref="S3.SS2.p4.8.m8.1.1.2.cmml">𝒯</mi><mi id="S3.SS2.p4.8.m8.1.1.3" xref="S3.SS2.p4.8.m8.1.1.3.cmml">k</mi></msub><annotation-xml encoding="MathML-Content" id="S3.SS2.p4.8.m8.1b"><apply id="S3.SS2.p4.8.m8.1.1.cmml" xref="S3.SS2.p4.8.m8.1.1"><csymbol cd="ambiguous" id="S3.SS2.p4.8.m8.1.1.1.cmml" xref="S3.SS2.p4.8.m8.1.1">subscript</csymbol><ci id="S3.SS2.p4.8.m8.1.1.2.cmml" xref="S3.SS2.p4.8.m8.1.1.2">𝒯</ci><ci id="S3.SS2.p4.8.m8.1.1.3.cmml" xref="S3.SS2.p4.8.m8.1.1.3">𝑘</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S3.SS2.p4.8.m8.1c">\mathcal{T}_{k}</annotation></semantics></math>.</p>
</div>
</section>
<section id="S3.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">3.3 </span>Meta-Memory for Better Generalization</h3>

<div id="S3.SS3.p1" class="ltx_para ltx_noindent">
<p id="S3.SS3.p1.1" class="ltx_p">Updating memory based on past insights and the current trial to influence future trials for the same task in the current environment configuration during test-time adaptation. However, to generalize across tasks or environment configurations, the memory needs to contain more generalized causal abstractions than memories used across trials in an episode. We call this as <span id="S3.SS3.p1.1.1" class="ltx_text ltx_font_bold">meta-memory</span>, abstracted across multiple episodes of solving different tasks in different environment configurations to be applicable in future episodes.</p>
</div>
<div id="S3.SS3.p2" class="ltx_para ltx_noindent">
<p id="S3.SS3.p2.1" class="ltx_p"><span id="S3.SS3.p2.1.1" class="ltx_text ltx_font_bold">Auto-curriculum Selection.</span> Before we generate the meta-memory, it is important to choose memories extracted from the best trials from previous episodes because random sampling may not benefit <span id="S3.SS3.p2.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> in zero-shot generalization <cite class="ltx_cite ltx_citemacro_cite">Adaptive-Agent-Team et al. (<a href="#bib.bib1" title="" class="ltx_ref">2023</a>)</cite>. Following the prioritized level replay scheme <cite class="ltx_cite ltx_citemacro_cite">Jiang et al. (<a href="#bib.bib13" title="" class="ltx_ref">2021</a>)</cite>, we choose the most successful trial per episode and retrieve memories abstracted from those trials with a fixed archive of size 10, a hyperparameter.</p>
</div>
<div id="S3.SS3.p3" class="ltx_para ltx_noindent">
<p id="S3.SS3.p3.1" class="ltx_p"><span id="S3.SS3.p3.1.1" class="ltx_text ltx_font_bold">Generating Meta-Memory.</span> The goal of the meta-memory is to help <span id="S3.SS3.p3.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> generalize to unseen tasks and/or environments. While we keep the format of the causal abstractions the same as memories generated across trials, the prompt to generate the meta-memory is different than those used for generating per-trial memory. When the new memory is to be used for the <span id="S3.SS3.p3.1.3" class="ltx_text ltx_font_italic">same</span> task but in a <span id="S3.SS3.p3.1.4" class="ltx_text ltx_font_italic">different</span> environment, the prompt instructs for a meta-memory helpful “<span id="S3.SS3.p3.1.5" class="ltx_text ltx_font_typewriter">to solve the same task in a new environment configuration</span>” given the target task description with an expectation that meta-memory abstractions will entail generic causal insights about the task irrespective of environment configurations (see <a href="#A1.F8" title="In Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">8</span></a>). Similarly, when the new memory is to be used for a <span id="S3.SS3.p3.1.6" class="ltx_text ltx_font_italic">different</span> task, the prompt is modified accordingly to reflect this (<a href="#A1.F9" title="In Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">9</span></a>). Along with the target task description for better memory generation, each past memory selected to generate the meta-memory is attached to the final rewards for the associated trials, allowing the generator to combine insights across episodes and assign the levels of uncertainty using the evidence of success.</p>
</div>
</section>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Results and Analysis</h2>

<div id="S4.p1" class="ltx_para ltx_noindent">
<p id="S4.p1.1" class="ltx_p"><span id="S4.p1.1.1" class="ltx_text ltx_font_bold">Experimental Setup.</span>   
Test-time adaptation and generalization via continual learning require a variety of complex tasks and environment configurations to allow an agent to explore, learn latent causal insights from interactions, and exploit them in the future. We choose ScienceWorld <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib23" title="" class="ltx_ref">2022</a>)</cite>, a text-based interactive environment requiring complex interactive reasoning processes to solve a plethora of science-theory-based tasks spanning several diverse classes (e.g., thermodynamics, genetics, friction, etc.). The virtual space presents 10 sub-places:
foundry, greenhouse, outside area, an art studio, workshop, kitchen, living room, bedroom, bathroom, and a hallway connecting inside rooms. The presence of several objects, their individual states, and action templates renders the search space intractable for any agent. ScienceWorld presents strikingly different environment configurations across task types, making it a rich testbed for evaluating adaptation and generalization. ScienceWorld tasks are partitioned into Short (S), e.g., <span id="S4.p1.1.2" class="ltx_text ltx_font_italic">pick &amp; place</span> and Long (L), e.g., <span id="S4.p1.1.3" class="ltx_text ltx_font_italic">grow plant</span>, tasks based on the number of required actions to succeed.</p>
</div>
<div id="S4.p2" class="ltx_para ltx_noindent">
<p id="S4.p2.1" class="ltx_p">Here, we define our setups for zero-shot adaptation (<span id="S4.p2.1.1" class="ltx_text ltx_font_smallcaps">Adapt</span>) and generalization (<span id="S4.p2.1.2" class="ltx_text ltx_font_smallcaps">Gen-Env</span> and <span id="S4.p2.1.3" class="ltx_text ltx_font_smallcaps">Gen-Task</span>). For all setups, we test <span id="S4.p2.1.4" class="ltx_text ltx_font_smallcaps">CLIN</span> and competing baselines on 18 tasks (two task instances from 9 classes) in several environment configurations from the test split of the ScienceWorld benchmark resulting in a total of 164 task-environment combinations unless stated otherwise. We evaluate based on the final reward score provided by the ScienceWorld simulator.</p>
</div>
<div id="S4.p3" class="ltx_para ltx_noindent">
<p id="S4.p3.1" class="ltx_p"><span id="S4.p3.1.1" class="ltx_text ltx_font_bold ltx_font_smallcaps">Adapt</span>: This setup focuses on <span id="S4.p3.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span>’s ability to adapt to a task by attempting it for several trials in the same environment configuration. Most importantly, <span id="S4.p3.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> initializes with an empty memory at the beginning of the first trial and generates memory at the end of each trial. While the environment gets reset at the trial boundary, <span id="S4.p3.1.4" class="ltx_text ltx_font_smallcaps">CLIN</span>’s memory continues to be updated, capturing informative causal abstractions pertaining to both successful and failed actions. Here, we compare with Reflexion <cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>)</cite>, a SOTA, however, <span id="S4.p3.1.5" class="ltx_text ltx_font_smallcaps">CLIN</span> differs from Reflexion by how the memory is abstracted.</p>
</div>
<div id="S4.p4" class="ltx_para ltx_noindent">
<p id="S4.p4.1" class="ltx_p"><span id="S4.p4.1.1" class="ltx_text ltx_font_bold ltx_font_smallcaps">Gen-Env</span>: In this setup, we focus on <span id="S4.p4.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span>’s ability to transfer its learning from past experiences to solve tasks in an unseen environment. For a task <math id="S4.p4.1.m1.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S4.p4.1.m1.1a"><mi id="S4.p4.1.m1.1.1" xref="S4.p4.1.m1.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S4.p4.1.m1.1b"><ci id="S4.p4.1.m1.1.1.cmml" xref="S4.p4.1.m1.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p4.1.m1.1c">m</annotation></semantics></math>, we run <span id="S4.p4.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> for 10 different (train) environment settings (with varying objects and starting locations) and then create meta-memories from its exploration to solve the same task in an unseen (test) environment. Here, we compare <span id="S4.p4.1.4" class="ltx_text ltx_font_smallcaps">CLIN</span> with RL methods
DRRN <cite class="ltx_cite ltx_citemacro_cite">He et al. (<a href="#bib.bib11" title="" class="ltx_ref">2015</a>)</cite>, KG-A2C <cite class="ltx_cite ltx_citemacro_cite">Ammanabrolu &amp; Hausknecht (<a href="#bib.bib4" title="" class="ltx_ref">2020</a>)</cite>, and CALM <cite class="ltx_cite ltx_citemacro_cite">Yao et al. (<a href="#bib.bib24" title="" class="ltx_ref">2020</a>)</cite> trained on all (large) training variations with simulator reward and Generative Language agents, SayCan <cite class="ltx_cite ltx_citemacro_cite">Ahn et al. (<a href="#bib.bib2" title="" class="ltx_ref">2022</a>)</cite>, ReAct <cite class="ltx_cite ltx_citemacro_cite">Yao et al. (<a href="#bib.bib25" title="" class="ltx_ref">2022</a>)</cite>,
and Reflexion <cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>)</cite>, prompted with few-shot demonstrations.</p>
</div>
<div id="S4.p5" class="ltx_para ltx_noindent">
<p id="S4.p5.4" class="ltx_p"><span id="S4.p5.4.1" class="ltx_text ltx_font_bold ltx_font_smallcaps">Gen-Task</span>: In this setup, we focus on <span id="S4.p5.4.2" class="ltx_text ltx_font_smallcaps">CLIN</span>’s ability to transfer its learning from past experiences to solve a new task in the same environment. For an environment <math id="S4.p5.1.m1.1" class="ltx_Math" alttext="e" display="inline"><semantics id="S4.p5.1.m1.1a"><mi id="S4.p5.1.m1.1.1" xref="S4.p5.1.m1.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="S4.p5.1.m1.1b"><ci id="S4.p5.1.m1.1.1.cmml" xref="S4.p5.1.m1.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p5.1.m1.1c">e</annotation></semantics></math>, we run <span id="S4.p5.4.3" class="ltx_text ltx_font_smallcaps">CLIN</span> for to solve a task <math id="S4.p5.2.m2.1" class="ltx_Math" alttext="m" display="inline"><semantics id="S4.p5.2.m2.1a"><mi id="S4.p5.2.m2.1.1" xref="S4.p5.2.m2.1.1.cmml">m</mi><annotation-xml encoding="MathML-Content" id="S4.p5.2.m2.1b"><ci id="S4.p5.2.m2.1.1.cmml" xref="S4.p5.2.m2.1.1">𝑚</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p5.2.m2.1c">m</annotation></semantics></math> and then condense its learning to solve a novel task <math id="S4.p5.3.m3.1" class="ltx_Math" alttext="m^{\prime}" display="inline"><semantics id="S4.p5.3.m3.1a"><msup id="S4.p5.3.m3.1.1" xref="S4.p5.3.m3.1.1.cmml"><mi id="S4.p5.3.m3.1.1.2" xref="S4.p5.3.m3.1.1.2.cmml">m</mi><mo id="S4.p5.3.m3.1.1.3" xref="S4.p5.3.m3.1.1.3.cmml">′</mo></msup><annotation-xml encoding="MathML-Content" id="S4.p5.3.m3.1b"><apply id="S4.p5.3.m3.1.1.cmml" xref="S4.p5.3.m3.1.1"><csymbol cd="ambiguous" id="S4.p5.3.m3.1.1.1.cmml" xref="S4.p5.3.m3.1.1">superscript</csymbol><ci id="S4.p5.3.m3.1.1.2.cmml" xref="S4.p5.3.m3.1.1.2">𝑚</ci><ci id="S4.p5.3.m3.1.1.3.cmml" xref="S4.p5.3.m3.1.1.3">′</ci></apply></annotation-xml><annotation encoding="application/x-tex" id="S4.p5.3.m3.1c">m^{\prime}</annotation></semantics></math> in the environment <math id="S4.p5.4.m4.1" class="ltx_Math" alttext="e" display="inline"><semantics id="S4.p5.4.m4.1a"><mi id="S4.p5.4.m4.1.1" xref="S4.p5.4.m4.1.1.cmml">e</mi><annotation-xml encoding="MathML-Content" id="S4.p5.4.m4.1b"><ci id="S4.p5.4.m4.1.1.cmml" xref="S4.p5.4.m4.1.1">𝑒</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.p5.4.m4.1c">e</annotation></semantics></math>. We took all test examples where we have a different task defined in the same environment configuration. <cite class="ltx_cite ltx_citemacro_cite">Adaptive-Agent-Team et al. (<a href="#bib.bib1" title="" class="ltx_ref">2023</a>)</cite> suggests that transferring learning from a random task can be very hard; hence we couple tasks that are related (revolve around overlapping task-critical objects/locations such water, kitchen), such as <span id="S4.p5.4.4" class="ltx_text ltx_font_italic">boil</span> and <span id="S4.p5.4.5" class="ltx_text ltx_font_italic">freeze</span> to measure transfer learning from one to the other. This is a novel setup where we do not have any off-the-shelf baselines. However, here, we compare against <span id="S4.p5.4.6" class="ltx_text ltx_font_smallcaps">CLIN</span>-<span id="S4.p5.4.7" class="ltx_text ltx_font_smallcaps">Base</span>, a strong baseline agent.</p>
</div>
<div id="S4.p6" class="ltx_para ltx_noindent">
<p id="S4.p6.1" class="ltx_p"><span id="S4.p6.1.1" class="ltx_text ltx_font_bold ltx_font_smallcaps">Gen-Adapt</span> (<span id="S4.p6.1.2" class="ltx_text ltx_font_smallcaps">G+A</span>): If <span id="S4.p6.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span>, in <span id="S4.p6.1.4" class="ltx_text ltx_font_smallcaps">Gen-Env</span> or <span id="S4.p6.1.5" class="ltx_text ltx_font_smallcaps">Gen-Task</span> setting, does not successfully complete the new task, it can continue learning and retrying that task. We refer to this setup as <span id="S4.p6.1.6" class="ltx_text ltx_font_smallcaps">Gen-Adapt</span>.
<span id="S4.p6.1.7" class="ltx_text ltx_font_smallcaps">CLIN</span> can use any instruction-tuned LLM <cite class="ltx_cite ltx_citemacro_cite">Chung et al. (<a href="#bib.bib9" title="" class="ltx_ref">2022</a>)</cite> as part of the controller, executor, and memory generator. In this paper, we use <span id="S4.p6.1.8" class="ltx_text ltx_font_typewriter">gpt-4</span>, the same as our generative agent baselines.</p>
</div>
<figure id="S4.F4" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<div id="S4.F4.1" class="ltx_block ltx_figure_panel ltx_minipage ltx_align_middle" style="width:278.2pt;">
<img src="/html/2310.10134/assets/x4.png" id="S4.F4.1.g1" class="ltx_graphics ltx_img_landscape" width="304" height="127" alt="Refer to caption">
</div>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<div id="S4.F4.2" class="ltx_block ltx_figure_panel ltx_minipage ltx_align_middle" style="width:119.2pt;">
<div id="S4.F4.2.1" class="ltx_inline-block ltx_transformed_outer" style="width:433.6pt;height:352.9pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(150.5pt,-122.5pt) scale(3.26780374355687,3.26780374355687) ;">
<table id="S4.F4.2.1.1" class="ltx_tabular ltx_align_middle">
<tr id="S4.F4.2.1.1.2" class="ltx_tr">
<td id="S4.F4.2.1.1.2.1" class="ltx_td ltx_align_center" colspan="3">
<span id="S4.F4.2.1.1.2.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span><span id="S4.F4.2.1.1.2.1.2" class="ltx_text ltx_font_bold">’s </span><span id="S4.F4.2.1.1.2.1.3" class="ltx_text ltx_font_smallcaps">Adapt</span><span id="S4.F4.2.1.1.2.1.4" class="ltx_text ltx_font_bold"> improvements</span>
</td>
</tr>
<tr id="S4.F4.2.1.1.3" class="ltx_tr">
<td id="S4.F4.2.1.1.3.1" class="ltx_td ltx_align_left ltx_border_tt"><span id="S4.F4.2.1.1.3.1.1" class="ltx_text ltx_font_bold">Type</span></td>
<td id="S4.F4.2.1.1.3.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.F4.2.1.1.3.2.1" class="ltx_text ltx_font_bold">#trials to</span></td>
<td id="S4.F4.2.1.1.3.3" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_tt"><span id="S4.F4.2.1.1.3.3.1" class="ltx_text ltx_font_bold">%ep.</span></td>
</tr>
<tr id="S4.F4.2.1.1.1" class="ltx_tr">
<td id="S4.F4.2.1.1.1.2" class="ltx_td"></td>
<td id="S4.F4.2.1.1.1.1" class="ltx_td ltx_align_center"><span id="S4.F4.2.1.1.1.1.1" class="ltx_text ltx_font_bold">success (<math id="S4.F4.2.1.1.1.1.1.m1.1" class="ltx_Math" alttext="\downarrow" display="inline"><semantics id="S4.F4.2.1.1.1.1.1.m1.1a"><mo stretchy="false" id="S4.F4.2.1.1.1.1.1.m1.1.1" xref="S4.F4.2.1.1.1.1.1.m1.1.1.cmml">↓</mo><annotation-xml encoding="MathML-Content" id="S4.F4.2.1.1.1.1.1.m1.1b"><ci id="S4.F4.2.1.1.1.1.1.m1.1.1.cmml" xref="S4.F4.2.1.1.1.1.1.m1.1.1">↓</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F4.2.1.1.1.1.1.m1.1c">\downarrow</annotation></semantics></math>)</span></td>
<td id="S4.F4.2.1.1.1.3" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.F4.2.1.1.1.3.1" class="ltx_text ltx_font_bold">improv.</span></td>
</tr>
<tr id="S4.F4.2.1.1.4" class="ltx_tr">
<td id="S4.F4.2.1.1.4.1" class="ltx_td ltx_align_left ltx_border_t">S</td>
<td id="S4.F4.2.1.1.4.2" class="ltx_td ltx_align_center ltx_border_t">3.3</td>
<td id="S4.F4.2.1.1.4.3" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t">29.2</td>
</tr>
<tr id="S4.F4.2.1.1.5" class="ltx_tr">
<td id="S4.F4.2.1.1.5.1" class="ltx_td ltx_align_left">L</td>
<td id="S4.F4.2.1.1.5.2" class="ltx_td ltx_align_center">3.2</td>
<td id="S4.F4.2.1.1.5.3" class="ltx_td ltx_nopad_r ltx_align_center">37.2</td>
</tr>
<tr id="S4.F4.2.1.1.6" class="ltx_tr">
<td id="S4.F4.2.1.1.6.1" class="ltx_td ltx_align_left ltx_border_bb">All</td>
<td id="S4.F4.2.1.1.6.2" class="ltx_td ltx_align_center ltx_border_bb">3.3</td>
<td id="S4.F4.2.1.1.6.3" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_bb">33.2</td>
</tr>
</table>
</span></div>
<p id="S4.F4.2.2" class="ltx_p ltx_align_center">(c)</p>
</div>
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 4: </span><span id="S4.F4.15.1" class="ltx_text ltx_font_bold">Rapid task adaptation with <span id="S4.F4.15.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span>.</span>
<span id="S4.F4.16.2" class="ltx_text ltx_font_bold">(a)</span> Example tasks where <span id="S4.F4.17.3" class="ltx_text ltx_font_smallcaps">CLIN</span> improves scores across trials. For <span id="S4.F4.18.4" class="ltx_text ltx_font_smallcaps">CLIN</span>, Trial-0 is the <span id="S4.F4.19.5" class="ltx_text ltx_font_smallcaps">Base</span>, Trial-4 is the <span id="S4.F4.20.6" class="ltx_text ltx_font_smallcaps">Adapt</span>. <span id="S4.F4.21.7" class="ltx_text ltx_font_bold">(b)</span> Comparison of <span id="S4.F4.22.8" class="ltx_text ltx_font_smallcaps">CLIN</span> with Reflexion <cite class="ltx_cite ltx_citemacro_cite">Shinn et al. (<a href="#bib.bib20" title="" class="ltx_ref">2023</a>)</cite>.
<span id="S4.F4.23.9" class="ltx_text ltx_font_bold">(c)</span> <span id="S4.F4.24.10" class="ltx_text ltx_font_smallcaps">CLIN</span> improves from <span id="S4.F4.25.11" class="ltx_text ltx_font_smallcaps">Base</span> to <span id="S4.F4.26.12" class="ltx_text ltx_font_smallcaps">Adapt</span> (full results in <a href="#A3" title="Appendix C More results ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">C</span></a>). </figcaption>
</figure>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span><span id="S4.SS1.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> Exhibits Rapid Task Adaptation</h3>

<div id="S4.SS1.p1" class="ltx_para ltx_noindent">
<p id="S4.SS1.p1.1" class="ltx_p"><a href="#S4.F4" title="In 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">4</span></a>a demonstrates two example trends where <span id="S4.SS1.p1.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> learns from its own prior attempts (<span id="S4.SS1.p1.1.2" class="ltx_text ltx_font_smallcaps">Adapt</span>) and gets better at solving a given task. Apart from length, the difficulty level of a task also depends on the environment configuration (hence, variance across environment configurations for each task). <span id="S4.SS1.p1.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> quickly adapts to a short task, <span id="S4.SS1.p1.1.4" class="ltx_text ltx_font_italic">Pick &amp; Place</span>, solving it in its 4th attempt, whereas for a longer task, <span id="S4.SS1.p1.1.5" class="ltx_text ltx_font_italic">Grow Fruit</span>, it is not solved after 5th (max) attempts.
Furthermore, <a href="#S4.F4" title="In 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">4</span></a>a depicts, <span id="S4.SS1.p1.1.6" class="ltx_text ltx_font_smallcaps">CLIN</span> becomes more efficient in later trials by solving the tasks with a lower number of (average) steps. Figure <a href="#S4.F4" title="Figure 4 ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>c shows an average number of attempts<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note">2</span>During <span id="footnote2.1" class="ltx_text ltx_font_smallcaps">Adapt</span>, <span id="footnote2.2" class="ltx_text ltx_font_smallcaps">CLIN</span> tries up to 5 trials. If it solves a task with a score of 100, it stops retrying.</span></span></span> for <span id="S4.SS1.p1.1.7" class="ltx_text ltx_font_smallcaps">CLIN</span> to solve a task and % episodes per task where scores improved compared to its own first trial.</p>
</div>
<div id="S4.SS1.p2" class="ltx_para ltx_noindent">
<p id="S4.SS1.p2.1" class="ltx_p">Next, we compare <span id="S4.SS1.p2.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> with Reflexion, the reflective SOTA agent, in <a href="#S4.F4" title="In 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">4</span></a>b. <span id="S4.SS1.p2.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> already starts off with a stronger base performance (see discussion in <a href="#S4.SS3" title="4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a>), however, <span id="S4.SS1.p2.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span>’s  relative improvement in <span id="S4.SS1.p2.1.4" class="ltx_text ltx_font_smallcaps">Adapt</span> is significantly stronger than Reflexion’s gain from its base agent ReAct. Furthermore, <span id="S4.SS1.p2.1.5" class="ltx_text ltx_font_smallcaps">CLIN</span>’s relative improvement is higher for longer tasks.
This can be attributed to <span id="S4.SS1.p2.1.6" class="ltx_text ltx_font_smallcaps">CLIN</span>’s persistent memory, which gets refined over past trials, whereas Reflexion may fall short of collecting useful learnings from earlier trials as it only focuses on the current trial for its reflections (hence not long-term). Furthermore, <span id="S4.SS1.p2.1.7" class="ltx_text ltx_font_smallcaps">CLIN</span> accumulates both useful (for the task) and harmful (for the task) causal learnings, whereas Reflexion only learns from its mistakes, lacking comprehensive learning.</p>
</div>
<figure id="S4.T1" class="ltx_table">
<div id="S4.T1.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:390.3pt;height:343.3pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(-40.2pt,35.4pt) scale(0.829177703663224,0.829177703663224) ;">
<table id="S4.T1.1.1" class="ltx_tabular ltx_align_middle">
<tr id="S4.T1.1.1.1" class="ltx_tr">
<td id="S4.T1.1.1.1.1" class="ltx_td ltx_nopad_r ltx_border_tt"></td>
<td id="S4.T1.1.1.1.2" class="ltx_td ltx_nopad_l ltx_border_rr ltx_border_tt"></td>
<td id="S4.T1.1.1.1.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_tt" colspan="3"><span id="S4.T1.1.1.1.3.1" class="ltx_text ltx_font_bold">RL Methods</span></td>
<td id="S4.T1.1.1.1.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_tt" colspan="3"><span id="S4.T1.1.1.1.4.1" class="ltx_text ltx_font_bold">Generative Language Agents</span></td>
<td id="S4.T1.1.1.1.5" class="ltx_td ltx_align_center ltx_border_tt" colspan="3">
<span id="S4.T1.1.1.1.5.1" class="ltx_text ltx_font_smallcaps">CLIN</span><span id="S4.T1.1.1.1.5.2" class="ltx_text ltx_font_bold"> (ours)</span>
</td>
</tr>
<tr id="S4.T1.1.1.2" class="ltx_tr">
<td id="S4.T1.1.1.2.1" class="ltx_td ltx_nopad_r ltx_align_left ltx_border_t"><span id="S4.T1.1.1.2.1.1" class="ltx_text ltx_font_bold">Task</span></td>
<td id="S4.T1.1.1.2.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr ltx_border_t"><span id="S4.T1.1.1.2.2.1" class="ltx_text ltx_font_bold">Type</span></td>
<td id="S4.T1.1.1.2.3" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.2.3.1" class="ltx_text ltx_font_bold">DRRN</span></td>
<td id="S4.T1.1.1.2.4" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.2.4.1" class="ltx_text ltx_font_bold">KGA2C</span></td>
<td id="S4.T1.1.1.2.5" class="ltx_td ltx_align_center ltx_border_r ltx_border_t"><span id="S4.T1.1.1.2.5.1" class="ltx_text ltx_font_bold">CALM</span></td>
<td id="S4.T1.1.1.2.6" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.2.6.1" class="ltx_text ltx_font_bold">SayCan</span></td>
<td id="S4.T1.1.1.2.7" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.2.7.1" class="ltx_text ltx_font_bold">ReAct</span></td>
<td id="S4.T1.1.1.2.8" class="ltx_td ltx_align_center ltx_border_r ltx_border_t"><span id="S4.T1.1.1.2.8.1" class="ltx_text ltx_font_bold">Reflexion</span></td>
<td id="S4.T1.1.1.2.9" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.2.9.1" class="ltx_text ltx_font_smallcaps">Base</span></td>
<td id="S4.T1.1.1.2.10" class="ltx_td ltx_align_center ltx_border_r ltx_border_t"><span id="S4.T1.1.1.2.10.1" class="ltx_text ltx_font_smallcaps">Gen-Env</span></td>
<td id="S4.T1.1.1.2.11" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t"><span id="S4.T1.1.1.2.11.1" class="ltx_text ltx_font_smallcaps">G+A</span></td>
</tr>
<tr id="S4.T1.1.1.3" class="ltx_tr">
<td id="S4.T1.1.1.3.1" class="ltx_td ltx_nopad_r ltx_align_left ltx_border_t">Temp<sub id="S4.T1.1.1.3.1.1" class="ltx_sub">1</sub>
</td>
<td id="S4.T1.1.1.3.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr ltx_border_t">S</td>
<td id="S4.T1.1.1.3.3" class="ltx_td ltx_align_center ltx_border_t">6.6</td>
<td id="S4.T1.1.1.3.4" class="ltx_td ltx_align_center ltx_border_t">6.0</td>
<td id="S4.T1.1.1.3.5" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">1.0</td>
<td id="S4.T1.1.1.3.6" class="ltx_td ltx_align_center ltx_border_t"><span id="S4.T1.1.1.3.6.1" class="ltx_text ltx_font_bold">26.4</span></td>
<td id="S4.T1.1.1.3.7" class="ltx_td ltx_align_center ltx_border_t">7.2</td>
<td id="S4.T1.1.1.3.8" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">5.9</td>
<td id="S4.T1.1.1.3.9" class="ltx_td ltx_align_center ltx_border_t">25.2</td>
<td id="S4.T1.1.1.3.10" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">15.7</td>
<td id="S4.T1.1.1.3.11" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t">13.8</td>
</tr>
<tr id="S4.T1.1.1.4" class="ltx_tr">
<td id="S4.T1.1.1.4.1" class="ltx_td ltx_nopad_r ltx_align_left">Temp<sub id="S4.T1.1.1.4.1.1" class="ltx_sub">2</sub>
</td>
<td id="S4.T1.1.1.4.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.4.3" class="ltx_td ltx_align_center">5.5</td>
<td id="S4.T1.1.1.4.4" class="ltx_td ltx_align_center">11.0</td>
<td id="S4.T1.1.1.4.5" class="ltx_td ltx_align_center ltx_border_r">1.0</td>
<td id="S4.T1.1.1.4.6" class="ltx_td ltx_align_center">8.0</td>
<td id="S4.T1.1.1.4.7" class="ltx_td ltx_align_center">6.1</td>
<td id="S4.T1.1.1.4.8" class="ltx_td ltx_align_center ltx_border_r">28.6</td>
<td id="S4.T1.1.1.4.9" class="ltx_td ltx_align_center">53.2</td>
<td id="S4.T1.1.1.4.10" class="ltx_td ltx_align_center ltx_border_r">49.7</td>
<td id="S4.T1.1.1.4.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.4.11.1" class="ltx_text ltx_font_bold">58.2</span></td>
</tr>
<tr id="S4.T1.1.1.5" class="ltx_tr">
<td id="S4.T1.1.1.5.1" class="ltx_td ltx_nopad_r ltx_align_left">Pick&amp;Place<sub id="S4.T1.1.1.5.1.1" class="ltx_sub">1</sub>
</td>
<td id="S4.T1.1.1.5.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.5.3" class="ltx_td ltx_align_center">15.0</td>
<td id="S4.T1.1.1.5.4" class="ltx_td ltx_align_center">18.0</td>
<td id="S4.T1.1.1.5.5" class="ltx_td ltx_align_center ltx_border_r">10.0</td>
<td id="S4.T1.1.1.5.6" class="ltx_td ltx_align_center">22.9</td>
<td id="S4.T1.1.1.5.7" class="ltx_td ltx_align_center">26.7</td>
<td id="S4.T1.1.1.5.8" class="ltx_td ltx_align_center ltx_border_r">64.9</td>
<td id="S4.T1.1.1.5.9" class="ltx_td ltx_align_center">92.5</td>
<td id="S4.T1.1.1.5.10" class="ltx_td ltx_align_center ltx_border_r">59.2</td>
<td id="S4.T1.1.1.5.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.5.11.1" class="ltx_text ltx_font_bold">100.0</span></td>
</tr>
<tr id="S4.T1.1.1.6" class="ltx_tr">
<td id="S4.T1.1.1.6.1" class="ltx_td ltx_nopad_r ltx_align_left">Pick&amp;Place<sub id="S4.T1.1.1.6.1.1" class="ltx_sub">2</sub>
</td>
<td id="S4.T1.1.1.6.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.6.3" class="ltx_td ltx_align_center">21.7</td>
<td id="S4.T1.1.1.6.4" class="ltx_td ltx_align_center">16.0</td>
<td id="S4.T1.1.1.6.5" class="ltx_td ltx_align_center ltx_border_r">10.0</td>
<td id="S4.T1.1.1.6.6" class="ltx_td ltx_align_center">20.9</td>
<td id="S4.T1.1.1.6.7" class="ltx_td ltx_align_center">53.3</td>
<td id="S4.T1.1.1.6.8" class="ltx_td ltx_align_center ltx_border_r">16.4</td>
<td id="S4.T1.1.1.6.9" class="ltx_td ltx_align_center">55.0</td>
<td id="S4.T1.1.1.6.10" class="ltx_td ltx_align_center ltx_border_r"><span id="S4.T1.1.1.6.10.1" class="ltx_text ltx_font_bold">100.0</span></td>
<td id="S4.T1.1.1.6.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.6.11.1" class="ltx_text ltx_font_bold">100.0</span></td>
</tr>
<tr id="S4.T1.1.1.7" class="ltx_tr">
<td id="S4.T1.1.1.7.1" class="ltx_td ltx_nopad_r ltx_align_left">Chemistry<sub id="S4.T1.1.1.7.1.1" class="ltx_sub">1</sub>
</td>
<td id="S4.T1.1.1.7.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.7.3" class="ltx_td ltx_align_center">15.8</td>
<td id="S4.T1.1.1.7.4" class="ltx_td ltx_align_center">17.0</td>
<td id="S4.T1.1.1.7.5" class="ltx_td ltx_align_center ltx_border_r">3.0</td>
<td id="S4.T1.1.1.7.6" class="ltx_td ltx_align_center">47.8</td>
<td id="S4.T1.1.1.7.7" class="ltx_td ltx_align_center">51.0</td>
<td id="S4.T1.1.1.7.8" class="ltx_td ltx_align_center ltx_border_r"><span id="S4.T1.1.1.7.8.1" class="ltx_text ltx_font_bold">70.4</span></td>
<td id="S4.T1.1.1.7.9" class="ltx_td ltx_align_center">44.5</td>
<td id="S4.T1.1.1.7.10" class="ltx_td ltx_align_center ltx_border_r">42.2</td>
<td id="S4.T1.1.1.7.11" class="ltx_td ltx_nopad_r ltx_align_center">51.7</td>
</tr>
<tr id="S4.T1.1.1.8" class="ltx_tr">
<td id="S4.T1.1.1.8.1" class="ltx_td ltx_nopad_r ltx_align_left">Chemistry<sub id="S4.T1.1.1.8.1.1" class="ltx_sub">2</sub>
</td>
<td id="S4.T1.1.1.8.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.8.3" class="ltx_td ltx_align_center">26.7</td>
<td id="S4.T1.1.1.8.4" class="ltx_td ltx_align_center">19.0</td>
<td id="S4.T1.1.1.8.5" class="ltx_td ltx_align_center ltx_border_r">6.0</td>
<td id="S4.T1.1.1.8.6" class="ltx_td ltx_align_center">39.3</td>
<td id="S4.T1.1.1.8.7" class="ltx_td ltx_align_center">58.9</td>
<td id="S4.T1.1.1.8.8" class="ltx_td ltx_align_center ltx_border_r">70.7</td>
<td id="S4.T1.1.1.8.9" class="ltx_td ltx_align_center">56.7</td>
<td id="S4.T1.1.1.8.10" class="ltx_td ltx_align_center ltx_border_r">85.6</td>
<td id="S4.T1.1.1.8.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.8.11.1" class="ltx_text ltx_font_bold">93.3</span></td>
</tr>
<tr id="S4.T1.1.1.9" class="ltx_tr">
<td id="S4.T1.1.1.9.1" class="ltx_td ltx_nopad_r ltx_align_left">Lifespan<sub id="S4.T1.1.1.9.1.1" class="ltx_sub">1</sub>
</td>
<td id="S4.T1.1.1.9.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.9.3" class="ltx_td ltx_align_center">50.0</td>
<td id="S4.T1.1.1.9.4" class="ltx_td ltx_align_center">43.0</td>
<td id="S4.T1.1.1.9.5" class="ltx_td ltx_align_center ltx_border_r">6.0</td>
<td id="S4.T1.1.1.9.6" class="ltx_td ltx_align_center">80.0</td>
<td id="S4.T1.1.1.9.7" class="ltx_td ltx_align_center">60.0</td>
<td id="S4.T1.1.1.9.8" class="ltx_td ltx_align_center ltx_border_r"><span id="S4.T1.1.1.9.8.1" class="ltx_text ltx_font_bold">100.0</span></td>
<td id="S4.T1.1.1.9.9" class="ltx_td ltx_align_center">85.0</td>
<td id="S4.T1.1.1.9.10" class="ltx_td ltx_align_center ltx_border_r">65.0</td>
<td id="S4.T1.1.1.9.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.9.11.1" class="ltx_text ltx_font_bold">100.0</span></td>
</tr>
<tr id="S4.T1.1.1.10" class="ltx_tr">
<td id="S4.T1.1.1.10.1" class="ltx_td ltx_nopad_r ltx_align_left">Lifespan<sub id="S4.T1.1.1.10.1.1" class="ltx_sub">2</sub>
</td>
<td id="S4.T1.1.1.10.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.10.3" class="ltx_td ltx_align_center">50.0</td>
<td id="S4.T1.1.1.10.4" class="ltx_td ltx_align_center">32.0</td>
<td id="S4.T1.1.1.10.5" class="ltx_td ltx_align_center ltx_border_r">10.0</td>
<td id="S4.T1.1.1.10.6" class="ltx_td ltx_align_center">67.5</td>
<td id="S4.T1.1.1.10.7" class="ltx_td ltx_align_center">67.5</td>
<td id="S4.T1.1.1.10.8" class="ltx_td ltx_align_center ltx_border_r">84.4</td>
<td id="S4.T1.1.1.10.9" class="ltx_td ltx_align_center">70.0</td>
<td id="S4.T1.1.1.10.10" class="ltx_td ltx_align_center ltx_border_r">75.0</td>
<td id="S4.T1.1.1.10.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.10.11.1" class="ltx_text ltx_font_bold">90.0</span></td>
</tr>
<tr id="S4.T1.1.1.11" class="ltx_tr">
<td id="S4.T1.1.1.11.1" class="ltx_td ltx_nopad_r ltx_align_left">Biology<sub id="S4.T1.1.1.11.1.1" class="ltx_sub">1</sub>
</td>
<td id="S4.T1.1.1.11.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="S4.T1.1.1.11.3" class="ltx_td ltx_align_center">8.0</td>
<td id="S4.T1.1.1.11.4" class="ltx_td ltx_align_center">10.0</td>
<td id="S4.T1.1.1.11.5" class="ltx_td ltx_align_center ltx_border_r">0.0</td>
<td id="S4.T1.1.1.11.6" class="ltx_td ltx_align_center">16.0</td>
<td id="S4.T1.1.1.11.7" class="ltx_td ltx_align_center">8.0</td>
<td id="S4.T1.1.1.11.8" class="ltx_td ltx_align_center ltx_border_r">8.0</td>
<td id="S4.T1.1.1.11.9" class="ltx_td ltx_align_center">10.0</td>
<td id="S4.T1.1.1.11.10" class="ltx_td ltx_align_center ltx_border_r">32.0</td>
<td id="S4.T1.1.1.11.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.11.11.1" class="ltx_text ltx_font_bold">32.0</span></td>
</tr>
<tr id="S4.T1.1.1.12" class="ltx_tr">
<td id="S4.T1.1.1.12.1" class="ltx_td ltx_nopad_r ltx_align_left">Boil</td>
<td id="S4.T1.1.1.12.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.12.3" class="ltx_td ltx_align_center">3.5</td>
<td id="S4.T1.1.1.12.4" class="ltx_td ltx_align_center">0.0</td>
<td id="S4.T1.1.1.12.5" class="ltx_td ltx_align_center ltx_border_r">0.0</td>
<td id="S4.T1.1.1.12.6" class="ltx_td ltx_align_center"><span id="S4.T1.1.1.12.6.1" class="ltx_text ltx_font_bold">33.1</span></td>
<td id="S4.T1.1.1.12.7" class="ltx_td ltx_align_center">3.5</td>
<td id="S4.T1.1.1.12.8" class="ltx_td ltx_align_center ltx_border_r">4.2</td>
<td id="S4.T1.1.1.12.9" class="ltx_td ltx_align_center">7.0</td>
<td id="S4.T1.1.1.12.10" class="ltx_td ltx_align_center ltx_border_r">4.4</td>
<td id="S4.T1.1.1.12.11" class="ltx_td ltx_nopad_r ltx_align_center">16.3</td>
</tr>
<tr id="S4.T1.1.1.13" class="ltx_tr">
<td id="S4.T1.1.1.13.1" class="ltx_td ltx_nopad_r ltx_align_left">Freeze</td>
<td id="S4.T1.1.1.13.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.13.3" class="ltx_td ltx_align_center">0.0</td>
<td id="S4.T1.1.1.13.4" class="ltx_td ltx_align_center">4.0</td>
<td id="S4.T1.1.1.13.5" class="ltx_td ltx_align_center ltx_border_r">0.0</td>
<td id="S4.T1.1.1.13.6" class="ltx_td ltx_align_center">3.9</td>
<td id="S4.T1.1.1.13.7" class="ltx_td ltx_align_center">7.8</td>
<td id="S4.T1.1.1.13.8" class="ltx_td ltx_align_center ltx_border_r">7.8</td>
<td id="S4.T1.1.1.13.9" class="ltx_td ltx_align_center"><span id="S4.T1.1.1.13.9.1" class="ltx_text ltx_font_bold">10.0</span></td>
<td id="S4.T1.1.1.13.10" class="ltx_td ltx_align_center ltx_border_r">8.9</td>
<td id="S4.T1.1.1.13.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.13.11.1" class="ltx_text ltx_font_bold">10.0</span></td>
</tr>
<tr id="S4.T1.1.1.14" class="ltx_tr">
<td id="S4.T1.1.1.14.1" class="ltx_td ltx_nopad_r ltx_align_left">GrowPlant</td>
<td id="S4.T1.1.1.14.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.14.3" class="ltx_td ltx_align_center">8.0</td>
<td id="S4.T1.1.1.14.4" class="ltx_td ltx_align_center">6.0</td>
<td id="S4.T1.1.1.14.5" class="ltx_td ltx_align_center ltx_border_r">2.0</td>
<td id="S4.T1.1.1.14.6" class="ltx_td ltx_align_center">9.9</td>
<td id="S4.T1.1.1.14.7" class="ltx_td ltx_align_center">9.1</td>
<td id="S4.T1.1.1.14.8" class="ltx_td ltx_align_center ltx_border_r">7.3</td>
<td id="S4.T1.1.1.14.9" class="ltx_td ltx_align_center">10.2</td>
<td id="S4.T1.1.1.14.10" class="ltx_td ltx_align_center ltx_border_r">10.9</td>
<td id="S4.T1.1.1.14.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.14.11.1" class="ltx_text ltx_font_bold">11.2</span></td>
</tr>
<tr id="S4.T1.1.1.15" class="ltx_tr">
<td id="S4.T1.1.1.15.1" class="ltx_td ltx_nopad_r ltx_align_left">GrowFruit</td>
<td id="S4.T1.1.1.15.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.15.3" class="ltx_td ltx_align_center">14.3</td>
<td id="S4.T1.1.1.15.4" class="ltx_td ltx_align_center">11.0</td>
<td id="S4.T1.1.1.15.5" class="ltx_td ltx_align_center ltx_border_r">4.0</td>
<td id="S4.T1.1.1.15.6" class="ltx_td ltx_align_center">13.9</td>
<td id="S4.T1.1.1.15.7" class="ltx_td ltx_align_center">18.6</td>
<td id="S4.T1.1.1.15.8" class="ltx_td ltx_align_center ltx_border_r">13.0</td>
<td id="S4.T1.1.1.15.9" class="ltx_td ltx_align_center">35.9</td>
<td id="S4.T1.1.1.15.10" class="ltx_td ltx_align_center ltx_border_r">70.8</td>
<td id="S4.T1.1.1.15.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.15.11.1" class="ltx_text ltx_font_bold">94.5</span></td>
</tr>
<tr id="S4.T1.1.1.16" class="ltx_tr">
<td id="S4.T1.1.1.16.1" class="ltx_td ltx_nopad_r ltx_align_left">Biology<sub id="S4.T1.1.1.16.1.1" class="ltx_sub">2</sub>
</td>
<td id="S4.T1.1.1.16.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.16.3" class="ltx_td ltx_align_center">21.0</td>
<td id="S4.T1.1.1.16.4" class="ltx_td ltx_align_center">5.0</td>
<td id="S4.T1.1.1.16.5" class="ltx_td ltx_align_center ltx_border_r">4.0</td>
<td id="S4.T1.1.1.16.6" class="ltx_td ltx_align_center">20.9</td>
<td id="S4.T1.1.1.16.7" class="ltx_td ltx_align_center">27.7</td>
<td id="S4.T1.1.1.16.8" class="ltx_td ltx_align_center ltx_border_r">2.6</td>
<td id="S4.T1.1.1.16.9" class="ltx_td ltx_align_center">70.0</td>
<td id="S4.T1.1.1.16.10" class="ltx_td ltx_align_center ltx_border_r">42.8</td>
<td id="S4.T1.1.1.16.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.16.11.1" class="ltx_text ltx_font_bold">85.6</span></td>
</tr>
<tr id="S4.T1.1.1.17" class="ltx_tr">
<td id="S4.T1.1.1.17.1" class="ltx_td ltx_nopad_r ltx_align_left">Force</td>
<td id="S4.T1.1.1.17.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.17.3" class="ltx_td ltx_align_center">10.0</td>
<td id="S4.T1.1.1.17.4" class="ltx_td ltx_align_center">4.0</td>
<td id="S4.T1.1.1.17.5" class="ltx_td ltx_align_center ltx_border_r">0.0</td>
<td id="S4.T1.1.1.17.6" class="ltx_td ltx_align_center">21.9</td>
<td id="S4.T1.1.1.17.7" class="ltx_td ltx_align_center">40.5</td>
<td id="S4.T1.1.1.17.8" class="ltx_td ltx_align_center ltx_border_r">50.6</td>
<td id="S4.T1.1.1.17.9" class="ltx_td ltx_align_center">53.5</td>
<td id="S4.T1.1.1.17.10" class="ltx_td ltx_align_center ltx_border_r">70.0</td>
<td id="S4.T1.1.1.17.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.17.11.1" class="ltx_text ltx_font_bold">100.0</span></td>
</tr>
<tr id="S4.T1.1.1.18" class="ltx_tr">
<td id="S4.T1.1.1.18.1" class="ltx_td ltx_nopad_r ltx_align_left">Friction</td>
<td id="S4.T1.1.1.18.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.18.3" class="ltx_td ltx_align_center">10.0</td>
<td id="S4.T1.1.1.18.4" class="ltx_td ltx_align_center">4.0</td>
<td id="S4.T1.1.1.18.5" class="ltx_td ltx_align_center ltx_border_r">3.0</td>
<td id="S4.T1.1.1.18.6" class="ltx_td ltx_align_center">32.3</td>
<td id="S4.T1.1.1.18.7" class="ltx_td ltx_align_center">44.0</td>
<td id="S4.T1.1.1.18.8" class="ltx_td ltx_align_center ltx_border_r"><span id="S4.T1.1.1.18.8.1" class="ltx_text ltx_font_bold">100.0</span></td>
<td id="S4.T1.1.1.18.9" class="ltx_td ltx_align_center">56.5</td>
<td id="S4.T1.1.1.18.10" class="ltx_td ltx_align_center ltx_border_r">70.0</td>
<td id="S4.T1.1.1.18.11" class="ltx_td ltx_nopad_r ltx_align_center">94.0</td>
</tr>
<tr id="S4.T1.1.1.19" class="ltx_tr">
<td id="S4.T1.1.1.19.1" class="ltx_td ltx_nopad_r ltx_align_left">Genetics<sub id="S4.T1.1.1.19.1.1" class="ltx_sub">1</sub>
</td>
<td id="S4.T1.1.1.19.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.19.3" class="ltx_td ltx_align_center">16.8</td>
<td id="S4.T1.1.1.19.4" class="ltx_td ltx_align_center">11.0</td>
<td id="S4.T1.1.1.19.5" class="ltx_td ltx_align_center ltx_border_r">2.0</td>
<td id="S4.T1.1.1.19.6" class="ltx_td ltx_align_center">67.5</td>
<td id="S4.T1.1.1.19.7" class="ltx_td ltx_align_center">25.7</td>
<td id="S4.T1.1.1.19.8" class="ltx_td ltx_align_center ltx_border_r">50.9</td>
<td id="S4.T1.1.1.19.9" class="ltx_td ltx_align_center">77.4</td>
<td id="S4.T1.1.1.19.10" class="ltx_td ltx_align_center ltx_border_r">84.5</td>
<td id="S4.T1.1.1.19.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.19.11.1" class="ltx_text ltx_font_bold">100.0</span></td>
</tr>
<tr id="S4.T1.1.1.20" class="ltx_tr">
<td id="S4.T1.1.1.20.1" class="ltx_td ltx_nopad_r ltx_align_left">Genetics<sub id="S4.T1.1.1.20.1.1" class="ltx_sub">2</sub>
</td>
<td id="S4.T1.1.1.20.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="S4.T1.1.1.20.3" class="ltx_td ltx_align_center">17.0</td>
<td id="S4.T1.1.1.20.4" class="ltx_td ltx_align_center">11.0</td>
<td id="S4.T1.1.1.20.5" class="ltx_td ltx_align_center ltx_border_r">2.0</td>
<td id="S4.T1.1.1.20.6" class="ltx_td ltx_align_center">59.5</td>
<td id="S4.T1.1.1.20.7" class="ltx_td ltx_align_center">16.8</td>
<td id="S4.T1.1.1.20.8" class="ltx_td ltx_align_center ltx_border_r">23.7</td>
<td id="S4.T1.1.1.20.9" class="ltx_td ltx_align_center">62.3</td>
<td id="S4.T1.1.1.20.10" class="ltx_td ltx_align_center ltx_border_r">61.4</td>
<td id="S4.T1.1.1.20.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.20.11.1" class="ltx_text ltx_font_bold">100.0</span></td>
</tr>
<tr id="S4.T1.1.1.21" class="ltx_tr">
<td id="S4.T1.1.1.21.1" class="ltx_td ltx_nopad_r ltx_border_t"></td>
<td id="S4.T1.1.1.21.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr ltx_border_t"><span id="S4.T1.1.1.21.2.1" class="ltx_text ltx_font_bold">S</span></td>
<td id="S4.T1.1.1.21.3" class="ltx_td ltx_align_center ltx_border_t">22.1</td>
<td id="S4.T1.1.1.21.4" class="ltx_td ltx_align_center ltx_border_t">19.1</td>
<td id="S4.T1.1.1.21.5" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">5.2</td>
<td id="S4.T1.1.1.21.6" class="ltx_td ltx_align_center ltx_border_t">36.5</td>
<td id="S4.T1.1.1.21.7" class="ltx_td ltx_align_center ltx_border_t">37.6</td>
<td id="S4.T1.1.1.21.8" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">49.9</td>
<td id="S4.T1.1.1.21.9" class="ltx_td ltx_align_center ltx_border_t">54.7</td>
<td id="S4.T1.1.1.21.10" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">58.3</td>
<td id="S4.T1.1.1.21.11" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t"><span id="S4.T1.1.1.21.11.1" class="ltx_text ltx_font_bold">71.0</span></td>
</tr>
<tr id="S4.T1.1.1.22" class="ltx_tr">
<td id="S4.T1.1.1.22.1" class="ltx_td ltx_nopad_r"></td>
<td id="S4.T1.1.1.22.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr"><span id="S4.T1.1.1.22.2.1" class="ltx_text ltx_font_bold">L</span></td>
<td id="S4.T1.1.1.22.3" class="ltx_td ltx_align_center">11.2</td>
<td id="S4.T1.1.1.22.4" class="ltx_td ltx_align_center">6.2</td>
<td id="S4.T1.1.1.22.5" class="ltx_td ltx_align_center ltx_border_r">1.9</td>
<td id="S4.T1.1.1.22.6" class="ltx_td ltx_align_center">29.2</td>
<td id="S4.T1.1.1.22.7" class="ltx_td ltx_align_center">21.5</td>
<td id="S4.T1.1.1.22.8" class="ltx_td ltx_align_center ltx_border_r">28.9</td>
<td id="S4.T1.1.1.22.9" class="ltx_td ltx_align_center">42.5</td>
<td id="S4.T1.1.1.22.10" class="ltx_td ltx_align_center ltx_border_r">47.1</td>
<td id="S4.T1.1.1.22.11" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.T1.1.1.22.11.1" class="ltx_text ltx_font_bold">68.0</span></td>
</tr>
<tr id="S4.T1.1.1.23" class="ltx_tr">
<td id="S4.T1.1.1.23.1" class="ltx_td ltx_nopad_r ltx_border_bb"></td>
<td id="S4.T1.1.1.23.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_bb ltx_border_rr"><span id="S4.T1.1.1.23.2.1" class="ltx_text ltx_font_bold">All</span></td>
<td id="S4.T1.1.1.23.3" class="ltx_td ltx_align_center ltx_border_bb">16.7</td>
<td id="S4.T1.1.1.23.4" class="ltx_td ltx_align_center ltx_border_bb">12.7</td>
<td id="S4.T1.1.1.23.5" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">3.6</td>
<td id="S4.T1.1.1.23.6" class="ltx_td ltx_align_center ltx_border_bb">32.9</td>
<td id="S4.T1.1.1.23.7" class="ltx_td ltx_align_center ltx_border_bb">29.6</td>
<td id="S4.T1.1.1.23.8" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">39.4</td>
<td id="S4.T1.1.1.23.9" class="ltx_td ltx_align_center ltx_border_bb">48.6</td>
<td id="S4.T1.1.1.23.10" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">52.7</td>
<td id="S4.T1.1.1.23.11" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_bb"><span id="S4.T1.1.1.23.11.1" class="ltx_text ltx_font_bold">69.5</span></td>
</tr>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 1: </span>Comparing <span id="S4.T1.4.1" class="ltx_text ltx_font_smallcaps">CLIN</span> with baselines for <span id="S4.T1.5.2" class="ltx_text ltx_font_bold">generalization across unseen environments</span> </figcaption>
</figure>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span><span id="S4.SS2.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> Outperforms SOTA, Generalizing to Novel Environments and Tasks</h3>

<div id="S4.SS2.p1" class="ltx_para ltx_noindent">
<p id="S4.SS2.p1.1" class="ltx_p"><span id="S4.SS2.p1.1.1" class="ltx_text ltx_font_bold">Generalizing to new environments.</span>    <a href="#S4.T1" title="In 4.1 CLIN Exhibits Rapid Task Adaptation ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">1</span></a> compares <span id="S4.SS2.p1.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> with baselines that learn from training environmental variants for a task to improve its performance in a novel environment <span id="footnote3" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">3</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">3</sup><span class="ltx_tag ltx_tag_note">3</span>Baseline numbers are derived from
Table 1 in <cite class="ltx_cite ltx_citemacro_cite">Lin et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite></span></span></span>. Language agents (including <span id="S4.SS2.p1.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span>) that use NL feedback from the ScienceWorld (e.g., “Door to the kitchen is closed”) perform significantly better compared to RL methods that purely rely on (sparse) numeric rewards from the environment to learn a policy.
We observe a positive generalization effect in <span id="S4.SS2.p1.1.4" class="ltx_text ltx_font_smallcaps">Gen-Env</span> (average 4 point gain) compared to <span id="S4.SS2.p1.1.5" class="ltx_text ltx_font_smallcaps">Base</span> where <span id="S4.SS2.p1.1.6" class="ltx_text ltx_font_smallcaps">CLIN</span> tries to solve the tasks zero-shot. With a strong <span id="S4.SS2.p1.1.7" class="ltx_text ltx_font_smallcaps">Base</span> performance, <span id="S4.SS2.p1.1.8" class="ltx_text ltx_font_smallcaps">CLIN</span> beats all baselines in generalization performance. Furthermore, in <span id="S4.SS2.p1.1.9" class="ltx_text ltx_font_smallcaps">G+A</span>, <span id="S4.SS2.p1.1.10" class="ltx_text ltx_font_smallcaps">CLIN</span> shows a substantial 16 additional improvement, beating the SOTA reflective agent by 23 points.
Figure <a href="#S4.F5" title="Figure 5 ‣ 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>a additionally shows trend of improvement compared to when <span id="S4.SS2.p1.1.11" class="ltx_text ltx_font_smallcaps">CLIN</span> does not start with a meta-memory. Meta-memory helps <span id="S4.SS2.p1.1.12" class="ltx_text ltx_font_smallcaps">CLIN</span> with a stronger start than <span id="S4.SS2.p1.1.13" class="ltx_text ltx_font_smallcaps">Base</span> (52.7 vs. 48.6), with a continued gain in scores till the end of Trial-4 (<span id="S4.SS2.p1.1.14" class="ltx_text ltx_font_smallcaps">G+A</span>: 69.5 vs. <span id="S4.SS2.p1.1.15" class="ltx_text ltx_font_smallcaps">Adapt</span>: 62.2). The stronger start for <span id="S4.SS2.p1.1.16" class="ltx_text ltx_font_smallcaps">CLIN</span> with meta-memory also results in fewer steps to solve a task.
Unlike imitation learning-based agents, TDT <cite class="ltx_cite ltx_citemacro_cite">Wang et al. (<a href="#bib.bib23" title="" class="ltx_ref">2022</a>)</cite> and SwiftSage <cite class="ltx_cite ltx_citemacro_cite">Lin et al. (<a href="#bib.bib14" title="" class="ltx_ref">2023</a>)</cite>, <span id="S4.SS2.p1.1.17" class="ltx_text ltx_font_smallcaps">CLIN</span> (and most baselines) does not use any gold trajectories. Refining its memory only from self-generated trajectories, <span id="S4.SS2.p1.1.18" class="ltx_text ltx_font_smallcaps">CLIN</span> outperforms TDT on all 18 tasks and SwiftSage on 8/18 (mostly long) tasks.</p>
</div>
<div id="S4.SS2.p2" class="ltx_para ltx_noindent">
<p id="S4.SS2.p2.1" class="ltx_p"><span id="S4.SS2.p2.1.1" class="ltx_text ltx_font_bold">Generalizing to new tasks.</span>   
Mirroring trends from <span id="S4.SS2.p2.1.2" class="ltx_text ltx_font_smallcaps">Gen-Env</span>, <span id="S4.SS2.p2.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span>  demonstrates strong transfer learning to new tasks with 13-point improvement over its <span id="S4.SS2.p2.1.4" class="ltx_text ltx_font_smallcaps">Base</span> performance, being better at 38.8% of times (<a href="#S4.F5" title="In 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">5</span></a>c). The improvement attributes to critical learning about the environment (“apple juice is in the fridge”, required for both boiling and freezing it), leading to improvement in previously low-performing tasks in both <span id="S4.SS2.p2.1.5" class="ltx_text ltx_font_smallcaps">Adapt</span> and <span id="S4.SS2.p2.1.6" class="ltx_text ltx_font_smallcaps">Gen-Env</span> setups. This transfer learning in <span id="S4.SS2.p2.1.7" class="ltx_text ltx_font_smallcaps">Gen-Task</span> and <span id="S4.SS2.p2.1.8" class="ltx_text ltx_font_smallcaps">G+A</span> helps <span id="S4.SS2.p2.1.9" class="ltx_text ltx_font_smallcaps">CLIN</span> to solve the tasks with a lesser number of steps<span id="footnote4" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">4</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">4</sup><span class="ltx_tag ltx_tag_note">4</span># steps in <a href="#S4.F5" title="In 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">5</span></a>a,b are normalized between 0-1, 1 being maximum # steps allowed for a task.</span></span></span> and achieve higher rewards.</p>
</div>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>Discussion</h3>

<div id="S4.SS3.p1" class="ltx_para ltx_noindent">
<p id="S4.SS3.p1.1" class="ltx_p"><span id="S4.SS3.p1.1.1" class="ltx_text ltx_font_bold">Importance of memory structure. </span>  
<span id="S4.SS3.p1.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> extracts causal abstractions structured around ‘necessary’ and ‘does not contribute’ relations. As an ablation study, we modified our memory generator to generate free-form advice for future trials (without any constraint on their formats).
We find that the average reward drops by 6 points (in 10% cases compared to <span id="S4.SS3.p1.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span>) when using the unstructured memory, indicating the usefulness of causal abstractions, as shown in <a href="#S4.F5" title="In 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">5</span></a>d.</p>
</div>
<div id="S4.SS3.p2" class="ltx_para ltx_noindent">
<p id="S4.SS3.p2.1" class="ltx_p"><span id="S4.SS3.p2.1.1" class="ltx_text ltx_font_bold">Superior <span id="S4.SS3.p2.1.1.1" class="ltx_text ltx_font_smallcaps">Base</span> performance.</span>   <a href="#S4.F4" title="In 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">4</span></a> depicts a superior <span id="S4.SS3.p2.1.2" class="ltx_text ltx_font_smallcaps">Base</span> performance for <span id="S4.SS3.p2.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> than the final performance of both ReAct and Reflexion despite using the same underlying LLM (here, <span id="S4.SS3.p2.1.4" class="ltx_text ltx_font_typewriter">gpt-4</span>). We find if we ablate for the controller module in <span id="S4.SS3.p2.1.5" class="ltx_text ltx_font_smallcaps">CLIN</span>, responsible for generating a goal before outputting the next action, <span id="S4.SS3.p2.1.6" class="ltx_text ltx_font_smallcaps">CLIN</span>’s <span id="S4.SS3.p2.1.7" class="ltx_text ltx_font_smallcaps">Base</span> performance drops in 44% cases. With an 18 point drop in average reward (see <a href="#S4.F5" title="In 4.3 Discussion ‣ 4 Results and Analysis ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">5</span></a>d), Abl-Contoller-<span id="S4.SS3.p2.1.8" class="ltx_text ltx_font_smallcaps">Base</span> version of <span id="S4.SS3.p2.1.9" class="ltx_text ltx_font_smallcaps">CLIN</span> becomes equivalent to ReAct, the base agent for Reflexion, demonstrating the importance of controller even in <span id="S4.SS3.p2.1.10" class="ltx_text ltx_font_smallcaps">Base</span> setup.</p>
</div>
<div id="S4.SS3.p3" class="ltx_para ltx_noindent">
<p id="S4.SS3.p3.1" class="ltx_p"><span id="S4.SS3.p3.1.1" class="ltx_text ltx_font_bold">A qualitative example.</span>   
<a href="#S3.F3" title="In 3.1 CLIN: A Generative Agent Continually Learning from INteractions ‣ 3 Approach ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figure</span> <span class="ltx_text ltx_ref_tag">3</span></a> depicts how memory items get refined during task adaptation and for generalization for a task <span id="S4.SS3.p3.1.2" class="ltx_text ltx_font_italic">boil</span>. <span id="S4.SS3.p3.1.3" class="ltx_text ltx_font_typewriter">Env2</span> has a working stove, whereas in <span id="S4.SS3.p3.1.4" class="ltx_text ltx_font_typewriter">Env1</span>, the stove is broken, but a lighter is available as an alternative. With a number of trials in these environments, <span id="S4.SS3.p3.1.5" class="ltx_text ltx_font_smallcaps">CLIN</span> learns how to use these two devices to generate heat. In an unseen environment with a broken stove, <span id="S4.SS3.p3.1.6" class="ltx_text ltx_font_smallcaps">CLIN</span> quickly receives a positive reward by using a lighter to heat a substance. While insights within an episode are often specific, e.g., “Using the <span id="S4.SS3.p3.1.7" class="ltx_text ltx_font_italic">lighter</span> on the metal pot should be necessary to heat the <span id="S4.SS3.p3.1.8" class="ltx_text ltx_font_italic">water</span> in the pot”, <span id="S4.SS3.p3.1.9" class="ltx_text ltx_font_smallcaps">CLIN</span> compiles these insights for a new target environment (as meta-memory), e.g., “Using a <span id="S4.SS3.p3.1.10" class="ltx_text ltx_font_italic">heat source</span> (stove, lighter) on the container should be necessary to heat a <span id="S4.SS3.p3.1.11" class="ltx_text ltx_font_italic">substance</span>.” <a href="#A2" title="Appendix B Example Memories ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Appendix</span> <span class="ltx_text ltx_ref_tag">B</span></a> contains examples of memories generated during adaptation and generalization.</p>
</div>
<figure id="S4.F5" class="ltx_figure">
<div class="ltx_flex_figure">
<div class="ltx_flex_cell ltx_flex_size_2">
<div id="S4.F5.1" class="ltx_block ltx_figure_panel ltx_minipage ltx_align_middle" style="width:198.7pt;">
<img src="/html/2310.10134/assets/x5.png" id="S4.F5.1.g1" class="ltx_graphics ltx_img_square" width="274" height="285" alt="Refer to caption">
</div>
</div>
<div class="ltx_flex_cell ltx_flex_size_2">
<div id="S4.F5.6" class="ltx_block ltx_figure_panel ltx_minipage ltx_align_middle" style="width:198.7pt;">
<div id="S4.F5.3.2" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:346.9pt;height:230.2pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(78.5pt,-52.1pt) scale(1.82683999937965,1.82683999937965) ;">
<table id="S4.F5.3.2.2" class="ltx_tabular ltx_align_middle">
<tr id="S4.F5.3.2.2.3" class="ltx_tr">
<td id="S4.F5.3.2.2.3.1" class="ltx_td ltx_align_center" colspan="5">
<span id="S4.F5.3.2.2.3.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span><span id="S4.F5.3.2.2.3.1.2" class="ltx_text ltx_font_bold">’s </span><span id="S4.F5.3.2.2.3.1.3" class="ltx_text ltx_font_smallcaps">Gen-Task</span><span id="S4.F5.3.2.2.3.1.4" class="ltx_text ltx_font_bold"> improvements</span>
</td>
</tr>
<tr id="S4.F5.3.2.2.4" class="ltx_tr">
<td id="S4.F5.3.2.2.4.1" class="ltx_td ltx_align_left ltx_border_rr ltx_border_tt"><span id="S4.F5.3.2.2.4.1.1" class="ltx_text ltx_font_bold">Type</span></td>
<td id="S4.F5.3.2.2.4.2" class="ltx_td ltx_align_center ltx_border_r ltx_border_tt" colspan="2"><span id="S4.F5.3.2.2.4.2.1" class="ltx_text ltx_font_smallcaps">Gen-Task</span></td>
<td id="S4.F5.3.2.2.4.3" class="ltx_td ltx_align_center ltx_border_tt" colspan="2"><span id="S4.F5.3.2.2.4.3.1" class="ltx_text ltx_font_smallcaps">G + A</span></td>
</tr>
<tr id="S4.F5.3.2.2.2" class="ltx_tr">
<td id="S4.F5.3.2.2.2.3" class="ltx_td ltx_border_rr"></td>
<td id="S4.F5.2.1.1.1.1" class="ltx_td ltx_align_center">
<math id="S4.F5.2.1.1.1.1.m1.1" class="ltx_Math" alttext="\Delta" display="inline"><semantics id="S4.F5.2.1.1.1.1.m1.1a"><mi mathvariant="normal" id="S4.F5.2.1.1.1.1.m1.1.1" xref="S4.F5.2.1.1.1.1.m1.1.1.cmml">Δ</mi><annotation-xml encoding="MathML-Content" id="S4.F5.2.1.1.1.1.m1.1b"><ci id="S4.F5.2.1.1.1.1.m1.1.1.cmml" xref="S4.F5.2.1.1.1.1.m1.1.1">Δ</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.2.1.1.1.1.m1.1c">\Delta</annotation></semantics></math><span id="S4.F5.2.1.1.1.1.1" class="ltx_text ltx_font_bold">avg</span>
</td>
<td id="S4.F5.3.2.2.2.4" class="ltx_td ltx_align_center ltx_border_r"><span id="S4.F5.3.2.2.2.4.1" class="ltx_text ltx_font_bold">%ep.</span></td>
<td id="S4.F5.3.2.2.2.2" class="ltx_td ltx_align_center">
<math id="S4.F5.3.2.2.2.2.m1.1" class="ltx_Math" alttext="\Delta" display="inline"><semantics id="S4.F5.3.2.2.2.2.m1.1a"><mi mathvariant="normal" id="S4.F5.3.2.2.2.2.m1.1.1" xref="S4.F5.3.2.2.2.2.m1.1.1.cmml">Δ</mi><annotation-xml encoding="MathML-Content" id="S4.F5.3.2.2.2.2.m1.1b"><ci id="S4.F5.3.2.2.2.2.m1.1.1.cmml" xref="S4.F5.3.2.2.2.2.m1.1.1">Δ</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.3.2.2.2.2.m1.1c">\Delta</annotation></semantics></math><span id="S4.F5.3.2.2.2.2.1" class="ltx_text ltx_font_bold">avg</span>
</td>
<td id="S4.F5.3.2.2.2.5" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.F5.3.2.2.2.5.1" class="ltx_text ltx_font_bold">%ep.</span></td>
</tr>
<tr id="S4.F5.3.2.2.5" class="ltx_tr">
<td id="S4.F5.3.2.2.5.1" class="ltx_td ltx_border_rr"></td>
<td id="S4.F5.3.2.2.5.2" class="ltx_td ltx_align_center"><span id="S4.F5.3.2.2.5.2.1" class="ltx_text ltx_font_bold">score</span></td>
<td id="S4.F5.3.2.2.5.3" class="ltx_td ltx_align_center ltx_border_r"><span id="S4.F5.3.2.2.5.3.1" class="ltx_text ltx_font_bold">improv.</span></td>
<td id="S4.F5.3.2.2.5.4" class="ltx_td ltx_align_center"><span id="S4.F5.3.2.2.5.4.1" class="ltx_text ltx_font_bold">score</span></td>
<td id="S4.F5.3.2.2.5.5" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.F5.3.2.2.5.5.1" class="ltx_text ltx_font_bold">improv.</span></td>
</tr>
<tr id="S4.F5.3.2.2.6" class="ltx_tr">
<td id="S4.F5.3.2.2.6.1" class="ltx_td ltx_align_left ltx_border_rr ltx_border_t">S</td>
<td id="S4.F5.3.2.2.6.2" class="ltx_td ltx_align_center ltx_border_t">14.6</td>
<td id="S4.F5.3.2.2.6.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">40.0</td>
<td id="S4.F5.3.2.2.6.4" class="ltx_td ltx_align_center ltx_border_t">4.9</td>
<td id="S4.F5.3.2.2.6.5" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t">5.7</td>
</tr>
<tr id="S4.F5.3.2.2.7" class="ltx_tr">
<td id="S4.F5.3.2.2.7.1" class="ltx_td ltx_align_left ltx_border_rr">L</td>
<td id="S4.F5.3.2.2.7.2" class="ltx_td ltx_align_center">10.3</td>
<td id="S4.F5.3.2.2.7.3" class="ltx_td ltx_align_center ltx_border_r">36.7</td>
<td id="S4.F5.3.2.2.7.4" class="ltx_td ltx_align_center">9.2</td>
<td id="S4.F5.3.2.2.7.5" class="ltx_td ltx_nopad_r ltx_align_center">15.6</td>
</tr>
<tr id="S4.F5.3.2.2.8" class="ltx_tr">
<td id="S4.F5.3.2.2.8.1" class="ltx_td ltx_align_left ltx_border_bb ltx_border_rr ltx_border_t">All</td>
<td id="S4.F5.3.2.2.8.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">13.0</td>
<td id="S4.F5.3.2.2.8.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r ltx_border_t">38.8</td>
<td id="S4.F5.3.2.2.8.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t">6.5</td>
<td id="S4.F5.3.2.2.8.5" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_bb ltx_border_t">9.3</td>
</tr>
</table>
</span></div>
<p id="S4.F5.6.6" class="ltx_p ltx_align_center">(c)</p>
<div id="S4.F5.6.5" class="ltx_block ltx_minipage ltx_align_center ltx_align_middle" style="width:397.5pt;">
<div id="S4.F5.6.5.3" class="ltx_inline-block ltx_transformed_outer" style="width:390.3pt;height:180.2pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(97.7pt,-45.1pt) scale(2.0021617212355,2.0021617212355) ;">
<table id="S4.F5.6.5.3.3" class="ltx_tabular ltx_align_middle">
<tr id="S4.F5.6.5.3.3.4" class="ltx_tr">
<td id="S4.F5.6.5.3.3.4.1" class="ltx_td ltx_align_center" colspan="3">
<span id="S4.F5.6.5.3.3.4.1.1" class="ltx_text ltx_font_bold">Ablations for </span><span id="S4.F5.6.5.3.3.4.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span>
</td>
</tr>
<tr id="S4.F5.4.3.1.1.1" class="ltx_tr">
<td id="S4.F5.4.3.1.1.1.2" class="ltx_td ltx_align_left ltx_border_rr ltx_border_tt"><span id="S4.F5.4.3.1.1.1.2.1" class="ltx_text ltx_font_bold">Ablation Setup</span></td>
<td id="S4.F5.4.3.1.1.1.1" class="ltx_td ltx_align_center ltx_border_tt">
<math id="S4.F5.4.3.1.1.1.1.m1.1" class="ltx_Math" alttext="\Delta" display="inline"><semantics id="S4.F5.4.3.1.1.1.1.m1.1a"><mi mathvariant="normal" id="S4.F5.4.3.1.1.1.1.m1.1.1" xref="S4.F5.4.3.1.1.1.1.m1.1.1.cmml">Δ</mi><annotation-xml encoding="MathML-Content" id="S4.F5.4.3.1.1.1.1.m1.1b"><ci id="S4.F5.4.3.1.1.1.1.m1.1.1.cmml" xref="S4.F5.4.3.1.1.1.1.m1.1.1">Δ</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.4.3.1.1.1.1.m1.1c">\Delta</annotation></semantics></math><span id="S4.F5.4.3.1.1.1.1.1" class="ltx_text ltx_font_bold">avg</span>
</td>
<td id="S4.F5.4.3.1.1.1.3" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_tt"><span id="S4.F5.4.3.1.1.1.3.1" class="ltx_text ltx_font_bold">%ep.</span></td>
</tr>
<tr id="S4.F5.6.5.3.3.3" class="ltx_tr">
<td id="S4.F5.6.5.3.3.3.3" class="ltx_td ltx_border_rr"></td>
<td id="S4.F5.5.4.2.2.2.1" class="ltx_td ltx_align_center"><span id="S4.F5.5.4.2.2.2.1.1" class="ltx_text ltx_font_bold">score (<math id="S4.F5.5.4.2.2.2.1.1.m1.1" class="ltx_Math" alttext="\downarrow" display="inline"><semantics id="S4.F5.5.4.2.2.2.1.1.m1.1a"><mo stretchy="false" id="S4.F5.5.4.2.2.2.1.1.m1.1.1" xref="S4.F5.5.4.2.2.2.1.1.m1.1.1.cmml">↓</mo><annotation-xml encoding="MathML-Content" id="S4.F5.5.4.2.2.2.1.1.m1.1b"><ci id="S4.F5.5.4.2.2.2.1.1.m1.1.1.cmml" xref="S4.F5.5.4.2.2.2.1.1.m1.1.1">↓</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.5.4.2.2.2.1.1.m1.1c">\downarrow</annotation></semantics></math>)</span></td>
<td id="S4.F5.6.5.3.3.3.2" class="ltx_td ltx_nopad_r ltx_align_center"><span id="S4.F5.6.5.3.3.3.2.1" class="ltx_text ltx_font_bold">drop. (<math id="S4.F5.6.5.3.3.3.2.1.m1.1" class="ltx_Math" alttext="\uparrow" display="inline"><semantics id="S4.F5.6.5.3.3.3.2.1.m1.1a"><mo stretchy="false" id="S4.F5.6.5.3.3.3.2.1.m1.1.1" xref="S4.F5.6.5.3.3.3.2.1.m1.1.1.cmml">↑</mo><annotation-xml encoding="MathML-Content" id="S4.F5.6.5.3.3.3.2.1.m1.1b"><ci id="S4.F5.6.5.3.3.3.2.1.m1.1.1.cmml" xref="S4.F5.6.5.3.3.3.2.1.m1.1.1">↑</ci></annotation-xml><annotation encoding="application/x-tex" id="S4.F5.6.5.3.3.3.2.1.m1.1c">\uparrow</annotation></semantics></math>)</span></td>
</tr>
<tr id="S4.F5.6.5.3.3.5" class="ltx_tr">
<td id="S4.F5.6.5.3.3.5.1" class="ltx_td ltx_align_left ltx_border_rr ltx_border_t">Abl-Causal-Memory</td>
<td id="S4.F5.6.5.3.3.5.2" class="ltx_td ltx_align_center ltx_border_t">-6.2</td>
<td id="S4.F5.6.5.3.3.5.3" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t">10</td>
</tr>
<tr id="S4.F5.6.5.3.3.6" class="ltx_tr">
<td id="S4.F5.6.5.3.3.6.1" class="ltx_td ltx_align_left ltx_border_bb ltx_border_rr">Abl-Controller-<span id="S4.F5.6.5.3.3.6.1.1" class="ltx_text ltx_font_smallcaps">Base</span>
</td>
<td id="S4.F5.6.5.3.3.6.2" class="ltx_td ltx_align_center ltx_border_bb">-18.1</td>
<td id="S4.F5.6.5.3.3.6.3" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_bb">44.8</td>
</tr>
</table>
</span></div>
<p id="S4.F5.6.5.4" class="ltx_p ltx_align_center">(d)</p>
</div>
</div>
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 5: </span>Reward and #steps trends for <span id="S4.F5.17.1" class="ltx_text ltx_font_smallcaps">CLIN</span> in <span id="S4.F5.18.2" class="ltx_text ltx_font_bold">(a)</span> <span id="S4.F5.19.3" class="ltx_text ltx_font_smallcaps">Gen-Env</span> and <span id="S4.F5.20.4" class="ltx_text ltx_font_bold">(b)</span> <span id="S4.F5.21.5" class="ltx_text ltx_font_smallcaps">Gen-Task</span>.
<span id="S4.F5.22.6" class="ltx_text ltx_font_bold">(c)</span> % episode improvements and score change than <span id="S4.F5.23.7" class="ltx_text ltx_font_smallcaps">CLIN</span> without meta-memory (<span id="S4.F5.24.8" class="ltx_text ltx_font_smallcaps">Gen-Task</span>). <span id="S4.F5.25.9" class="ltx_text ltx_font_bold">(d)</span> <span id="S4.F5.26.10" class="ltx_text ltx_font_smallcaps">CLIN</span> ablations.
</figcaption>
</figure>
<div id="S4.SS3.p4" class="ltx_para ltx_noindent">
<p id="S4.SS3.p4.1" class="ltx_p"><span id="S4.SS3.p4.1.1" class="ltx_text ltx_font_bold">Limitation: Lack of exploration.</span>   <span id="S4.SS3.p4.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span>’s learnings are dependent on its own past experience. If <span id="S4.SS3.p4.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> never explores a location in the environment or does not perform an action, an insight related to the unobserved activity can never be generated. Hence, exploration becomes important when task-critical location or action in unknown to <span id="S4.SS3.p4.1.4" class="ltx_text ltx_font_smallcaps">CLIN</span> from past trials. For example, in task of creating an orange paint, the agent is supposed to find red and yellow paints from the art studio. However, art studio is not visible when <span id="S4.SS3.p4.1.5" class="ltx_text ltx_font_smallcaps">CLIN</span> starts from location ‘outside’. Unless the <span id="S4.SS3.p4.1.6" class="ltx_text ltx_font_smallcaps">CLIN</span> knows that there exists an art studio, it tries alternative method to create orange paints from other irrelevant objects (e.g., an orange) and remains unsuccessful. When a memory related art-studio appears from past exploration, <span id="S4.SS3.p4.1.7" class="ltx_text ltx_font_smallcaps">CLIN</span> is able to successfully complete the task. Similarly, in boil or freeze tasks, <span id="S4.SS3.p4.1.8" class="ltx_text ltx_font_smallcaps">CLIN</span> is unable to perform well which requires it to consistently measure the temperature of the substance to know its boiling/freezing point—an act it could never perform successfully in past trials resulting into less useful memory insights and subsequent lower performance in future trials.</p>
</div>
<div id="S4.SS3.p5" class="ltx_para ltx_noindent">
<p id="S4.SS3.p5.1" class="ltx_p"><span id="S4.SS3.p5.1.1" class="ltx_text ltx_font_bold">Limitation: Poor memory retrieval.</span>  
For a task of boiling gallium, <span id="S4.SS3.p5.1.2" class="ltx_text ltx_font_smallcaps">CLIN</span> is supposed to use oven/blast furnace and not a stove. In the meta-memory for boiling tasks, there are two insights regarding the act of boiling: “Activating stove should be necessary to boil a substance” and “Using an alternative heat source (e.g., oven or fire pit) may be necessary if the initial heat source is insufficient.” However, <span id="S4.SS3.p5.1.3" class="ltx_text ltx_font_smallcaps">CLIN</span> repeatedly retrieves the former and hence failing at the task despite performing other actions (e.g., finding gallium) correctly. This problem intensifies at the initial trial during generalization due to the presence of insights with varied initial conditions for them to be applied.
This can be circumvented by improved memory representation, which we leave as a future work.</p>
</div>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Conclusion</h2>

<div id="S5.p1" class="ltx_para ltx_noindent">
<p id="S5.p1.1" class="ltx_p">Our goal is a system that can continually improve over time, both while rapidly adapting to a task by multiple retries and efficiently generalizing to novel tasks and environments.
We propose <span id="S5.p1.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span>, an architecture for language agents that constructs a persistent, dynamic memory of causal abstractions, refines it over time and uses it effectively to improve its performance on future tasks, achieving state-of-the-art performance. Our work systematically evaluates a novel nonparametric learning paradigm, promising never-ending learning abilities to frozen language agents.</p>
</div>
<section id="S5.SS0.SSS0.Px1" class="ltx_paragraph">
<h4 class="ltx_title ltx_title_paragraph">Acknowledgement</h4>

<div id="S5.SS0.SSS0.Px1.p1" class="ltx_para ltx_noindent">
<p id="S5.SS0.SSS0.Px1.p1.1" class="ltx_p">We sincerely thank Aristo team members Tushar Khot, Ashish Sabharwal, Shashank Gupta, Nathaniel Weir, Kyle Richardson, Jiangjie Chen, Archiki Prasad, and other members such as Faeze Brahman, Alexander Koller at the Allen Institute of AI for their generous feedback.</p>
</div>
</section>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Adaptive-Agent-Team et al. (2023)</span>
<span class="ltx_bibblock">
Adaptive-Agent-Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal M. P.
Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang,
Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol
Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw,
Jack Parker-Holder, Shreyaan Pathak, Nicolas Perez Nieves, Nemanja Rakicevic,
Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah
York, Alexander Zacherl, and Lei M. Zhang.

</span>
<span class="ltx_bibblock">Human-timescale adaptation in an open-ended task space.

</span>
<span class="ltx_bibblock">In <em id="bib.bib1.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:255998274" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:255998274</a>.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ahn et al. (2022)</span>
<span class="ltx_bibblock">
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron
David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander
Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric
Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil Jayant
Joshi, Ryan C. Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee,
Sergey Levine, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell
Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego M Reyes, Pierre Sermanet,
Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, F. Xia,
Ted Xiao, Peng Xu, Sichun Xu, and Mengyuan Yan.

</span>
<span class="ltx_bibblock">Do as i can, not as i say: Grounding language in robotic affordances.

</span>
<span class="ltx_bibblock">In <em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">Conference on Robot Learning</em>, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:247939706" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:247939706</a>.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Aineto et al. (2018)</span>
<span class="ltx_bibblock">
Diego Aineto, Sergio Jiménez, and Eva Onaindía.

</span>
<span class="ltx_bibblock">Learning strips action models with classical planning.

</span>
<span class="ltx_bibblock">In <em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">International Conference on Automated Planning and
Scheduling</em>, 2018.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:49405691" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:49405691</a>.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ammanabrolu &amp; Hausknecht (2020)</span>
<span class="ltx_bibblock">
Prithviraj Ammanabrolu and Matthew J. Hausknecht.

</span>
<span class="ltx_bibblock">Graph constrained reinforcement learning for natural language action
spaces.

</span>
<span class="ltx_bibblock">In <em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">ICLR</em>, 2020.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ammanabrolu et al. (2020)</span>
<span class="ltx_bibblock">
Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim
Rocktaschel, and Jason Weston.

</span>
<span class="ltx_bibblock">How to motivate your dragon: Teaching goal-driven agents to speak and
act in fantasy worlds.

</span>
<span class="ltx_bibblock">In <em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">North American Chapter of the Association for Computational
Linguistics</em>, 2020.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:222125301" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:222125301</a>.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Andrychowicz et al. (2017)</span>
<span class="ltx_bibblock">
Marcin Andrychowicz, Dwight Crow, Alex Ray, Jonas Schneider, Rachel Fong, Peter
Welinder, Bob McGrew, Joshua Tobin, P. Abbeel, and Wojciech Zaremba.

</span>
<span class="ltx_bibblock">Hindsight experience replay.

</span>
<span class="ltx_bibblock"><em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/1707.01495, 2017.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:3532908" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:3532908</a>.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Arora et al. (2018)</span>
<span class="ltx_bibblock">
Ankuj Arora, Humbert Fiorino, Damien Pellier, Marc Métivier, and Sylvie
Pesty.

</span>
<span class="ltx_bibblock">A review of learning planning action models.

</span>
<span class="ltx_bibblock"><em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">The Knowledge Engineering Review</em>, 33, 2018.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:56483203" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:56483203</a>.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2021)</span>
<span class="ltx_bibblock">
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael
Laskin, P. Abbeel, A. Srinivas, and Igor Mordatch.

</span>
<span class="ltx_bibblock">Decision transformer: Reinforcement learning via sequence modeling.

</span>
<span class="ltx_bibblock">In <em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">Neural Information Processing Systems</em>, 2021.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:235294299" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:235294299</a>.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chung et al. (2022)</span>
<span class="ltx_bibblock">
Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph, Yi Tay, William Fedus, Eric
Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson,
Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha
Chowdhery, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Wei Yu, Vincent
Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed Huai hsin
Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason
Wei.

</span>
<span class="ltx_bibblock">Scaling instruction-finetuned language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/2210.11416, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:253018554" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:253018554</a>.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dalvi et al. (2022)</span>
<span class="ltx_bibblock">
Bhavana Dalvi, Oyvind Tafjord, and Peter Clark.

</span>
<span class="ltx_bibblock">Towards teachable reasoning systems: Using a dynamic memory of user
feedback for continual system improvement.

</span>
<span class="ltx_bibblock">In <em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">EMNLP</em>, 2022.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">He et al. (2015)</span>
<span class="ltx_bibblock">
Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, and Mari
Ostendorf.

</span>
<span class="ltx_bibblock">Deep reinforcement learning with a natural language action space.

</span>
<span class="ltx_bibblock"><em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">arXiv: Artificial Intelligence</em>, 2015.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:15986631" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:15986631</a>.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Huang et al. (2022)</span>
<span class="ltx_bibblock">
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch.

</span>
<span class="ltx_bibblock">Language models as zero-shot planners: Extracting actionable
knowledge for embodied agents.

</span>
<span class="ltx_bibblock">In <em id="bib.bib12.1.1" class="ltx_emph ltx_font_italic">International Conference on Machine Learning</em>, pp. 9118–9147. PMLR, 2022.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jiang et al. (2021)</span>
<span class="ltx_bibblock">
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Jakob Nicolaus Foerster,
Edward Grefenstette, and Tim Rocktaschel.

</span>
<span class="ltx_bibblock">Replay-guided adversarial environment design.

</span>
<span class="ltx_bibblock">In <em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">Neural Information Processing Systems</em>, 2021.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:238408352" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:238408352</a>.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Lin et al. (2023)</span>
<span class="ltx_bibblock">
Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze
Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, and Xiang Ren.

</span>
<span class="ltx_bibblock">Swiftsage: A generative agent with fast and slow thinking for complex
interactive tasks.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/2305.17390, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:258960143" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:258960143</a>.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Madaan et al. (2023)</span>
<span class="ltx_bibblock">
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah
Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean
Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and
Peter Clark.

</span>
<span class="ltx_bibblock">Self-refine: Iterative refinement with self-feedback.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/2303.17651, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:257900871" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:257900871</a>.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Mnih et al. (2013)</span>
<span class="ltx_bibblock">
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis
Antonoglou, Daan Wierstra, and Martin A. Riedmiller.

</span>
<span class="ltx_bibblock">Playing atari with deep reinforcement learning.

</span>
<span class="ltx_bibblock"><em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/1312.5602, 2013.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:15238391" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:15238391</a>.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Park et al. (2023)</span>
<span class="ltx_bibblock">
Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy
Liang, and Michael S Bernstein.

</span>
<span class="ltx_bibblock">Generative agents: Interactive simulacra of human behavior.

</span>
<span class="ltx_bibblock"><em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2304.03442</em>, 2023.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Reed et al. (2022)</span>
<span class="ltx_bibblock">
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander
Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay,
Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley D.
Edwards, Nicolas Manfred Otto Heess, Yutian Chen, Raia Hadsell, Oriol
Vinyals, Mahyar Bordbar, and Nando de Freitas.

</span>
<span class="ltx_bibblock">A generalist agent.

</span>
<span class="ltx_bibblock"><em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">Trans. Mach. Learn. Res.</em>, 2022, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:248722148" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:248722148</a>.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Reimers &amp; Gurevych (2019)</span>
<span class="ltx_bibblock">
Nils Reimers and Iryna Gurevych.

</span>
<span class="ltx_bibblock">Sentence-bert: Sentence embeddings using siamese bert-networks.

</span>
<span class="ltx_bibblock">In <em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing</em>. Association for Computational Linguistics, 11
2019.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://arxiv.org/abs/1908.10084" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://arxiv.org/abs/1908.10084</a>.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Shinn et al. (2023)</span>
<span class="ltx_bibblock">
Noah Shinn, Beck Labash, and Ashwin Gopinath.

</span>
<span class="ltx_bibblock">Reflexion: an autonomous agent with dynamic memory and
self-reflection.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2303.11366</em>, 2023.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Tandon et al. (2022)</span>
<span class="ltx_bibblock">
Niket Tandon, Aman Madaan, Peter Clark, and Yiming Yang.

</span>
<span class="ltx_bibblock">Memory-assisted prompt editing to improve GPT-3 after deployment.

</span>
<span class="ltx_bibblock">In <em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">ACL Workshop on Commonsense Representation and Reasoning
(CSRR’22)</em>, 2022.

</span>
<span class="ltx_bibblock">(also arxiv:2201.06009).

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2023)</span>
<span class="ltx_bibblock">
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu,
Linxi (Jim) Fan, and Anima Anandkumar.

</span>
<span class="ltx_bibblock">Voyager: An open-ended embodied agent with large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/2305.16291, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:258887849" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:258887849</a>.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2022)</span>
<span class="ltx_bibblock">
Ruoyao Wang, Peter Alexander Jansen, Marc-Alexandre Côté, and
Prithviraj Ammanabrolu.

</span>
<span class="ltx_bibblock">Scienceworld: Is your agent smarter than a 5th grader?

</span>
<span class="ltx_bibblock">In <em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">Conference on Empirical Methods in Natural Language
Processing</em>, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:247451124" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:247451124</a>.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yao et al. (2020)</span>
<span class="ltx_bibblock">
Shunyu Yao, Rohan Rao, Matthew J. Hausknecht, and Karthik Narasimhan.

</span>
<span class="ltx_bibblock">Keep calm and explore: Language models for action generation in
text-based games.

</span>
<span class="ltx_bibblock"><em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/2010.02903, 2020.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:222142129" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:222142129</a>.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yao et al. (2022)</span>
<span class="ltx_bibblock">
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan,
and Yuan Cao.

</span>
<span class="ltx_bibblock">React: Synergizing reasoning and acting in language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">ArXiv</em>, abs/2210.03629, 2022.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://api.semanticscholar.org/CorpusID:252762395" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://api.semanticscholar.org/CorpusID:252762395</a>.

</span>
</li>
</ul>
</section>
<div class="ltx_pagination ltx_role_newpage"></div>
<div class="ltx_pagination ltx_role_newpage"></div>
<section id="A1" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix A </span><span id="A1.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> prompts</h2>

<div id="A1.p1" class="ltx_para ltx_noindent">
<p id="A1.p1.1" class="ltx_p"><a href="#A1.F6" title="In Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figures</span> <span class="ltx_text ltx_ref_tag">6</span></a>, <a href="#A1.F7" title="Figure 7 ‣ Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>, <a href="#A1.F8" title="Figure 8 ‣ Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a> and <a href="#A1.F9" title="Figure 9 ‣ Appendix A CLIN prompts ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a> are the complete prompts for next-action generation (controller + executor), memory-generator during <span id="A1.p1.1.1" class="ltx_text ltx_font_smallcaps">Adapt</span>, <span id="A1.p1.1.2" class="ltx_text ltx_font_smallcaps">Gen-Env</span>, and <span id="A1.p1.1.3" class="ltx_text ltx_font_smallcaps">Gen-Task</span>.</p>
</div>
<figure id="A1.F6" class="ltx_figure">
<div id="A1.F6.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,CltTeXN0ZW1dOiBZb3UgYXJlIGFuIEFJIGFnZW50IGhlbHBpbmcgZXhlY3V0ZSBhIHNjaWVuY2UgZXhwZXJpbWVudCBpbiBhIHNpbXVsYXRlZCBlbnZpcm9ubWVudCB3aXRoIGxpbWl0ZWQgbnVtYmVyIG9mIG9iamVjdHMgYW5kIGFjdGlvbnMgYXZhaWxhYmxlIGF0IGVhY2ggc3RlcC4KXHBhcltVc2VyXToKUG9zc2libGUgb2JqZWN0cyAoIHZhbHVlIGFuIE9CSiBjYW4gdGFrZSApOgp7b2JqZWN0c19zdHJ9ClxwYXJZb3VyIG5leHQgYWN0aW9uIHNob3VsZCBiZSBpbiBvbmUgb2YgdGhlIGZvbGxvd2luZyBmb3JtYXRzOgpQb3NzaWJsZSBhY3Rpb25zOgp7YWN0aW9uc19zdHJ9ClxwYXJJZiBJIHNheSBcIkFtYmlndW91cyByZXF1ZXN0XCIsIHlvdXIgYWN0aW9uIG1pZ2h0IG1lYW4gbXVsdGlwbGUgdGhpbmdzLiBJbiB0aGF0IGNhc2UsIHJlc3BvbmQgd2l0aCB0aGUgbnVtYmVyIGNvcnJlc3BvbmRpbmcgdG8gdGhlIGFjdGlvbiB5b3Ugd2FudCB0byB0YWtlLgpccGFyV2hhdCBhY3Rpb24gd291bGQgeW91IGxpa2UgdG8gZG8gbmV4dD8KXHBhckZpcnN0LCBzY2FuIHRoZSAodW5vcmRlcmVkKSBsaXN0IG9mIGxlYXJuaW5ncywgaWYgcHJvdmlkZWQuIERlY2lkZSBpZiBhbnkgb2YgdGhlIGxlYXJuaW5ncyBhcmUgYXBwbGljYWJsZSBnaXZlbiB0aGUgbGFzdCBvYnNlcnZhdGlvbiB0byBtYWtlIHByb2dyZXNzIGluIHRoaXMgdGFzay4gVGhlbiBvbmx5IHVzZSBzZWxlY3RlZCBsZWFybmluZ3MsIGlmIGFueSwgdG8gY29uc3RydWN0IGEgcmF0aW9uYWxlIGZvciBwaWNraW5nIHRoZSBuZXh0IGFjdGlvbi4gSWYgbm8gTGVhcm5pbmcgaXMgc2VsZWN0ZWQsIGNvbnN0cnVjdCB0aGUgcmF0aW9uYWxlIGJhc2VkIG9uIHRoZSBsYXN0IG9ic2VydmF0aW9uLiBGb3JtYXQgeW91ciByZXNwb25zZSBhcyBmb2xsb3dzOgpccGFyV3JpdGUgJ0kgdXNlZCBsZWFybmluZyBpZChzKTonIGFzIGEgY29tbWEgc2VwYXJhdGVkIGxpc3Q7IHRoZSBsaXN0IGNhbiBiZSBlbXB0eSBpZiBubyBsZWFybmluZ3Mgc2VsZWN0ZWQuIFRoZW4sIHdyaXRlICQkJCBmb2xsb3dlZCBieSB0aGUgcmF0aW9uYWxlLiBGaW5hbGx5LCB3cml0ZSAjIyMgZm9sbG93ZWQgYnkgdGhlIHNpbmdsZSBuZXh0IGFjdGlvbiB5b3Ugd291bGQgbGlrZSB0byB0YWtlLgpccGFySWYgeW91IHRoaW5rIHlvdSBoYXZlIGNvbXBsZXRlZCB0aGUgdGFzaywgcGxlYXNlIHdyaXRlIFRBU0tfQ09NUExFVEUgYXMgdGhlIG5leHQgYWN0aW9uLgpccGFySWYgdGhlIHRhc2sgcmVxdWlyZXMgeW91IHRvICdmb2N1cycgb24gc29tZXRoaW5nIChPQkopLCBwbGVhc2Ugd3JpdGUgRk9DVVMgT04gPE9CSj4gYXMgdGhlIG5leHQgYWN0aW9uLiBGT0NVUyBpcyBhIGV4dHJlbWVseSBjcml0aWNhbCBhY3Rpb24gdGhhdCBjYW4gYmUgb25seSB1c2VkIHRoZSBudW1iZXIgb2YgdGltZXMgJ2ZvY3VzJyBpcyBtZW50aW9uZWQgaW4gdGhlIHRhc2sgZGVzY3JpcHRpb24uIFVzaW5nIGl0IG1vcmUgdGhhbiB0aGF0IG9yIGluYXBwcm9waWF0ZWx5IChzdWNoIGFzIG9uIGEgd3Jvbmcgb2JqZWN0KSB3aWxsIHRlcm1pbmF0ZSB0aGUgc2Vzc2lvbiBhbmQgdGhlIHRhc2sgd2lsbCBiZSByZW5kZXJlZCBhcyBpbmNvbXBsZXRlLgpccGFySWYgeW91IHBlcmZvcm1lZCBhbiBhY3Rpb24gdGhhdCByZXF1aXJlcyB3YWl0aW5nIHRvIHNlZSB0aGUgZWZmZWN0LCBwbGVhc2Ugd3JpdGUgJ3dhaXQnIGFzIHRoZSBuZXh0IGFjdGlvbi4K" download="">⬇</a></div>
<div id="lstnumberx1" class="ltx_listingline">
</div>
<div id="lstnumberx2" class="ltx_listingline">[<span id="lstnumberx2.1" class="ltx_text ltx_lst_identifier">System</span>]:<span id="lstnumberx2.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.3" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx2.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.5" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx2.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.7" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx2.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.9" class="ltx_text ltx_lst_identifier">AI</span><span id="lstnumberx2.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.11" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx2.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.13" class="ltx_text ltx_lst_identifier">helping</span><span id="lstnumberx2.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.15" class="ltx_text ltx_lst_identifier">execute</span><span id="lstnumberx2.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.17" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx2.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.19" class="ltx_text ltx_lst_identifier">science</span><span id="lstnumberx2.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.21" class="ltx_text ltx_lst_identifier">experiment</span><span id="lstnumberx2.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.23" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx2.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.25" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx2.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.27" class="ltx_text ltx_lst_identifier">simulated</span><span id="lstnumberx2.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.29" class="ltx_text ltx_lst_identifier">environment</span><span id="lstnumberx2.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.31" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx2.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.33" class="ltx_text ltx_lst_identifier">limited</span><span id="lstnumberx2.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.35" class="ltx_text ltx_lst_identifier">number</span><span id="lstnumberx2.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.37" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx2.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.39" class="ltx_text ltx_lst_identifier">objects</span><span id="lstnumberx2.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.41" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx2.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.43" class="ltx_text ltx_lst_identifier">actions</span><span id="lstnumberx2.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.45" class="ltx_text ltx_lst_identifier">available</span><span id="lstnumberx2.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.47" class="ltx_text ltx_lst_identifier">at</span><span id="lstnumberx2.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.49" class="ltx_text ltx_lst_identifier">each</span><span id="lstnumberx2.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx2.51" class="ltx_text ltx_lst_identifier">step</span>.
</div>
<div id="lstnumberx3" class="ltx_listingline">\<span id="lstnumberx3.1" class="ltx_text ltx_lst_identifier">par</span>[<span id="lstnumberx3.2" class="ltx_text ltx_lst_identifier">User</span>]:
</div>
<div id="lstnumberx4" class="ltx_listingline">
<span id="lstnumberx4.1" class="ltx_text ltx_lst_identifier">Possible</span><span id="lstnumberx4.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.3" class="ltx_text ltx_lst_identifier">objects</span><span id="lstnumberx4.4" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx4.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.6" class="ltx_text ltx_lst_identifier">value</span><span id="lstnumberx4.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.8" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx4.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.10" class="ltx_text ltx_lst_identifier">OBJ</span><span id="lstnumberx4.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.12" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx4.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx4.14" class="ltx_text ltx_lst_identifier">take</span><span id="lstnumberx4.15" class="ltx_text ltx_lst_space"> </span>):
</div>
<div id="lstnumberx5" class="ltx_listingline">{<span id="lstnumberx5.1" class="ltx_text ltx_lst_identifier">objects_str</span>}
</div>
<div id="lstnumberx6" class="ltx_listingline">\<span id="lstnumberx6.1" class="ltx_text ltx_lst_identifier">parYour</span><span id="lstnumberx6.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.3" class="ltx_text ltx_lst_identifier">next</span><span id="lstnumberx6.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.5" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx6.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.7" class="ltx_text ltx_lst_identifier">should</span><span id="lstnumberx6.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.9" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx6.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.11" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx6.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.13" class="ltx_text ltx_lst_identifier">one</span><span id="lstnumberx6.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.15" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx6.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.17" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx6.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.19" class="ltx_text ltx_lst_identifier">following</span><span id="lstnumberx6.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx6.21" class="ltx_text ltx_lst_identifier">formats</span>:
</div>
<div id="lstnumberx7" class="ltx_listingline">
<span id="lstnumberx7.1" class="ltx_text ltx_lst_identifier">Possible</span><span id="lstnumberx7.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx7.3" class="ltx_text ltx_lst_identifier">actions</span>:
</div>
<div id="lstnumberx8" class="ltx_listingline">{<span id="lstnumberx8.1" class="ltx_text ltx_lst_identifier">actions_str</span>}
</div>
<div id="lstnumberx9" class="ltx_listingline">\<span id="lstnumberx9.1" class="ltx_text ltx_lst_identifier">parIf</span><span id="lstnumberx9.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.3" class="ltx_text ltx_lst_identifier">I</span><span id="lstnumberx9.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.5" class="ltx_text ltx_lst_identifier">say</span><span id="lstnumberx9.6" class="ltx_text ltx_lst_space"> </span>\"<span id="lstnumberx9.7" class="ltx_text ltx_lst_identifier">Ambiguous</span><span id="lstnumberx9.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.9" class="ltx_text ltx_lst_identifier">request</span>\",<span id="lstnumberx9.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.11" class="ltx_text ltx_lst_identifier">your</span><span id="lstnumberx9.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.13" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx9.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.15" class="ltx_text ltx_lst_identifier">might</span><span id="lstnumberx9.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.17" class="ltx_text ltx_lst_identifier">mean</span><span id="lstnumberx9.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.19" class="ltx_text ltx_lst_identifier">multiple</span><span id="lstnumberx9.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.21" class="ltx_text ltx_lst_identifier">things</span>.<span id="lstnumberx9.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.23" class="ltx_text ltx_lst_identifier">In</span><span id="lstnumberx9.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.25" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx9.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.27" class="ltx_text ltx_lst_identifier">case</span>,<span id="lstnumberx9.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.29" class="ltx_text ltx_lst_identifier">respond</span><span id="lstnumberx9.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.31" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx9.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.33" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx9.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.35" class="ltx_text ltx_lst_identifier">number</span><span id="lstnumberx9.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.37" class="ltx_text ltx_lst_identifier">corresponding</span><span id="lstnumberx9.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.39" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx9.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.41" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx9.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.43" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx9.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.45" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx9.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.47" class="ltx_text ltx_lst_identifier">want</span><span id="lstnumberx9.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.49" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx9.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx9.51" class="ltx_text ltx_lst_identifier">take</span>.
</div>
<div id="lstnumberx10" class="ltx_listingline">\<span id="lstnumberx10.1" class="ltx_text ltx_lst_identifier">parWhat</span><span id="lstnumberx10.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.3" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx10.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.5" class="ltx_text ltx_lst_identifier">would</span><span id="lstnumberx10.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.7" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx10.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.9" class="ltx_text ltx_lst_identifier">like</span><span id="lstnumberx10.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.11" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx10.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.13" class="ltx_text ltx_lst_identifier">do</span><span id="lstnumberx10.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx10.15" class="ltx_text ltx_lst_identifier">next</span>?
</div>
<div id="lstnumberx11" class="ltx_listingline">\<span id="lstnumberx11.1" class="ltx_text ltx_lst_identifier">parFirst</span>,<span id="lstnumberx11.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.3" class="ltx_text ltx_lst_identifier">scan</span><span id="lstnumberx11.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.5" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx11.6" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx11.7" class="ltx_text ltx_lst_identifier">unordered</span>)<span id="lstnumberx11.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.9" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx11.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.11" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx11.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.13" class="ltx_text ltx_lst_identifier">learnings</span>,<span id="lstnumberx11.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.15" class="ltx_text ltx_lst_identifier">if</span><span id="lstnumberx11.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.17" class="ltx_text ltx_lst_identifier">provided</span>.<span id="lstnumberx11.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.19" class="ltx_text ltx_lst_identifier">Decide</span><span id="lstnumberx11.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.21" class="ltx_text ltx_lst_identifier">if</span><span id="lstnumberx11.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.23" class="ltx_text ltx_lst_identifier">any</span><span id="lstnumberx11.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.25" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx11.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.27" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx11.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.29" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx11.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.31" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx11.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.33" class="ltx_text ltx_lst_identifier">applicable</span><span id="lstnumberx11.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.35" class="ltx_text ltx_lst_identifier">given</span><span id="lstnumberx11.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.37" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx11.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.39" class="ltx_text ltx_lst_identifier">last</span><span id="lstnumberx11.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.41" class="ltx_text ltx_lst_identifier">observation</span><span id="lstnumberx11.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.43" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx11.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.45" class="ltx_text ltx_lst_identifier">make</span><span id="lstnumberx11.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.47" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx11.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.49" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx11.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.51" class="ltx_text ltx_lst_identifier">this</span><span id="lstnumberx11.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.53" class="ltx_text ltx_lst_identifier">task</span>.<span id="lstnumberx11.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.55" class="ltx_text ltx_lst_identifier">Then</span><span id="lstnumberx11.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.57" class="ltx_text ltx_lst_identifier">only</span><span id="lstnumberx11.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.59" class="ltx_text ltx_lst_identifier">use</span><span id="lstnumberx11.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.61" class="ltx_text ltx_lst_identifier">selected</span><span id="lstnumberx11.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.63" class="ltx_text ltx_lst_identifier">learnings</span>,<span id="lstnumberx11.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.65" class="ltx_text ltx_lst_identifier">if</span><span id="lstnumberx11.66" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.67" class="ltx_text ltx_lst_identifier">any</span>,<span id="lstnumberx11.68" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.69" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx11.70" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.71" class="ltx_text ltx_lst_identifier">construct</span><span id="lstnumberx11.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.73" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx11.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.75" class="ltx_text ltx_lst_identifier">rationale</span><span id="lstnumberx11.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.77" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx11.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.79" class="ltx_text ltx_lst_identifier">picking</span><span id="lstnumberx11.80" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.81" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx11.82" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.83" class="ltx_text ltx_lst_identifier">next</span><span id="lstnumberx11.84" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.85" class="ltx_text ltx_lst_identifier">action</span>.<span id="lstnumberx11.86" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.87" class="ltx_text ltx_lst_identifier">If</span><span id="lstnumberx11.88" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.89" class="ltx_text ltx_lst_identifier">no</span><span id="lstnumberx11.90" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.91" class="ltx_text ltx_lst_identifier">Learning</span><span id="lstnumberx11.92" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.93" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx11.94" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.95" class="ltx_text ltx_lst_identifier">selected</span>,<span id="lstnumberx11.96" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.97" class="ltx_text ltx_lst_identifier">construct</span><span id="lstnumberx11.98" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.99" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx11.100" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.101" class="ltx_text ltx_lst_identifier">rationale</span><span id="lstnumberx11.102" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.103" class="ltx_text ltx_lst_identifier">based</span><span id="lstnumberx11.104" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.105" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx11.106" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.107" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx11.108" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.109" class="ltx_text ltx_lst_identifier">last</span><span id="lstnumberx11.110" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.111" class="ltx_text ltx_lst_identifier">observation</span>.<span id="lstnumberx11.112" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.113" class="ltx_text ltx_lst_identifier">Format</span><span id="lstnumberx11.114" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.115" class="ltx_text ltx_lst_identifier">your</span><span id="lstnumberx11.116" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.117" class="ltx_text ltx_lst_identifier">response</span><span id="lstnumberx11.118" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.119" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx11.120" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx11.121" class="ltx_text ltx_lst_identifier">follows</span>:
</div>
<div id="lstnumberx12" class="ltx_listingline">\<span id="lstnumberx12.1" class="ltx_text ltx_lst_identifier">parWrite</span><span id="lstnumberx12.2" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx12.3" class="ltx_text ltx_lst_identifier">I</span><span id="lstnumberx12.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.5" class="ltx_text ltx_lst_identifier">used</span><span id="lstnumberx12.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.7" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx12.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.9" class="ltx_text ltx_lst_identifier">id</span>(<span id="lstnumberx12.10" class="ltx_text ltx_lst_identifier">s</span>):’<span id="lstnumberx12.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.12" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx12.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.14" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx12.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.16" class="ltx_text ltx_lst_identifier">comma</span><span id="lstnumberx12.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.18" class="ltx_text ltx_lst_identifier">separated</span><span id="lstnumberx12.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.20" class="ltx_text ltx_lst_identifier">list</span>;<span id="lstnumberx12.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.22" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx12.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.24" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx12.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.26" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx12.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.28" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx12.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.30" class="ltx_text ltx_lst_identifier">empty</span><span id="lstnumberx12.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.32" class="ltx_text ltx_lst_identifier">if</span><span id="lstnumberx12.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.34" class="ltx_text ltx_lst_identifier">no</span><span id="lstnumberx12.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.36" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx12.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.38" class="ltx_text ltx_lst_identifier">selected</span>.<span id="lstnumberx12.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.40" class="ltx_text ltx_lst_identifier">Then</span>,<span id="lstnumberx12.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.42" class="ltx_text ltx_lst_identifier">write</span><span id="lstnumberx12.43" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.44" class="ltx_text ltx_lst_identifier">$$$</span><span id="lstnumberx12.45" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.46" class="ltx_text ltx_lst_identifier">followed</span><span id="lstnumberx12.47" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.48" class="ltx_text ltx_lst_identifier">by</span><span id="lstnumberx12.49" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.50" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx12.51" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.52" class="ltx_text ltx_lst_identifier">rationale</span>.<span id="lstnumberx12.53" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.54" class="ltx_text ltx_lst_identifier">Finally</span>,<span id="lstnumberx12.55" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.56" class="ltx_text ltx_lst_identifier">write</span><span id="lstnumberx12.57" class="ltx_text ltx_lst_space"> </span>###<span id="lstnumberx12.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.59" class="ltx_text ltx_lst_identifier">followed</span><span id="lstnumberx12.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.61" class="ltx_text ltx_lst_identifier">by</span><span id="lstnumberx12.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.63" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx12.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.65" class="ltx_text ltx_lst_identifier">single</span><span id="lstnumberx12.66" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.67" class="ltx_text ltx_lst_identifier">next</span><span id="lstnumberx12.68" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.69" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx12.70" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.71" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx12.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.73" class="ltx_text ltx_lst_identifier">would</span><span id="lstnumberx12.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.75" class="ltx_text ltx_lst_identifier">like</span><span id="lstnumberx12.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.77" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx12.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx12.79" class="ltx_text ltx_lst_identifier">take</span>.
</div>
<div id="lstnumberx13" class="ltx_listingline">\<span id="lstnumberx13.1" class="ltx_text ltx_lst_identifier">parIf</span><span id="lstnumberx13.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.3" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx13.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.5" class="ltx_text ltx_lst_identifier">think</span><span id="lstnumberx13.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.7" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx13.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.9" class="ltx_text ltx_lst_identifier">have</span><span id="lstnumberx13.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.11" class="ltx_text ltx_lst_identifier">completed</span><span id="lstnumberx13.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.13" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx13.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.15" class="ltx_text ltx_lst_identifier">task</span>,<span id="lstnumberx13.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.17" class="ltx_text ltx_lst_identifier">please</span><span id="lstnumberx13.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.19" class="ltx_text ltx_lst_identifier">write</span><span id="lstnumberx13.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.21" class="ltx_text ltx_lst_identifier">TASK_COMPLETE</span><span id="lstnumberx13.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.23" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx13.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.25" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx13.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.27" class="ltx_text ltx_lst_identifier">next</span><span id="lstnumberx13.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx13.29" class="ltx_text ltx_lst_identifier">action</span>.
</div>
<div id="lstnumberx14" class="ltx_listingline">\<span id="lstnumberx14.1" class="ltx_text ltx_lst_identifier">parIf</span><span id="lstnumberx14.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.3" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx14.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.5" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx14.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.7" class="ltx_text ltx_lst_identifier">requires</span><span id="lstnumberx14.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.9" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx14.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.11" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx14.12" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx14.13" class="ltx_text ltx_lst_identifier">focus</span>’<span id="lstnumberx14.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.15" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx14.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.17" class="ltx_text ltx_lst_identifier">something</span><span id="lstnumberx14.18" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx14.19" class="ltx_text ltx_lst_identifier">OBJ</span>),<span id="lstnumberx14.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.21" class="ltx_text ltx_lst_identifier">please</span><span id="lstnumberx14.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.23" class="ltx_text ltx_lst_identifier">write</span><span id="lstnumberx14.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.25" class="ltx_text ltx_lst_identifier">FOCUS</span><span id="lstnumberx14.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.27" class="ltx_text ltx_lst_identifier">ON</span><span id="lstnumberx14.28" class="ltx_text ltx_lst_space"> </span>&lt;<span id="lstnumberx14.29" class="ltx_text ltx_lst_identifier">OBJ</span>&gt;<span id="lstnumberx14.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.31" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx14.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.33" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx14.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.35" class="ltx_text ltx_lst_identifier">next</span><span id="lstnumberx14.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.37" class="ltx_text ltx_lst_identifier">action</span>.<span id="lstnumberx14.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.39" class="ltx_text ltx_lst_identifier">FOCUS</span><span id="lstnumberx14.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.41" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx14.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.43" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx14.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.45" class="ltx_text ltx_lst_identifier">extremely</span><span id="lstnumberx14.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.47" class="ltx_text ltx_lst_identifier">critical</span><span id="lstnumberx14.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.49" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx14.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.51" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx14.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.53" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx14.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.55" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx14.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.57" class="ltx_text ltx_lst_identifier">only</span><span id="lstnumberx14.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.59" class="ltx_text ltx_lst_identifier">used</span><span id="lstnumberx14.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.61" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx14.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.63" class="ltx_text ltx_lst_identifier">number</span><span id="lstnumberx14.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.65" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx14.66" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.67" class="ltx_text ltx_lst_identifier">times</span><span id="lstnumberx14.68" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx14.69" class="ltx_text ltx_lst_identifier">focus</span>’<span id="lstnumberx14.70" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.71" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx14.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.73" class="ltx_text ltx_lst_identifier">mentioned</span><span id="lstnumberx14.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.75" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx14.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.77" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx14.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.79" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx14.80" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.81" class="ltx_text ltx_lst_identifier">description</span>.<span id="lstnumberx14.82" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.83" class="ltx_text ltx_lst_identifier">Using</span><span id="lstnumberx14.84" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.85" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx14.86" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.87" class="ltx_text ltx_lst_identifier">more</span><span id="lstnumberx14.88" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.89" class="ltx_text ltx_lst_identifier">than</span><span id="lstnumberx14.90" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.91" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx14.92" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.93" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx14.94" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.95" class="ltx_text ltx_lst_identifier">inappropiately</span><span id="lstnumberx14.96" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx14.97" class="ltx_text ltx_lst_identifier">such</span><span id="lstnumberx14.98" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.99" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx14.100" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.101" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx14.102" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.103" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx14.104" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.105" class="ltx_text ltx_lst_identifier">wrong</span><span id="lstnumberx14.106" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.107" class="ltx_text ltx_lst_identifier">object</span>)<span id="lstnumberx14.108" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.109" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx14.110" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.111" class="ltx_text ltx_lst_identifier">terminate</span><span id="lstnumberx14.112" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.113" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx14.114" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.115" class="ltx_text ltx_lst_identifier">session</span><span id="lstnumberx14.116" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.117" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx14.118" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.119" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx14.120" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.121" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx14.122" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.123" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx14.124" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.125" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx14.126" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.127" class="ltx_text ltx_lst_identifier">rendered</span><span id="lstnumberx14.128" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.129" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx14.130" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx14.131" class="ltx_text ltx_lst_identifier">incomplete</span>.
</div>
<div id="lstnumberx15" class="ltx_listingline">\<span id="lstnumberx15.1" class="ltx_text ltx_lst_identifier">parIf</span><span id="lstnumberx15.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.3" class="ltx_text ltx_lst_identifier">you</span><span id="lstnumberx15.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.5" class="ltx_text ltx_lst_identifier">performed</span><span id="lstnumberx15.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.7" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx15.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.9" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx15.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.11" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx15.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.13" class="ltx_text ltx_lst_identifier">requires</span><span id="lstnumberx15.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.15" class="ltx_text ltx_lst_identifier">waiting</span><span id="lstnumberx15.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.17" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx15.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.19" class="ltx_text ltx_lst_identifier">see</span><span id="lstnumberx15.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx15.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.23" class="ltx_text ltx_lst_identifier">effect</span>,<span id="lstnumberx15.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.25" class="ltx_text ltx_lst_identifier">please</span><span id="lstnumberx15.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.27" class="ltx_text ltx_lst_identifier">write</span><span id="lstnumberx15.28" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx15.29" class="ltx_text ltx_lst_identifier">wait</span>’<span id="lstnumberx15.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.31" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx15.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.33" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx15.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.35" class="ltx_text ltx_lst_identifier">next</span><span id="lstnumberx15.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx15.37" class="ltx_text ltx_lst_identifier">action</span>.
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 6: </span>Prompt for the Controller and the Executor</figcaption>
</figure>
<figure id="A1.F7" class="ltx_figure">
<div id="A1.F7.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,CltTeXN0ZW1dOiBZb3UgYXJlIGFuIGV4cGVydCBhc3Npc3RhbnQuClxwYXJbVXNlcl06CllvdSBhcmUgZ2l2ZW4gQ1VSUkVOVCBUUkFDRSwgYSBzZXF1ZW5jZSBvZiBhY3Rpb25zIHRoYXQgYW4gYWdlbnQgbWFkZSBpbiBhIHdvcmxkIHRvIGFjY29tcGxpc2ggYSB0YXNrLgpccGFyVGFzayBpcyBkZXRhaWxlZCBhdCB0aGUgYmVnaW5uaW5nLgpGb3IgZWFjaCBhY3Rpb24sIHRoZXJlIGlzIGEgcmF0aW9uYWxlIHdoeSB0aGUgYWdlbnQgbWFkZSB0aGF0IGFjdGlvbi4KVGhlcmUgaXMgYW4gb2JzZXJ2YXRpb24gdGhhdCBwcm92aWRlIGRldGFpbHMgYWJvdXQgdGhlIG5ldyBzdGF0ZSBvZiB0aGUgd29ybGQgYWZ0ZXIgZWFjaCBhY3Rpb24gd2FzIGV4ZWN1dGVkLgpUaGUgQ1VSUkVOVCBUUkFDRSBpcyBhY2NvbXBhbmllZCBieSBhbiBFVkFMVUFUSU9OIFJFUE9SVCBpbmRpY2F0aW5nIHRoZSBzdWNjZXNzIG9mIHRoZSBhdHRlbXB0IHRvIHRoZSB0YXNrLgpccGFyWW91IGNhbiBhbHNvIGJlIHByb3ZpZGVkIHdpdGggUFJFVklPVVMgTEVBUk5JTkdTIHdoaWNoIGFyZSBsZWFybmluZ3MgZnJvbSB0aGUgcHJldmlvdXMgYXR0ZW1wdHMgYnkgdGhlIGFnZW50IGZvciB0aGUgc2FtZSB0YXNrIGluIHRoZSBzYW1lIGVudmlyb25tZW50L3dvcmxkLiBUQVNLIGluZGljYXRlcyB0aGUgdGFzayBkZXNjcmlwdGlvbi4gRVBJU09ERSBpbmRpY2F0ZXMgdGhlIG51bWJlciBvZiBwcmV2aW91cyBhdHRlbXB0cyBvZiB0aGUgdGFzay4KXHBhckdlbmVyYXRlIGEgc3VtbWFyeSBvZiBsZWFybmluZywgYXMgYSBudW1iZXJlZCBsaXN0LCB0aGF0IHdpbGwgaGVscCB0aGUgYWdlbnQgdG8gc3VjY2Vzc2Z1bGx5IGFjY29tcGxpc2ggdGhlIFNBTUUgdGFzayBBR0FJTiwgaW4gdGhlIFNBTUUgd29ybGQuClxwYXJFYWNoIG51bWJlcmVkIGl0ZW0gaW4gdGhlIHN1bW1hcnkgY2FuIE9OTFkgYmUgb2YgdGhlIGZvcm06ClggTUFZIEJFIE5FQ0NFU1NBUlkgdG8gWS4KWCBTSE9VTEQgQkUgTkVDQ0VTU0FSWSB0byBZLgpYIE1BWSBCRSBDT05UUklCVVRFIHRvIFkuClggRE9FUyBOT1QgQ09OVFJJQlVURSB0byBZLgpccGFye0NVUlJFTlQgVFJBQ0V9CkFjdGlvbjogLi4uCk9ic2VydmF0aW9uOiAuLi4KLi4uCkVWQUxVQVRJT04gUkVQT1JUOgpSRVdBUkRfRklOQUw6IDEwMC4gVGhpcyBtZWFuczogVGhlIGFnZW50IGhhcyBwZXJmb3JtZWQgZXhjZXB0aW9uYWxseSB3ZWxsIGFuZCBzdWNjZXNzZnVsbHkgc29sdmVkIHRoZSB0YXNrLgpccGFyU3VtbWFyeSBvZiBsZWFybmluZyBhcyBhIG51bWJlcmVkIGxpc3Q6Cg==" download="">⬇</a></div>
<div id="lstnumberx16" class="ltx_listingline">
</div>
<div id="lstnumberx17" class="ltx_listingline">[<span id="lstnumberx17.1" class="ltx_text ltx_lst_identifier">System</span>]:<span id="lstnumberx17.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.3" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx17.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.5" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx17.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.7" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx17.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.9" class="ltx_text ltx_lst_identifier">expert</span><span id="lstnumberx17.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx17.11" class="ltx_text ltx_lst_identifier">assistant</span>.
</div>
<div id="lstnumberx18" class="ltx_listingline">\<span id="lstnumberx18.1" class="ltx_text ltx_lst_identifier">par</span>[<span id="lstnumberx18.2" class="ltx_text ltx_lst_identifier">User</span>]:
</div>
<div id="lstnumberx19" class="ltx_listingline">
<span id="lstnumberx19.1" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx19.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.3" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx19.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.5" class="ltx_text ltx_lst_identifier">given</span><span id="lstnumberx19.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.7" class="ltx_text ltx_lst_identifier">CURRENT</span><span id="lstnumberx19.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.9" class="ltx_text ltx_lst_identifier">TRACE</span>,<span id="lstnumberx19.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.11" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx19.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.13" class="ltx_text ltx_lst_identifier">sequence</span><span id="lstnumberx19.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.15" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx19.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.17" class="ltx_text ltx_lst_identifier">actions</span><span id="lstnumberx19.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.19" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx19.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.21" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx19.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.23" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx19.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.25" class="ltx_text ltx_lst_identifier">made</span><span id="lstnumberx19.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.27" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx19.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.29" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx19.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.31" class="ltx_text ltx_lst_identifier">world</span><span id="lstnumberx19.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.33" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx19.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.35" class="ltx_text ltx_lst_identifier">accomplish</span><span id="lstnumberx19.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.37" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx19.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx19.39" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx20" class="ltx_listingline">\<span id="lstnumberx20.1" class="ltx_text ltx_lst_identifier">parTask</span><span id="lstnumberx20.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.3" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx20.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.5" class="ltx_text ltx_lst_identifier">detailed</span><span id="lstnumberx20.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.7" class="ltx_text ltx_lst_identifier">at</span><span id="lstnumberx20.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx20.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx20.11" class="ltx_text ltx_lst_identifier">beginning</span>.
</div>
<div id="lstnumberx21" class="ltx_listingline">
<span id="lstnumberx21.1" class="ltx_text ltx_lst_identifier">For</span><span id="lstnumberx21.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.3" class="ltx_text ltx_lst_identifier">each</span><span id="lstnumberx21.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.5" class="ltx_text ltx_lst_identifier">action</span>,<span id="lstnumberx21.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.7" class="ltx_text ltx_lst_identifier">there</span><span id="lstnumberx21.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.9" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx21.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.11" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx21.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.13" class="ltx_text ltx_lst_identifier">rationale</span><span id="lstnumberx21.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.15" class="ltx_text ltx_lst_identifier">why</span><span id="lstnumberx21.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.17" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx21.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.19" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx21.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.21" class="ltx_text ltx_lst_identifier">made</span><span id="lstnumberx21.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.23" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx21.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx21.25" class="ltx_text ltx_lst_identifier">action</span>.
</div>
<div id="lstnumberx22" class="ltx_listingline">
<span id="lstnumberx22.1" class="ltx_text ltx_lst_identifier">There</span><span id="lstnumberx22.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.3" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx22.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.5" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx22.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.7" class="ltx_text ltx_lst_identifier">observation</span><span id="lstnumberx22.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.9" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx22.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.11" class="ltx_text ltx_lst_identifier">provide</span><span id="lstnumberx22.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.13" class="ltx_text ltx_lst_identifier">details</span><span id="lstnumberx22.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.15" class="ltx_text ltx_lst_identifier">about</span><span id="lstnumberx22.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.17" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx22.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.19" class="ltx_text ltx_lst_identifier">new</span><span id="lstnumberx22.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.21" class="ltx_text ltx_lst_identifier">state</span><span id="lstnumberx22.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.23" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx22.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.25" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx22.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.27" class="ltx_text ltx_lst_identifier">world</span><span id="lstnumberx22.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.29" class="ltx_text ltx_lst_identifier">after</span><span id="lstnumberx22.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.31" class="ltx_text ltx_lst_identifier">each</span><span id="lstnumberx22.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.33" class="ltx_text ltx_lst_identifier">action</span><span id="lstnumberx22.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.35" class="ltx_text ltx_lst_identifier">was</span><span id="lstnumberx22.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx22.37" class="ltx_text ltx_lst_identifier">executed</span>.
</div>
<div id="lstnumberx23" class="ltx_listingline">
<span id="lstnumberx23.1" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx23.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.3" class="ltx_text ltx_lst_identifier">CURRENT</span><span id="lstnumberx23.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.5" class="ltx_text ltx_lst_identifier">TRACE</span><span id="lstnumberx23.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx23.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.9" class="ltx_text ltx_lst_identifier">accompanied</span><span id="lstnumberx23.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.11" class="ltx_text ltx_lst_identifier">by</span><span id="lstnumberx23.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.13" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx23.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.15" class="ltx_text ltx_lst_identifier">EVALUATION</span><span id="lstnumberx23.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.17" class="ltx_text ltx_lst_identifier">REPORT</span><span id="lstnumberx23.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.19" class="ltx_text ltx_lst_identifier">indicating</span><span id="lstnumberx23.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx23.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.23" class="ltx_text ltx_lst_identifier">success</span><span id="lstnumberx23.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.25" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx23.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.27" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx23.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.29" class="ltx_text ltx_lst_identifier">attempt</span><span id="lstnumberx23.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.31" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx23.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.33" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx23.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx23.35" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx24" class="ltx_listingline">\<span id="lstnumberx24.1" class="ltx_text ltx_lst_identifier">parYou</span><span id="lstnumberx24.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.3" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx24.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.5" class="ltx_text ltx_lst_identifier">also</span><span id="lstnumberx24.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.7" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx24.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.9" class="ltx_text ltx_lst_identifier">provided</span><span id="lstnumberx24.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.11" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx24.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.13" class="ltx_text ltx_lst_identifier">PREVIOUS</span><span id="lstnumberx24.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.15" class="ltx_text ltx_lst_identifier">LEARNINGS</span><span id="lstnumberx24.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.17" class="ltx_text ltx_lst_identifier">which</span><span id="lstnumberx24.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.19" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx24.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.21" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx24.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.23" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx24.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.25" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.27" class="ltx_text ltx_lst_identifier">previous</span><span id="lstnumberx24.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.29" class="ltx_text ltx_lst_identifier">attempts</span><span id="lstnumberx24.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.31" class="ltx_text ltx_lst_identifier">by</span><span id="lstnumberx24.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.33" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.35" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx24.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.37" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx24.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.39" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.41" class="ltx_text ltx_lst_identifier">same</span><span id="lstnumberx24.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.43" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx24.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.45" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx24.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.47" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.49" class="ltx_text ltx_lst_identifier">same</span><span id="lstnumberx24.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.51" class="ltx_text ltx_lst_identifier">environment</span>/<span id="lstnumberx24.52" class="ltx_text ltx_lst_identifier">world</span>.<span id="lstnumberx24.53" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.54" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx24.55" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.56" class="ltx_text ltx_lst_identifier">indicates</span><span id="lstnumberx24.57" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.58" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.59" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.60" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx24.61" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.62" class="ltx_text ltx_lst_identifier">description</span>.<span id="lstnumberx24.63" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.64" class="ltx_text ltx_lst_identifier">EPISODE</span><span id="lstnumberx24.65" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.66" class="ltx_text ltx_lst_identifier">indicates</span><span id="lstnumberx24.67" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.68" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.69" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.70" class="ltx_text ltx_lst_identifier">number</span><span id="lstnumberx24.71" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.72" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx24.73" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.74" class="ltx_text ltx_lst_identifier">previous</span><span id="lstnumberx24.75" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.76" class="ltx_text ltx_lst_identifier">attempts</span><span id="lstnumberx24.77" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.78" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx24.79" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.80" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx24.81" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx24.82" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx25" class="ltx_listingline">\<span id="lstnumberx25.1" class="ltx_text ltx_lst_identifier">parGenerate</span><span id="lstnumberx25.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.3" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx25.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.5" class="ltx_text ltx_lst_identifier">summary</span><span id="lstnumberx25.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.7" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx25.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.9" class="ltx_text ltx_lst_identifier">learning</span>,<span id="lstnumberx25.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.11" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx25.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.13" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx25.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.15" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx25.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.17" class="ltx_text ltx_lst_identifier">list</span>,<span id="lstnumberx25.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.19" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx25.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.21" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx25.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.23" class="ltx_text ltx_lst_identifier">help</span><span id="lstnumberx25.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.25" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx25.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.27" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx25.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.29" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx25.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.31" class="ltx_text ltx_lst_identifier">successfully</span><span id="lstnumberx25.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.33" class="ltx_text ltx_lst_identifier">accomplish</span><span id="lstnumberx25.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.35" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx25.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.37" class="ltx_text ltx_lst_identifier">SAME</span><span id="lstnumberx25.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.39" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx25.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.41" class="ltx_text ltx_lst_identifier">AGAIN</span>,<span id="lstnumberx25.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.43" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx25.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.45" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx25.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.47" class="ltx_text ltx_lst_identifier">SAME</span><span id="lstnumberx25.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx25.49" class="ltx_text ltx_lst_identifier">world</span>.
</div>
<div id="lstnumberx26" class="ltx_listingline">\<span id="lstnumberx26.1" class="ltx_text ltx_lst_identifier">parEach</span><span id="lstnumberx26.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.3" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx26.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.5" class="ltx_text ltx_lst_identifier">item</span><span id="lstnumberx26.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.7" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx26.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx26.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.11" class="ltx_text ltx_lst_identifier">summary</span><span id="lstnumberx26.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.13" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx26.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.15" class="ltx_text ltx_lst_identifier">ONLY</span><span id="lstnumberx26.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.17" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx26.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.19" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx26.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx26.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx26.23" class="ltx_text ltx_lst_identifier">form</span>:
</div>
<div id="lstnumberx27" class="ltx_listingline">
<span id="lstnumberx27.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx27.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx27.3" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx27.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx27.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx27.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx27.7" class="ltx_text ltx_lst_identifier">NECCESSARY</span><span id="lstnumberx27.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx27.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx27.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx27.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx28" class="ltx_listingline">
<span id="lstnumberx28.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx28.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.3" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx28.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx28.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.7" class="ltx_text ltx_lst_identifier">NECCESSARY</span><span id="lstnumberx28.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx28.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx28.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx29" class="ltx_listingline">
<span id="lstnumberx29.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx29.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.3" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx29.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx29.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.7" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx29.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx29.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx29.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx30" class="ltx_listingline">
<span id="lstnumberx30.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx30.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx30.3" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx30.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx30.5" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx30.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx30.7" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx30.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx30.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx30.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx30.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx31" class="ltx_listingline">\<span id="lstnumberx31.1" class="ltx_text ltx_lst_identifier">par</span>{<span id="lstnumberx31.2" class="ltx_text ltx_lst_identifier">CURRENT</span><span id="lstnumberx31.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx31.4" class="ltx_text ltx_lst_identifier">TRACE</span>}
</div>
<div id="lstnumberx32" class="ltx_listingline">
<span id="lstnumberx32.1" class="ltx_text ltx_lst_identifier">Action</span>:<span id="lstnumberx32.2" class="ltx_text ltx_lst_space"> </span>…
</div>
<div id="lstnumberx33" class="ltx_listingline">
<span id="lstnumberx33.1" class="ltx_text ltx_lst_identifier">Observation</span>:<span id="lstnumberx33.2" class="ltx_text ltx_lst_space"> </span>…
</div>
<div id="lstnumberx34" class="ltx_listingline">…
</div>
<div id="lstnumberx35" class="ltx_listingline">
<span id="lstnumberx35.1" class="ltx_text ltx_lst_identifier">EVALUATION</span><span id="lstnumberx35.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx35.3" class="ltx_text ltx_lst_identifier">REPORT</span>:
</div>
<div id="lstnumberx36" class="ltx_listingline">
<span id="lstnumberx36.1" class="ltx_text ltx_lst_identifier">REWARD_FINAL</span>:<span id="lstnumberx36.2" class="ltx_text ltx_lst_space"> </span>100.<span id="lstnumberx36.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.4" class="ltx_text ltx_lst_identifier">This</span><span id="lstnumberx36.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.6" class="ltx_text ltx_lst_identifier">means</span>:<span id="lstnumberx36.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.8" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx36.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.10" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx36.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.12" class="ltx_text ltx_lst_identifier">has</span><span id="lstnumberx36.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.14" class="ltx_text ltx_lst_identifier">performed</span><span id="lstnumberx36.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.16" class="ltx_text ltx_lst_identifier">exceptionally</span><span id="lstnumberx36.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.18" class="ltx_text ltx_lst_identifier">well</span><span id="lstnumberx36.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.20" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx36.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.22" class="ltx_text ltx_lst_identifier">successfully</span><span id="lstnumberx36.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.24" class="ltx_text ltx_lst_identifier">solved</span><span id="lstnumberx36.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx36.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx36.28" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx37" class="ltx_listingline">\<span id="lstnumberx37.1" class="ltx_text ltx_lst_identifier">parSummary</span><span id="lstnumberx37.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.3" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx37.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.5" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx37.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.7" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx37.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.9" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx37.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.11" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx37.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx37.13" class="ltx_text ltx_lst_identifier">list</span>:
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 7: </span>Prompt for <span id="A1.F7.4.1" class="ltx_text ltx_font_smallcaps">CLIN</span>’s memory generator during <span id="A1.F7.5.2" class="ltx_text ltx_font_smallcaps">Adapt</span></figcaption>
</figure>
<figure id="A1.F8" class="ltx_figure">
<div id="A1.F8.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,CltTeXN0ZW1dOiBZb3UgYXJlIGFuIGV4cGVydCBhc3Npc3RhbnQuClxwYXJbVXNlcl06IFlvdSBhcmUgZ2l2ZW4gYSBjb2xsZWN0aW9uIG9mIGxlYXJuaW5nIGxpc3RzLCB0aGF0IGFyZSBkZXJpdmVkIGZyb20gYWN0aW9ucyBtYWRlIGJ5IGFuIGFnZW50IGFuZCBzdWJzZXF1ZW50IG9ic2VydmF0aW9ucyBmcm9tIGEgd29ybGQgdG8gYWNjb21wbGlzaCBhIFRZUEUgb2YgVEFTS3MuIEFsbCBvZiB0aGVzZSBUQVNLcyBiZWxvbmcgdG8gYSBzYW1lIFRZUEUgKHN1Y2ggYXMgJ2JvaWxpbmcnKSBidXQgdGhleSBhcmUgZXhlY3V0ZWQgaW4gZGlmZmVyZW50IEVOVklST05NRU5UIGNvbmZpZ3VyYXRpb25zLiBBIGRpZmZlcmVudCBFTlZJUk9OTUVOVCBjb25maWd1cmF0aW9uIG1lYW5zIHRoZXJlIGFyZSBwcmVzZW5jZSBvZiBhIGRpZmZlcmVudCBzZXQgb2Ygb2JqZWN0cyAobGlnaHRlciBpbnN0ZWFkIG9mIGEgc3RvdmUpIHRoYXQgYXJlIGNyaXRpY2FsIGZvciBzb2x2aW5nIHRoZSBUQVNLLCBwcmVzZW5jZSBvZiBhIGRpZmZlcmVudCBzZXQgb2YgZGlzdHJhY3RvciBvYmplY3RzIHRoYXQgYXJlIG5vdCB1c2VmdWwgZm9yIHRoZSBUQVNLLCBhIGRpZmZlcmVudCBmbG9vciBwbGFuLCBldGMuClxwYXJGb3IgZWFjaCBsZWFybmluZyBsaXN0LCB0aGUgVEFTSyBkZXNjcmlwdGlvbiBpcyBwcm92aWRlZCBhdCB0aGUgYmVnaW5uaW5nIGFzIFRBU0s6ClxwYXJFYWNoIGxlYXJuaW5nIGxpc3QgaW5kaWNhdGVzIGEgbGlzdCBvZiBsZWFybmluZ3MgZnJvbSB0aGUgYWdlbnQncyBiZXN0IGF0dGVtcHQgdG8gc29sdmUgdGhlIFRBU0suClxwYXJFYWNoIGxlYXJuaW5nIGxpc3QgaXMgYXNzb2NpYXRlZCB3aXRoIGFuIEVWQUxVQVRJT04gUkVQT1JUIGluZGljYXRlZCBob3cgc3VjZXNzZnVsIHRoZSByZXNwZWN0aXZlIGF0dGVtcHQgd2FzIGZvciBzb2x2aW5nIHRoZSB0YXNrLgpccGFyQ29uc2lkZXIgYWxsIGxlYXJuaW5nIGxpc3RzIGFuZCBjb21iaW5lIHRoZW0gaW4gdG8gYSBzdW1tYXJ5IG9mIGxlYXJuaW5ncywgYXMgYSBudW1iZXJlZCBsaXN0LCB0aGF0IHdpbGwgaGVscCB0aGUgYWdlbnQgdG8gc3VjY2Vzc2Z1bGx5IGFjY29tcGxpc2ggYSBORVcgVEFTSyByZWxhdGVkIHRvIHRoZSBwcmV2aW91cyBUQVNLcyAoc3VjaCBhcyAnYm9saWluZycpIGluIGFuIEVOVklST05NRU5UIGNvbmZpZ3VyYXRpb24gdGhhdCBpdCBoYXMgbm90IHNlZW4gYmVmb3JlLiBUaGUgTkVXIFRBU0sgZGVzY3JpcHRpb24gd2lsbCBiZSBwcm92aWRlZC4KXHBhckVhY2ggbnVtYmVyZWQgaXRlbSBpbiB0aGUgc3VtbWFyeSBjYW4gT05MWSBiZSBvZiB0aGUgZm9ybToKWCBNQVkgQkUgTkVDQ0VTU0FSWSB0byBZLgpYIFNIT1VMRCBCRSBORUNDRVNTQVJZIHRvIFkuClggTUFZIE5PVCBDT05UUklCVVRFIHRvIFkuClggRE9FUyBOT1QgQ09OVFJJQlVURSB0byBZLgpccGFye1BSRVZJT1VTIExFQVJOSU5HU30KVEFTSzogLi4uCkxFQVJOSU5HUzouLi4KRVZBTFVBVElPTiBSRVBPUlQ6ClJFV0FSRF9GSU5BTDogMTAwLiBUaGlzIG1lYW5zOiBUaGUgYWdlbnQgaGFzIHBlcmZvcm1lZCBleGNlcHRpb25hbGx5IHdlbGwgYW5kIHN1Y2Nlc3NmdWxseSBzb2x2ZWQgdGhlIHRhc2suCi4uLgpccGFyTkVXIFRBU0s6IC4uLgpTdW1tYXJ5IG9mIGxlYXJuaW5nIGFzIGEgbnVtYmVyZWQgbGlzdDoK" download="">⬇</a></div>
<div id="lstnumberx38" class="ltx_listingline">
</div>
<div id="lstnumberx39" class="ltx_listingline">[<span id="lstnumberx39.1" class="ltx_text ltx_lst_identifier">System</span>]:<span id="lstnumberx39.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx39.3" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx39.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx39.5" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx39.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx39.7" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx39.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx39.9" class="ltx_text ltx_lst_identifier">expert</span><span id="lstnumberx39.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx39.11" class="ltx_text ltx_lst_identifier">assistant</span>.
</div>
<div id="lstnumberx40" class="ltx_listingline">\<span id="lstnumberx40.1" class="ltx_text ltx_lst_identifier">par</span>[<span id="lstnumberx40.2" class="ltx_text ltx_lst_identifier">User</span>]:<span id="lstnumberx40.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.4" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx40.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.6" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx40.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.8" class="ltx_text ltx_lst_identifier">given</span><span id="lstnumberx40.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.10" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.12" class="ltx_text ltx_lst_identifier">collection</span><span id="lstnumberx40.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.14" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.16" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx40.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.18" class="ltx_text ltx_lst_identifier">lists</span>,<span id="lstnumberx40.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.20" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx40.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.22" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx40.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.24" class="ltx_text ltx_lst_identifier">derived</span><span id="lstnumberx40.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.26" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx40.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.28" class="ltx_text ltx_lst_identifier">actions</span><span id="lstnumberx40.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.30" class="ltx_text ltx_lst_identifier">made</span><span id="lstnumberx40.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.32" class="ltx_text ltx_lst_identifier">by</span><span id="lstnumberx40.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.34" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx40.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.36" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx40.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.38" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx40.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.40" class="ltx_text ltx_lst_identifier">subsequent</span><span id="lstnumberx40.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.42" class="ltx_text ltx_lst_identifier">observations</span><span id="lstnumberx40.43" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.44" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx40.45" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.46" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.47" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.48" class="ltx_text ltx_lst_identifier">world</span><span id="lstnumberx40.49" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.50" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx40.51" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.52" class="ltx_text ltx_lst_identifier">accomplish</span><span id="lstnumberx40.53" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.54" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.55" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.56" class="ltx_text ltx_lst_identifier">TYPE</span><span id="lstnumberx40.57" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.58" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.59" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.60" class="ltx_text ltx_lst_identifier">TASKs</span>.<span id="lstnumberx40.61" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.62" class="ltx_text ltx_lst_identifier">All</span><span id="lstnumberx40.63" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.64" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.65" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.66" class="ltx_text ltx_lst_identifier">these</span><span id="lstnumberx40.67" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.68" class="ltx_text ltx_lst_identifier">TASKs</span><span id="lstnumberx40.69" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.70" class="ltx_text ltx_lst_identifier">belong</span><span id="lstnumberx40.71" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.72" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx40.73" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.74" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.75" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.76" class="ltx_text ltx_lst_identifier">same</span><span id="lstnumberx40.77" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.78" class="ltx_text ltx_lst_identifier">TYPE</span><span id="lstnumberx40.79" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx40.80" class="ltx_text ltx_lst_identifier">such</span><span id="lstnumberx40.81" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.82" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx40.83" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx40.84" class="ltx_text ltx_lst_identifier">boiling</span>’)<span id="lstnumberx40.85" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.86" class="ltx_text ltx_lst_identifier">but</span><span id="lstnumberx40.87" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.88" class="ltx_text ltx_lst_identifier">they</span><span id="lstnumberx40.89" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.90" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx40.91" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.92" class="ltx_text ltx_lst_identifier">executed</span><span id="lstnumberx40.93" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.94" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx40.95" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.96" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx40.97" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.98" class="ltx_text ltx_lst_identifier">ENVIRONMENT</span><span id="lstnumberx40.99" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.100" class="ltx_text ltx_lst_identifier">configurations</span>.<span id="lstnumberx40.101" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.102" class="ltx_text ltx_lst_identifier">A</span><span id="lstnumberx40.103" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.104" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx40.105" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.106" class="ltx_text ltx_lst_identifier">ENVIRONMENT</span><span id="lstnumberx40.107" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.108" class="ltx_text ltx_lst_identifier">configuration</span><span id="lstnumberx40.109" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.110" class="ltx_text ltx_lst_identifier">means</span><span id="lstnumberx40.111" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.112" class="ltx_text ltx_lst_identifier">there</span><span id="lstnumberx40.113" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.114" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx40.115" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.116" class="ltx_text ltx_lst_identifier">presence</span><span id="lstnumberx40.117" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.118" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.119" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.120" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.121" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.122" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx40.123" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.124" class="ltx_text ltx_lst_identifier">set</span><span id="lstnumberx40.125" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.126" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.127" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.128" class="ltx_text ltx_lst_identifier">objects</span><span id="lstnumberx40.129" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx40.130" class="ltx_text ltx_lst_identifier">lighter</span><span id="lstnumberx40.131" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.132" class="ltx_text ltx_lst_identifier">instead</span><span id="lstnumberx40.133" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.134" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.135" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.136" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.137" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.138" class="ltx_text ltx_lst_identifier">stove</span>)<span id="lstnumberx40.139" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.140" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx40.141" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.142" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx40.143" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.144" class="ltx_text ltx_lst_identifier">critical</span><span id="lstnumberx40.145" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.146" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx40.147" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.148" class="ltx_text ltx_lst_identifier">solving</span><span id="lstnumberx40.149" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.150" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx40.151" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.152" class="ltx_text ltx_lst_identifier">TASK</span>,<span id="lstnumberx40.153" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.154" class="ltx_text ltx_lst_identifier">presence</span><span id="lstnumberx40.155" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.156" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.157" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.158" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.159" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.160" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx40.161" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.162" class="ltx_text ltx_lst_identifier">set</span><span id="lstnumberx40.163" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.164" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx40.165" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.166" class="ltx_text ltx_lst_identifier">distractor</span><span id="lstnumberx40.167" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.168" class="ltx_text ltx_lst_identifier">objects</span><span id="lstnumberx40.169" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.170" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx40.171" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.172" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx40.173" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.174" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx40.175" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.176" class="ltx_text ltx_lst_identifier">useful</span><span id="lstnumberx40.177" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.178" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx40.179" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.180" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx40.181" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.182" class="ltx_text ltx_lst_identifier">TASK</span>,<span id="lstnumberx40.183" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.184" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx40.185" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.186" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx40.187" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.188" class="ltx_text ltx_lst_identifier">floor</span><span id="lstnumberx40.189" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.190" class="ltx_text ltx_lst_identifier">plan</span>,<span id="lstnumberx40.191" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx40.192" class="ltx_text ltx_lst_identifier">etc</span>.
</div>
<div id="lstnumberx41" class="ltx_listingline">\<span id="lstnumberx41.1" class="ltx_text ltx_lst_identifier">parFor</span><span id="lstnumberx41.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.3" class="ltx_text ltx_lst_identifier">each</span><span id="lstnumberx41.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.5" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx41.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.7" class="ltx_text ltx_lst_identifier">list</span>,<span id="lstnumberx41.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx41.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.11" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx41.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.13" class="ltx_text ltx_lst_identifier">description</span><span id="lstnumberx41.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.15" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx41.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.17" class="ltx_text ltx_lst_identifier">provided</span><span id="lstnumberx41.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.19" class="ltx_text ltx_lst_identifier">at</span><span id="lstnumberx41.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx41.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.23" class="ltx_text ltx_lst_identifier">beginning</span><span id="lstnumberx41.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.25" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx41.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx41.27" class="ltx_text ltx_lst_identifier">TASK</span>:
</div>
<div id="lstnumberx42" class="ltx_listingline">\<span id="lstnumberx42.1" class="ltx_text ltx_lst_identifier">parEach</span><span id="lstnumberx42.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.3" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx42.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.5" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx42.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.7" class="ltx_text ltx_lst_identifier">indicates</span><span id="lstnumberx42.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.9" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx42.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.11" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx42.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.13" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx42.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.15" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx42.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.17" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx42.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.19" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx42.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.21" class="ltx_text ltx_lst_identifier">agent</span>’<span id="lstnumberx42.22" class="ltx_text ltx_lst_identifier">s</span><span id="lstnumberx42.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.24" class="ltx_text ltx_lst_identifier">best</span><span id="lstnumberx42.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.26" class="ltx_text ltx_lst_identifier">attempt</span><span id="lstnumberx42.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.28" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx42.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.30" class="ltx_text ltx_lst_identifier">solve</span><span id="lstnumberx42.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.32" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx42.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx42.34" class="ltx_text ltx_lst_identifier">TASK</span>.
</div>
<div id="lstnumberx43" class="ltx_listingline">\<span id="lstnumberx43.1" class="ltx_text ltx_lst_identifier">parEach</span><span id="lstnumberx43.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.3" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx43.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.5" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx43.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx43.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.9" class="ltx_text ltx_lst_identifier">associated</span><span id="lstnumberx43.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.11" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx43.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.13" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx43.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.15" class="ltx_text ltx_lst_identifier">EVALUATION</span><span id="lstnumberx43.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.17" class="ltx_text ltx_lst_identifier">REPORT</span><span id="lstnumberx43.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.19" class="ltx_text ltx_lst_identifier">indicated</span><span id="lstnumberx43.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.21" class="ltx_text ltx_lst_identifier">how</span><span id="lstnumberx43.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.23" class="ltx_text ltx_lst_identifier">sucessful</span><span id="lstnumberx43.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.25" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx43.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.27" class="ltx_text ltx_lst_identifier">respective</span><span id="lstnumberx43.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.29" class="ltx_text ltx_lst_identifier">attempt</span><span id="lstnumberx43.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.31" class="ltx_text ltx_lst_identifier">was</span><span id="lstnumberx43.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.33" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx43.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.35" class="ltx_text ltx_lst_identifier">solving</span><span id="lstnumberx43.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.37" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx43.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx43.39" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx44" class="ltx_listingline">\<span id="lstnumberx44.1" class="ltx_text ltx_lst_identifier">parConsider</span><span id="lstnumberx44.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.3" class="ltx_text ltx_lst_identifier">all</span><span id="lstnumberx44.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.5" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx44.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.7" class="ltx_text ltx_lst_identifier">lists</span><span id="lstnumberx44.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.9" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx44.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.11" class="ltx_text ltx_lst_identifier">combine</span><span id="lstnumberx44.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.13" class="ltx_text ltx_lst_identifier">them</span><span id="lstnumberx44.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.15" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx44.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.17" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx44.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.19" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx44.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.21" class="ltx_text ltx_lst_identifier">summary</span><span id="lstnumberx44.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.23" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx44.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.25" class="ltx_text ltx_lst_identifier">learnings</span>,<span id="lstnumberx44.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.27" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx44.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.29" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx44.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.31" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx44.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.33" class="ltx_text ltx_lst_identifier">list</span>,<span id="lstnumberx44.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.35" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx44.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.37" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx44.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.39" class="ltx_text ltx_lst_identifier">help</span><span id="lstnumberx44.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.41" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx44.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.43" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx44.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.45" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx44.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.47" class="ltx_text ltx_lst_identifier">successfully</span><span id="lstnumberx44.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.49" class="ltx_text ltx_lst_identifier">accomplish</span><span id="lstnumberx44.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.51" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx44.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.53" class="ltx_text ltx_lst_identifier">NEW</span><span id="lstnumberx44.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.55" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx44.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.57" class="ltx_text ltx_lst_identifier">related</span><span id="lstnumberx44.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.59" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx44.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.61" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx44.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.63" class="ltx_text ltx_lst_identifier">previous</span><span id="lstnumberx44.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.65" class="ltx_text ltx_lst_identifier">TASKs</span><span id="lstnumberx44.66" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx44.67" class="ltx_text ltx_lst_identifier">such</span><span id="lstnumberx44.68" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.69" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx44.70" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx44.71" class="ltx_text ltx_lst_identifier">boliing</span>’)<span id="lstnumberx44.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.73" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx44.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.75" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx44.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.77" class="ltx_text ltx_lst_identifier">ENVIRONMENT</span><span id="lstnumberx44.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.79" class="ltx_text ltx_lst_identifier">configuration</span><span id="lstnumberx44.80" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.81" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx44.82" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.83" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx44.84" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.85" class="ltx_text ltx_lst_identifier">has</span><span id="lstnumberx44.86" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.87" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx44.88" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.89" class="ltx_text ltx_lst_identifier">seen</span><span id="lstnumberx44.90" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.91" class="ltx_text ltx_lst_identifier">before</span>.<span id="lstnumberx44.92" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.93" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx44.94" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.95" class="ltx_text ltx_lst_identifier">NEW</span><span id="lstnumberx44.96" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.97" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx44.98" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.99" class="ltx_text ltx_lst_identifier">description</span><span id="lstnumberx44.100" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.101" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx44.102" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.103" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx44.104" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx44.105" class="ltx_text ltx_lst_identifier">provided</span>.
</div>
<div id="lstnumberx45" class="ltx_listingline">\<span id="lstnumberx45.1" class="ltx_text ltx_lst_identifier">parEach</span><span id="lstnumberx45.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.3" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx45.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.5" class="ltx_text ltx_lst_identifier">item</span><span id="lstnumberx45.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.7" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx45.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx45.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.11" class="ltx_text ltx_lst_identifier">summary</span><span id="lstnumberx45.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.13" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx45.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.15" class="ltx_text ltx_lst_identifier">ONLY</span><span id="lstnumberx45.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.17" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx45.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.19" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx45.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx45.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx45.23" class="ltx_text ltx_lst_identifier">form</span>:
</div>
<div id="lstnumberx46" class="ltx_listingline">
<span id="lstnumberx46.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx46.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.3" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx46.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx46.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.7" class="ltx_text ltx_lst_identifier">NECCESSARY</span><span id="lstnumberx46.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx46.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx46.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx47" class="ltx_listingline">
<span id="lstnumberx47.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx47.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx47.3" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx47.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx47.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx47.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx47.7" class="ltx_text ltx_lst_identifier">NECCESSARY</span><span id="lstnumberx47.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx47.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx47.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx47.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx48" class="ltx_listingline">
<span id="lstnumberx48.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx48.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.3" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx48.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.5" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx48.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.7" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx48.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx48.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx48.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx49" class="ltx_listingline">
<span id="lstnumberx49.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx49.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx49.3" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx49.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx49.5" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx49.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx49.7" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx49.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx49.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx49.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx49.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx50" class="ltx_listingline">\<span id="lstnumberx50.1" class="ltx_text ltx_lst_identifier">par</span>{<span id="lstnumberx50.2" class="ltx_text ltx_lst_identifier">PREVIOUS</span><span id="lstnumberx50.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx50.4" class="ltx_text ltx_lst_identifier">LEARNINGS</span>}
</div>
<div id="lstnumberx51" class="ltx_listingline">
<span id="lstnumberx51.1" class="ltx_text ltx_lst_identifier">TASK</span>:<span id="lstnumberx51.2" class="ltx_text ltx_lst_space"> </span>…
</div>
<div id="lstnumberx52" class="ltx_listingline">
<span id="lstnumberx52.1" class="ltx_text ltx_lst_identifier">LEARNINGS</span>:…
</div>
<div id="lstnumberx53" class="ltx_listingline">
<span id="lstnumberx53.1" class="ltx_text ltx_lst_identifier">EVALUATION</span><span id="lstnumberx53.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx53.3" class="ltx_text ltx_lst_identifier">REPORT</span>:
</div>
<div id="lstnumberx54" class="ltx_listingline">
<span id="lstnumberx54.1" class="ltx_text ltx_lst_identifier">REWARD_FINAL</span>:<span id="lstnumberx54.2" class="ltx_text ltx_lst_space"> </span>100.<span id="lstnumberx54.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.4" class="ltx_text ltx_lst_identifier">This</span><span id="lstnumberx54.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.6" class="ltx_text ltx_lst_identifier">means</span>:<span id="lstnumberx54.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.8" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx54.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.10" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx54.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.12" class="ltx_text ltx_lst_identifier">has</span><span id="lstnumberx54.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.14" class="ltx_text ltx_lst_identifier">performed</span><span id="lstnumberx54.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.16" class="ltx_text ltx_lst_identifier">exceptionally</span><span id="lstnumberx54.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.18" class="ltx_text ltx_lst_identifier">well</span><span id="lstnumberx54.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.20" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx54.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.22" class="ltx_text ltx_lst_identifier">successfully</span><span id="lstnumberx54.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.24" class="ltx_text ltx_lst_identifier">solved</span><span id="lstnumberx54.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx54.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx54.28" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx55" class="ltx_listingline">…
</div>
<div id="lstnumberx56" class="ltx_listingline">\<span id="lstnumberx56.1" class="ltx_text ltx_lst_identifier">parNEW</span><span id="lstnumberx56.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx56.3" class="ltx_text ltx_lst_identifier">TASK</span>:<span id="lstnumberx56.4" class="ltx_text ltx_lst_space"> </span>…
</div>
<div id="lstnumberx57" class="ltx_listingline">
<span id="lstnumberx57.1" class="ltx_text ltx_lst_identifier">Summary</span><span id="lstnumberx57.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.3" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx57.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.5" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx57.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.7" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx57.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.9" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx57.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.11" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx57.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx57.13" class="ltx_text ltx_lst_identifier">list</span>:
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 8: </span>Prompt for <span id="A1.F8.4.1" class="ltx_text ltx_font_smallcaps">CLIN</span>’s memory generator during <span id="A1.F8.5.2" class="ltx_text ltx_font_smallcaps">Gen-Env</span></figcaption>
</figure>
<figure id="A1.F9" class="ltx_figure">
<div id="A1.F9.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,CltTeXN0ZW1dOiBZb3UgYXJlIGFuIGV4cGVydCBhc3Npc3RhbnQuClxwYXJbVXNlcl06IFlvdSBtYXkgYmUgZ2l2ZW4gYSBsaXN0IG9mIGxlYXJuaW5ncywgdGhhdCBhcmUgZGVyaXZlZCBmcm9tIGFjdGlvbnMgbWFkZSBieSBhbiBhZ2VudCBhbmQgc3Vic2VxdWVudCBvYnNlcnZhdGlvbnMgZnJvbSBhIHdvcmxkIHRvIGFjY29tcGxpc2ggYSBUQVNLIGluIGFuIEVOVklST05NRU5UIENPTkZJR1VSQVRJT04uClxwYXJGb3IgdGhlIGxlYXJuaW5nIGxpc3QsIHRoZSBUQVNLIGRlc2NyaXB0aW9uIGlzIHByb3ZpZGVkIGF0IHRoZSBiZWdpbm5pbmcgYXMgVEFTSzoKXHBhclRoZSBsZWFybmluZ3MgYXJlIGZyb20gdGhlIGFnZW50J3MgYmVzdCBhdHRlbXB0IHRvIHNvbHZlIHRoZSBUQVNLLgpccGFyVGhlIGxlYXJuaW5nIGxpc3QgaXMgYXNzb2NpYXRlZCB3aXRoIGFuIEVWQUxVQVRJT04gUkVQT1JUIGluZGljYXRlZCBob3cgc3VjZXNzZnVsIHRoZSBhdHRlbXB0IHdhcyBmb3Igc29sdmluZyB0aGUgdGFzay4KXHBhck5vdywgZ2VuZXJhdGUgYSBzdW1tYXJ5IG9mIGxlYXJuaW5ncyBmcm9tIHRoZSBleGlzdGluZyBvbmVzIGlmIHByb3ZpZGVkLCBzdWNoIHRoYXQgdGhleSB3aWxsIGJlIHVzZWZ1bCB0byB0aGUgTkVXIFRBU0sgaW4gdGhlIFNBTUUgRU5WSVJPTk1FTlQgQ09ORklHVVJBVElPTi4gVGhlIE5FVyBUQVNLIG1heSByZXF1aXJlIGRpZmZlcmVudCBhY3Rpb25zIHdoaWNoIGFyZSBub3QgY2FwdHVyZWQgaW4gdGhlIGdpdmVuIGxlYXJuaW5ncyBidXQgZ2l2ZW4gbGVhcm5pbmdzIGNhbiBiZSB1c2VkIHRvIGluZmVyIGFib3V0IHRoZSBFTlZJUk9OTUVOVCBDT05GSUdVUkFUSU9OLiBUaGUgTkVXIFRBU0sgZGVzY3JpcHRpb24gd2lsbCBiZSBnaXZlbi4gSWYgUFJFVklPVVMgTEVBUk5JTkdTIHNheXMgJ05vIGxlYXJuaW5ncyBhdmFpbGFibGUnLCBpbXByb3Zpc2UgbGVhcm5pbmdzIGZvciB0aGUgTkVXIFRBU0suClxwYXJFYWNoIG51bWJlcmVkIGl0ZW0gaW4gdGhlIHN1bW1hcnkgY2FuIE9OTFkgYmUgb2YgdGhlIGZvcm06ClggTUFZIEJFIE5FQ0NFU1NBUlkgdG8gWS4KWCBTSE9VTEQgQkUgTkVDQ0VTU0FSWSB0byBZLgpYIE1BWSBOT1QgQ09OVFJJQlVURSB0byBZLgpYIERPRVMgTk9UIENPTlRSSUJVVEUgdG8gWS4KXHBhcntQUkVWSU9VUyBMRUFSTklOR1N9ClRBU0s6IC4uLgpMRUFSTklOR1M6Li4uCkVWQUxVQVRJT04gUkVQT1JUOgpSRVdBUkRfRklOQUw6IDEwMC4gVGhpcyBtZWFuczogVGhlIGFnZW50IGhhcyBwZXJmb3JtZWQgZXhjZXB0aW9uYWxseSB3ZWxsIGFuZCBzdWNjZXNzZnVsbHkgc29sdmVkIHRoZSB0YXNrLgouLi4KXHBhck5FVyBUQVNLOiAuLi4KU3VtbWFyeSBvZiBsZWFybmluZyBhcyBhIG51bWJlcmVkIGxpc3Q6Cg==" download="">⬇</a></div>
<div id="lstnumberx58" class="ltx_listingline">
</div>
<div id="lstnumberx59" class="ltx_listingline">[<span id="lstnumberx59.1" class="ltx_text ltx_lst_identifier">System</span>]:<span id="lstnumberx59.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx59.3" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx59.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx59.5" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx59.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx59.7" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx59.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx59.9" class="ltx_text ltx_lst_identifier">expert</span><span id="lstnumberx59.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx59.11" class="ltx_text ltx_lst_identifier">assistant</span>.
</div>
<div id="lstnumberx60" class="ltx_listingline">\<span id="lstnumberx60.1" class="ltx_text ltx_lst_identifier">par</span>[<span id="lstnumberx60.2" class="ltx_text ltx_lst_identifier">User</span>]:<span id="lstnumberx60.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.4" class="ltx_text ltx_lst_identifier">You</span><span id="lstnumberx60.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.6" class="ltx_text ltx_lst_identifier">may</span><span id="lstnumberx60.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.8" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx60.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.10" class="ltx_text ltx_lst_identifier">given</span><span id="lstnumberx60.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.12" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx60.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.14" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx60.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.16" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx60.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.18" class="ltx_text ltx_lst_identifier">learnings</span>,<span id="lstnumberx60.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.20" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx60.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.22" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx60.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.24" class="ltx_text ltx_lst_identifier">derived</span><span id="lstnumberx60.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.26" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx60.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.28" class="ltx_text ltx_lst_identifier">actions</span><span id="lstnumberx60.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.30" class="ltx_text ltx_lst_identifier">made</span><span id="lstnumberx60.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.32" class="ltx_text ltx_lst_identifier">by</span><span id="lstnumberx60.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.34" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx60.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.36" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx60.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.38" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx60.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.40" class="ltx_text ltx_lst_identifier">subsequent</span><span id="lstnumberx60.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.42" class="ltx_text ltx_lst_identifier">observations</span><span id="lstnumberx60.43" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.44" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx60.45" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.46" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx60.47" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.48" class="ltx_text ltx_lst_identifier">world</span><span id="lstnumberx60.49" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.50" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx60.51" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.52" class="ltx_text ltx_lst_identifier">accomplish</span><span id="lstnumberx60.53" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.54" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx60.55" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.56" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx60.57" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.58" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx60.59" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.60" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx60.61" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.62" class="ltx_text ltx_lst_identifier">ENVIRONMENT</span><span id="lstnumberx60.63" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx60.64" class="ltx_text ltx_lst_identifier">CONFIGURATION</span>.
</div>
<div id="lstnumberx61" class="ltx_listingline">\<span id="lstnumberx61.1" class="ltx_text ltx_lst_identifier">parFor</span><span id="lstnumberx61.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.3" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx61.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.5" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx61.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.7" class="ltx_text ltx_lst_identifier">list</span>,<span id="lstnumberx61.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx61.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.11" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx61.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.13" class="ltx_text ltx_lst_identifier">description</span><span id="lstnumberx61.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.15" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx61.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.17" class="ltx_text ltx_lst_identifier">provided</span><span id="lstnumberx61.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.19" class="ltx_text ltx_lst_identifier">at</span><span id="lstnumberx61.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx61.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.23" class="ltx_text ltx_lst_identifier">beginning</span><span id="lstnumberx61.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.25" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx61.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx61.27" class="ltx_text ltx_lst_identifier">TASK</span>:
</div>
<div id="lstnumberx62" class="ltx_listingline">\<span id="lstnumberx62.1" class="ltx_text ltx_lst_identifier">parThe</span><span id="lstnumberx62.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.3" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx62.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.5" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx62.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.7" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx62.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx62.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.11" class="ltx_text ltx_lst_identifier">agent</span>’<span id="lstnumberx62.12" class="ltx_text ltx_lst_identifier">s</span><span id="lstnumberx62.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.14" class="ltx_text ltx_lst_identifier">best</span><span id="lstnumberx62.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.16" class="ltx_text ltx_lst_identifier">attempt</span><span id="lstnumberx62.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.18" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx62.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.20" class="ltx_text ltx_lst_identifier">solve</span><span id="lstnumberx62.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.22" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx62.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx62.24" class="ltx_text ltx_lst_identifier">TASK</span>.
</div>
<div id="lstnumberx63" class="ltx_listingline">\<span id="lstnumberx63.1" class="ltx_text ltx_lst_identifier">parThe</span><span id="lstnumberx63.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.3" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx63.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.5" class="ltx_text ltx_lst_identifier">list</span><span id="lstnumberx63.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx63.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.9" class="ltx_text ltx_lst_identifier">associated</span><span id="lstnumberx63.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.11" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx63.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.13" class="ltx_text ltx_lst_identifier">an</span><span id="lstnumberx63.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.15" class="ltx_text ltx_lst_identifier">EVALUATION</span><span id="lstnumberx63.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.17" class="ltx_text ltx_lst_identifier">REPORT</span><span id="lstnumberx63.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.19" class="ltx_text ltx_lst_identifier">indicated</span><span id="lstnumberx63.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.21" class="ltx_text ltx_lst_identifier">how</span><span id="lstnumberx63.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.23" class="ltx_text ltx_lst_identifier">sucessful</span><span id="lstnumberx63.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.25" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx63.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.27" class="ltx_text ltx_lst_identifier">attempt</span><span id="lstnumberx63.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.29" class="ltx_text ltx_lst_identifier">was</span><span id="lstnumberx63.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.31" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx63.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.33" class="ltx_text ltx_lst_identifier">solving</span><span id="lstnumberx63.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.35" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx63.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx63.37" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx64" class="ltx_listingline">\<span id="lstnumberx64.1" class="ltx_text ltx_lst_identifier">parNow</span>,<span id="lstnumberx64.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.3" class="ltx_text ltx_lst_identifier">generate</span><span id="lstnumberx64.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.5" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx64.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.7" class="ltx_text ltx_lst_identifier">summary</span><span id="lstnumberx64.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.9" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx64.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.11" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx64.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.13" class="ltx_text ltx_lst_identifier">from</span><span id="lstnumberx64.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.15" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx64.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.17" class="ltx_text ltx_lst_identifier">existing</span><span id="lstnumberx64.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.19" class="ltx_text ltx_lst_identifier">ones</span><span id="lstnumberx64.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.21" class="ltx_text ltx_lst_identifier">if</span><span id="lstnumberx64.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.23" class="ltx_text ltx_lst_identifier">provided</span>,<span id="lstnumberx64.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.25" class="ltx_text ltx_lst_identifier">such</span><span id="lstnumberx64.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.27" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx64.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.29" class="ltx_text ltx_lst_identifier">they</span><span id="lstnumberx64.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.31" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx64.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.33" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx64.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.35" class="ltx_text ltx_lst_identifier">useful</span><span id="lstnumberx64.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.37" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx64.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.39" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx64.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.41" class="ltx_text ltx_lst_identifier">NEW</span><span id="lstnumberx64.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.43" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx64.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.45" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx64.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.47" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx64.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.49" class="ltx_text ltx_lst_identifier">SAME</span><span id="lstnumberx64.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.51" class="ltx_text ltx_lst_identifier">ENVIRONMENT</span><span id="lstnumberx64.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.53" class="ltx_text ltx_lst_identifier">CONFIGURATION</span>.<span id="lstnumberx64.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.55" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx64.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.57" class="ltx_text ltx_lst_identifier">NEW</span><span id="lstnumberx64.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.59" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx64.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.61" class="ltx_text ltx_lst_identifier">may</span><span id="lstnumberx64.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.63" class="ltx_text ltx_lst_identifier">require</span><span id="lstnumberx64.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.65" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx64.66" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.67" class="ltx_text ltx_lst_identifier">actions</span><span id="lstnumberx64.68" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.69" class="ltx_text ltx_lst_identifier">which</span><span id="lstnumberx64.70" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.71" class="ltx_text ltx_lst_identifier">are</span><span id="lstnumberx64.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.73" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx64.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.75" class="ltx_text ltx_lst_identifier">captured</span><span id="lstnumberx64.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.77" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx64.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.79" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx64.80" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.81" class="ltx_text ltx_lst_identifier">given</span><span id="lstnumberx64.82" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.83" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx64.84" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.85" class="ltx_text ltx_lst_identifier">but</span><span id="lstnumberx64.86" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.87" class="ltx_text ltx_lst_identifier">given</span><span id="lstnumberx64.88" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.89" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx64.90" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.91" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx64.92" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.93" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx64.94" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.95" class="ltx_text ltx_lst_identifier">used</span><span id="lstnumberx64.96" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.97" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx64.98" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.99" class="ltx_text ltx_lst_identifier">infer</span><span id="lstnumberx64.100" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.101" class="ltx_text ltx_lst_identifier">about</span><span id="lstnumberx64.102" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.103" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx64.104" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.105" class="ltx_text ltx_lst_identifier">ENVIRONMENT</span><span id="lstnumberx64.106" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.107" class="ltx_text ltx_lst_identifier">CONFIGURATION</span>.<span id="lstnumberx64.108" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.109" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx64.110" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.111" class="ltx_text ltx_lst_identifier">NEW</span><span id="lstnumberx64.112" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.113" class="ltx_text ltx_lst_identifier">TASK</span><span id="lstnumberx64.114" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.115" class="ltx_text ltx_lst_identifier">description</span><span id="lstnumberx64.116" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.117" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx64.118" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.119" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx64.120" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.121" class="ltx_text ltx_lst_identifier">given</span>.<span id="lstnumberx64.122" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.123" class="ltx_text ltx_lst_identifier">If</span><span id="lstnumberx64.124" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.125" class="ltx_text ltx_lst_identifier">PREVIOUS</span><span id="lstnumberx64.126" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.127" class="ltx_text ltx_lst_identifier">LEARNINGS</span><span id="lstnumberx64.128" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.129" class="ltx_text ltx_lst_identifier">says</span><span id="lstnumberx64.130" class="ltx_text ltx_lst_space"> </span>’<span id="lstnumberx64.131" class="ltx_text ltx_lst_identifier">No</span><span id="lstnumberx64.132" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.133" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx64.134" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.135" class="ltx_text ltx_lst_identifier">available</span>’,<span id="lstnumberx64.136" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.137" class="ltx_text ltx_lst_identifier">improvise</span><span id="lstnumberx64.138" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.139" class="ltx_text ltx_lst_identifier">learnings</span><span id="lstnumberx64.140" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.141" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx64.142" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.143" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx64.144" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.145" class="ltx_text ltx_lst_identifier">NEW</span><span id="lstnumberx64.146" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx64.147" class="ltx_text ltx_lst_identifier">TASK</span>.
</div>
<div id="lstnumberx65" class="ltx_listingline">\<span id="lstnumberx65.1" class="ltx_text ltx_lst_identifier">parEach</span><span id="lstnumberx65.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.3" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx65.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.5" class="ltx_text ltx_lst_identifier">item</span><span id="lstnumberx65.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.7" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx65.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.9" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx65.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.11" class="ltx_text ltx_lst_identifier">summary</span><span id="lstnumberx65.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.13" class="ltx_text ltx_lst_identifier">can</span><span id="lstnumberx65.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.15" class="ltx_text ltx_lst_identifier">ONLY</span><span id="lstnumberx65.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.17" class="ltx_text ltx_lst_identifier">be</span><span id="lstnumberx65.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.19" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx65.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx65.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx65.23" class="ltx_text ltx_lst_identifier">form</span>:
</div>
<div id="lstnumberx66" class="ltx_listingline">
<span id="lstnumberx66.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx66.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx66.3" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx66.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx66.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx66.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx66.7" class="ltx_text ltx_lst_identifier">NECCESSARY</span><span id="lstnumberx66.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx66.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx66.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx66.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx67" class="ltx_listingline">
<span id="lstnumberx67.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx67.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.3" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx67.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.5" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx67.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.7" class="ltx_text ltx_lst_identifier">NECCESSARY</span><span id="lstnumberx67.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx67.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx67.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx68" class="ltx_listingline">
<span id="lstnumberx68.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx68.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx68.3" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx68.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx68.5" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx68.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx68.7" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx68.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx68.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx68.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx68.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx69" class="ltx_listingline">
<span id="lstnumberx69.1" class="ltx_text ltx_lst_identifier">X</span><span id="lstnumberx69.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx69.3" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx69.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx69.5" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx69.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx69.7" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx69.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx69.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx69.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx69.11" class="ltx_text ltx_lst_identifier">Y</span>.
</div>
<div id="lstnumberx70" class="ltx_listingline">\<span id="lstnumberx70.1" class="ltx_text ltx_lst_identifier">par</span>{<span id="lstnumberx70.2" class="ltx_text ltx_lst_identifier">PREVIOUS</span><span id="lstnumberx70.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx70.4" class="ltx_text ltx_lst_identifier">LEARNINGS</span>}
</div>
<div id="lstnumberx71" class="ltx_listingline">
<span id="lstnumberx71.1" class="ltx_text ltx_lst_identifier">TASK</span>:<span id="lstnumberx71.2" class="ltx_text ltx_lst_space"> </span>…
</div>
<div id="lstnumberx72" class="ltx_listingline">
<span id="lstnumberx72.1" class="ltx_text ltx_lst_identifier">LEARNINGS</span>:…
</div>
<div id="lstnumberx73" class="ltx_listingline">
<span id="lstnumberx73.1" class="ltx_text ltx_lst_identifier">EVALUATION</span><span id="lstnumberx73.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx73.3" class="ltx_text ltx_lst_identifier">REPORT</span>:
</div>
<div id="lstnumberx74" class="ltx_listingline">
<span id="lstnumberx74.1" class="ltx_text ltx_lst_identifier">REWARD_FINAL</span>:<span id="lstnumberx74.2" class="ltx_text ltx_lst_space"> </span>100.<span id="lstnumberx74.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.4" class="ltx_text ltx_lst_identifier">This</span><span id="lstnumberx74.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.6" class="ltx_text ltx_lst_identifier">means</span>:<span id="lstnumberx74.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.8" class="ltx_text ltx_lst_identifier">The</span><span id="lstnumberx74.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.10" class="ltx_text ltx_lst_identifier">agent</span><span id="lstnumberx74.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.12" class="ltx_text ltx_lst_identifier">has</span><span id="lstnumberx74.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.14" class="ltx_text ltx_lst_identifier">performed</span><span id="lstnumberx74.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.16" class="ltx_text ltx_lst_identifier">exceptionally</span><span id="lstnumberx74.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.18" class="ltx_text ltx_lst_identifier">well</span><span id="lstnumberx74.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.20" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx74.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.22" class="ltx_text ltx_lst_identifier">successfully</span><span id="lstnumberx74.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.24" class="ltx_text ltx_lst_identifier">solved</span><span id="lstnumberx74.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx74.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx74.28" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx75" class="ltx_listingline">…
</div>
<div id="lstnumberx76" class="ltx_listingline">\<span id="lstnumberx76.1" class="ltx_text ltx_lst_identifier">parNEW</span><span id="lstnumberx76.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx76.3" class="ltx_text ltx_lst_identifier">TASK</span>:<span id="lstnumberx76.4" class="ltx_text ltx_lst_space"> </span>…
</div>
<div id="lstnumberx77" class="ltx_listingline">
<span id="lstnumberx77.1" class="ltx_text ltx_lst_identifier">Summary</span><span id="lstnumberx77.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx77.3" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx77.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx77.5" class="ltx_text ltx_lst_identifier">learning</span><span id="lstnumberx77.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx77.7" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx77.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx77.9" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx77.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx77.11" class="ltx_text ltx_lst_identifier">numbered</span><span id="lstnumberx77.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx77.13" class="ltx_text ltx_lst_identifier">list</span>:
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 9: </span>Prompt for <span id="A1.F9.4.1" class="ltx_text ltx_font_smallcaps">CLIN</span>’s memory generator during <span id="A1.F9.5.2" class="ltx_text ltx_font_smallcaps">Gen-Task</span></figcaption>
</figure>
</section>
<section id="A2" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix B </span>Example Memories</h2>

<div id="A2.p1" class="ltx_para ltx_noindent">
<p id="A2.p1.1" class="ltx_p">Example generated memory for <span id="A2.p1.1.1" class="ltx_text ltx_font_smallcaps">Adapt</span>, <span id="A2.p1.1.2" class="ltx_text ltx_font_smallcaps">Gen-Env</span>, and <span id="A2.p1.1.3" class="ltx_text ltx_font_smallcaps">Gen-Task</span>setups in <a href="#A2.F10" title="In Appendix B Example Memories ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Figures</span> <span class="ltx_text ltx_ref_tag">10</span></a>, <a href="#A2.F11" title="Figure 11 ‣ Appendix B Example Memories ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">11</span></a> and <a href="#A2.F12" title="Figure 12 ‣ Appendix B Example Memories ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">12</span></a>.</p>
</div>
<figure id="A2.F10" class="ltx_figure">
<div id="A2.F10.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,ClxwYXJUYXNrOiBZb3VyIHRhc2sgaXMgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgcm91bmQgc2VlZCBzaGFwZSBpcyBhIGRvbWluYW50IG9yIHJlY2Vzc2l2ZSB0cmFpdCBpbiB0aGUgcGVhIHBsYW50LiBJZiB0aGUgdHJhaXQgaXMgZG9taW5hbnQsIGZvY3VzIG9uIHRoZSBibHVlIGJveC4gSWYgdGhlIHRyYWl0IGlzIHJlY2Vzc2l2ZSwgZm9jdXMgb24gdGhlIG9yYW5nZSBib3guClxwYXJNZW1vcnkgKEFEQVBUKSwgYWZ0ZXIgVHJpYWw1OgoxLiBMb29raW5nIGFyb3VuZCBNQVkgQkUgTkVDRVNTQVJZIHRvIGdhdGhlciBpbmZvcm1hdGlvbiBhYm91dCB0aGUgZW52aXJvbm1lbnQuCjIuIE1vdmluZyB0byB0aGUgZ3JlZW5ob3VzZSBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIGZpbmQgdGhlIHBlYSBzZWVkcy4KMy4gUGlja2luZyB0aGUgamFyIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gYWNjZXNzIHRoZSBwZWEgc2VlZHMuCjQuIE1vdmluZyB0aGUgcm91bmQgZ3JlZW4gcGVhIHNlZWQgdG8gZmxvd2VyIHBvdCAyIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gZ3JvdyB0aGUgcGVhIHBsYW50cy4KNS4gTW92aW5nIHRoZSB3cmlua2x5IGdyZWVuIHBlYSBzZWVkIHRvIGZsb3dlciBwb3QgMyBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIGdyb3cgdGhlIHBlYSBwbGFudHMuCjYuIFdhaXRpbmcgU0hPVUxEIEJFIE5FQ0VTU0FSWSBmb3IgdGhlIHBlYSBwbGFudHMgdG8gZ3JvdyBhbmQgc2hvdyB0aGVpciB0cmFpdHMuCjcuIExvb2tpbmcgYXQgdGhlIGFkdWx0IHBlYSBwbGFudCBpbiBmbG93ZXIgcG90cyAyIGFuZCAzIE1BWSBCRSBORUNFU1NBUlkgdG8gb2JzZXJ2ZSB0aGUgdHJhaXRzIG9mIHRoZSBvZmZzcHJpbmcuCjguIEZvY3VzaW5nIG9uIHRoZSBibHVlIGJveCBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIGluZGljYXRlIHRoZSB0cmFpdCBvZiB0aGUgc2VlZCBzaGFwZS4KOS4gQ29udGludW91c2x5IGxvb2tpbmcgYXQgdGhlIHNhbWUgZW52aXJvbm1lbnQgRE9FUyBOT1QgQ09OVFJJQlVURSB0byB0aGUgdGFzayBvZiBkZXRlcm1pbmluZyB0aGUgZG9taW5hbnQgb3IgcmVjZXNzaXZlIHRyYWl0IGluIHRoZSBwZWEgcGxhbnQuCjEwLiBNb3ZpbmcgdG8gdGhlIHdvcmtzaG9wIERPRVMgTk9UIENPTlRSSUJVVEUgdG8gZGV0ZXJtaW5pbmcgdGhlIGRvbWluYW50IG9yIHJlY2Vzc2l2ZSB0cmFpdCBpbiB0aGUgcGVhIHBsYW50LgoxMS4gUGlja2luZyB1cCB0aGUgYmF0dGVyeSBvciB0aGUgd2lyZXMgaW4gdGhlIHdvcmtzaG9wIERPRVMgTk9UIENPTlRSSUJVVEUgdG8gZmluZGluZyB0aGUgcGVhIHNlZWRzLgoxMi4gV2FpdGluZyBmb3IgbG9uZ2VyIHBlcmlvZHMgTUFZIEJFIE5FQ0VTU0FSWSB0byBhbGxvdyB0aGUgcGVhIHBsYW50cyB0byBmdWxseSBncm93IGFuZCBzaG93IHRoZWlyIHRyYWl0cy4K" download="">⬇</a></div>
<div id="lstnumberx78" class="ltx_listingline">
</div>
<div id="lstnumberx79" class="ltx_listingline">\<span id="lstnumberx79.1" class="ltx_text ltx_lst_identifier">parTask</span>:<span id="lstnumberx79.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.3" class="ltx_text ltx_lst_identifier">Your</span><span id="lstnumberx79.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.5" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx79.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx79.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx79.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.11" class="ltx_text ltx_lst_identifier">determine</span><span id="lstnumberx79.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.13" class="ltx_text ltx_lst_identifier">whether</span><span id="lstnumberx79.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.15" class="ltx_text ltx_lst_identifier">round</span><span id="lstnumberx79.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.17" class="ltx_text ltx_lst_identifier">seed</span><span id="lstnumberx79.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.19" class="ltx_text ltx_lst_identifier">shape</span><span id="lstnumberx79.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.21" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx79.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.23" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx79.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.25" class="ltx_text ltx_lst_identifier">dominant</span><span id="lstnumberx79.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.27" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx79.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.29" class="ltx_text ltx_lst_identifier">recessive</span><span id="lstnumberx79.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.31" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx79.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.33" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx79.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.35" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx79.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.37" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx79.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.39" class="ltx_text ltx_lst_identifier">plant</span>.<span id="lstnumberx79.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.41" class="ltx_text ltx_lst_identifier">If</span><span id="lstnumberx79.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.43" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx79.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.45" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx79.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.47" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx79.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.49" class="ltx_text ltx_lst_identifier">dominant</span>,<span id="lstnumberx79.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.51" class="ltx_text ltx_lst_identifier">focus</span><span id="lstnumberx79.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.53" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx79.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.55" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx79.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.57" class="ltx_text ltx_lst_identifier">blue</span><span id="lstnumberx79.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.59" class="ltx_text ltx_lst_identifier">box</span>.<span id="lstnumberx79.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.61" class="ltx_text ltx_lst_identifier">If</span><span id="lstnumberx79.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.63" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx79.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.65" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx79.66" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.67" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx79.68" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.69" class="ltx_text ltx_lst_identifier">recessive</span>,<span id="lstnumberx79.70" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.71" class="ltx_text ltx_lst_identifier">focus</span><span id="lstnumberx79.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.73" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx79.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.75" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx79.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.77" class="ltx_text ltx_lst_identifier">orange</span><span id="lstnumberx79.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx79.79" class="ltx_text ltx_lst_identifier">box</span>.
</div>
<div id="lstnumberx80" class="ltx_listingline">\<span id="lstnumberx80.1" class="ltx_text ltx_lst_identifier">parMemory</span><span id="lstnumberx80.2" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx80.3" class="ltx_text ltx_lst_identifier">ADAPT</span>),<span id="lstnumberx80.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx80.5" class="ltx_text ltx_lst_identifier">after</span><span id="lstnumberx80.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx80.7" class="ltx_text ltx_lst_identifier">Trial5</span>:
</div>
<div id="lstnumberx81" class="ltx_listingline">1.<span id="lstnumberx81.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.2" class="ltx_text ltx_lst_identifier">Looking</span><span id="lstnumberx81.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.4" class="ltx_text ltx_lst_identifier">around</span><span id="lstnumberx81.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.6" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx81.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.8" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx81.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.10" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx81.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.12" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx81.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.14" class="ltx_text ltx_lst_identifier">gather</span><span id="lstnumberx81.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.16" class="ltx_text ltx_lst_identifier">information</span><span id="lstnumberx81.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.18" class="ltx_text ltx_lst_identifier">about</span><span id="lstnumberx81.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx81.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx81.22" class="ltx_text ltx_lst_identifier">environment</span>.
</div>
<div id="lstnumberx82" class="ltx_listingline">2.<span id="lstnumberx82.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx82.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx82.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx82.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.8" class="ltx_text ltx_lst_identifier">greenhouse</span><span id="lstnumberx82.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx82.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx82.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx82.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx82.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.18" class="ltx_text ltx_lst_identifier">find</span><span id="lstnumberx82.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx82.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.22" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx82.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx82.24" class="ltx_text ltx_lst_identifier">seeds</span>.
</div>
<div id="lstnumberx83" class="ltx_listingline">3.<span id="lstnumberx83.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.2" class="ltx_text ltx_lst_identifier">Picking</span><span id="lstnumberx83.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx83.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.6" class="ltx_text ltx_lst_identifier">jar</span><span id="lstnumberx83.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.8" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx83.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.10" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx83.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.12" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx83.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.14" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx83.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.16" class="ltx_text ltx_lst_identifier">access</span><span id="lstnumberx83.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.18" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx83.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.20" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx83.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx83.22" class="ltx_text ltx_lst_identifier">seeds</span>.
</div>
<div id="lstnumberx84" class="ltx_listingline">4.<span id="lstnumberx84.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx84.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx84.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.6" class="ltx_text ltx_lst_identifier">round</span><span id="lstnumberx84.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.8" class="ltx_text ltx_lst_identifier">green</span><span id="lstnumberx84.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.10" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx84.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.12" class="ltx_text ltx_lst_identifier">seed</span><span id="lstnumberx84.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.14" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx84.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.16" class="ltx_text ltx_lst_identifier">flower</span><span id="lstnumberx84.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.18" class="ltx_text ltx_lst_identifier">pot</span><span id="lstnumberx84.19" class="ltx_text ltx_lst_space"> </span>2<span id="lstnumberx84.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.21" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx84.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.23" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx84.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.25" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx84.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.27" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx84.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.29" class="ltx_text ltx_lst_identifier">grow</span><span id="lstnumberx84.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.31" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx84.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.33" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx84.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx84.35" class="ltx_text ltx_lst_identifier">plants</span>.
</div>
<div id="lstnumberx85" class="ltx_listingline">5.<span id="lstnumberx85.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx85.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx85.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.6" class="ltx_text ltx_lst_identifier">wrinkly</span><span id="lstnumberx85.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.8" class="ltx_text ltx_lst_identifier">green</span><span id="lstnumberx85.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.10" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx85.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.12" class="ltx_text ltx_lst_identifier">seed</span><span id="lstnumberx85.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.14" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx85.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.16" class="ltx_text ltx_lst_identifier">flower</span><span id="lstnumberx85.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.18" class="ltx_text ltx_lst_identifier">pot</span><span id="lstnumberx85.19" class="ltx_text ltx_lst_space"> </span>3<span id="lstnumberx85.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.21" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx85.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.23" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx85.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.25" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx85.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.27" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx85.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.29" class="ltx_text ltx_lst_identifier">grow</span><span id="lstnumberx85.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.31" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx85.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.33" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx85.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx85.35" class="ltx_text ltx_lst_identifier">plants</span>.
</div>
<div id="lstnumberx86" class="ltx_listingline">6.<span id="lstnumberx86.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.2" class="ltx_text ltx_lst_identifier">Waiting</span><span id="lstnumberx86.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.4" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx86.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.6" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx86.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.8" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx86.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.10" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx86.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.12" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx86.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.14" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx86.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.16" class="ltx_text ltx_lst_identifier">plants</span><span id="lstnumberx86.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.18" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx86.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.20" class="ltx_text ltx_lst_identifier">grow</span><span id="lstnumberx86.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.22" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx86.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.24" class="ltx_text ltx_lst_identifier">show</span><span id="lstnumberx86.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.26" class="ltx_text ltx_lst_identifier">their</span><span id="lstnumberx86.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx86.28" class="ltx_text ltx_lst_identifier">traits</span>.
</div>
<div id="lstnumberx87" class="ltx_listingline">7.<span id="lstnumberx87.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.2" class="ltx_text ltx_lst_identifier">Looking</span><span id="lstnumberx87.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.4" class="ltx_text ltx_lst_identifier">at</span><span id="lstnumberx87.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx87.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.8" class="ltx_text ltx_lst_identifier">adult</span><span id="lstnumberx87.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.10" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx87.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.12" class="ltx_text ltx_lst_identifier">plant</span><span id="lstnumberx87.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.14" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx87.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.16" class="ltx_text ltx_lst_identifier">flower</span><span id="lstnumberx87.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.18" class="ltx_text ltx_lst_identifier">pots</span><span id="lstnumberx87.19" class="ltx_text ltx_lst_space"> </span>2<span id="lstnumberx87.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.21" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx87.22" class="ltx_text ltx_lst_space"> </span>3<span id="lstnumberx87.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.24" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx87.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.26" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx87.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.28" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx87.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.30" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx87.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.32" class="ltx_text ltx_lst_identifier">observe</span><span id="lstnumberx87.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.34" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx87.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.36" class="ltx_text ltx_lst_identifier">traits</span><span id="lstnumberx87.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.38" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx87.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.40" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx87.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx87.42" class="ltx_text ltx_lst_identifier">offspring</span>.
</div>
<div id="lstnumberx88" class="ltx_listingline">8.<span id="lstnumberx88.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.2" class="ltx_text ltx_lst_identifier">Focusing</span><span id="lstnumberx88.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.4" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx88.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx88.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.8" class="ltx_text ltx_lst_identifier">blue</span><span id="lstnumberx88.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.10" class="ltx_text ltx_lst_identifier">box</span><span id="lstnumberx88.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.12" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx88.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.14" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx88.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.16" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx88.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.18" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx88.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.20" class="ltx_text ltx_lst_identifier">indicate</span><span id="lstnumberx88.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.22" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx88.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.24" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx88.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.26" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx88.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.28" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx88.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.30" class="ltx_text ltx_lst_identifier">seed</span><span id="lstnumberx88.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx88.32" class="ltx_text ltx_lst_identifier">shape</span>.
</div>
<div id="lstnumberx89" class="ltx_listingline">9.<span id="lstnumberx89.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.2" class="ltx_text ltx_lst_identifier">Continuously</span><span id="lstnumberx89.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.4" class="ltx_text ltx_lst_identifier">looking</span><span id="lstnumberx89.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.6" class="ltx_text ltx_lst_identifier">at</span><span id="lstnumberx89.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.8" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx89.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.10" class="ltx_text ltx_lst_identifier">same</span><span id="lstnumberx89.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.12" class="ltx_text ltx_lst_identifier">environment</span><span id="lstnumberx89.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.14" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx89.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.16" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx89.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.18" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx89.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.20" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx89.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.22" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx89.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.24" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx89.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.26" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx89.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.28" class="ltx_text ltx_lst_identifier">determining</span><span id="lstnumberx89.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx89.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.32" class="ltx_text ltx_lst_identifier">dominant</span><span id="lstnumberx89.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.34" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx89.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.36" class="ltx_text ltx_lst_identifier">recessive</span><span id="lstnumberx89.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.38" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx89.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.40" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx89.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.42" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx89.43" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.44" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx89.45" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx89.46" class="ltx_text ltx_lst_identifier">plant</span>.
</div>
<div id="lstnumberx90" class="ltx_listingline">10.<span id="lstnumberx90.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx90.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx90.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx90.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.8" class="ltx_text ltx_lst_identifier">workshop</span><span id="lstnumberx90.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.10" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx90.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.12" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx90.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.14" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx90.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx90.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.18" class="ltx_text ltx_lst_identifier">determining</span><span id="lstnumberx90.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx90.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.22" class="ltx_text ltx_lst_identifier">dominant</span><span id="lstnumberx90.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.24" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx90.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.26" class="ltx_text ltx_lst_identifier">recessive</span><span id="lstnumberx90.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.28" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx90.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.30" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx90.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.32" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx90.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.34" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx90.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx90.36" class="ltx_text ltx_lst_identifier">plant</span>.
</div>
<div id="lstnumberx91" class="ltx_listingline">11.<span id="lstnumberx91.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.2" class="ltx_text ltx_lst_identifier">Picking</span><span id="lstnumberx91.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.4" class="ltx_text ltx_lst_identifier">up</span><span id="lstnumberx91.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx91.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.8" class="ltx_text ltx_lst_identifier">battery</span><span id="lstnumberx91.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.10" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx91.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.12" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx91.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.14" class="ltx_text ltx_lst_identifier">wires</span><span id="lstnumberx91.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.16" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx91.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.18" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx91.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.20" class="ltx_text ltx_lst_identifier">workshop</span><span id="lstnumberx91.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.22" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx91.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.24" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx91.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.26" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx91.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.28" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx91.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.30" class="ltx_text ltx_lst_identifier">finding</span><span id="lstnumberx91.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.32" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx91.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.34" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx91.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx91.36" class="ltx_text ltx_lst_identifier">seeds</span>.
</div>
<div id="lstnumberx92" class="ltx_listingline">12.<span id="lstnumberx92.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.2" class="ltx_text ltx_lst_identifier">Waiting</span><span id="lstnumberx92.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.4" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx92.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.6" class="ltx_text ltx_lst_identifier">longer</span><span id="lstnumberx92.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.8" class="ltx_text ltx_lst_identifier">periods</span><span id="lstnumberx92.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.10" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx92.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx92.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx92.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx92.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.18" class="ltx_text ltx_lst_identifier">allow</span><span id="lstnumberx92.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx92.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.22" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx92.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.24" class="ltx_text ltx_lst_identifier">plants</span><span id="lstnumberx92.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.26" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx92.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.28" class="ltx_text ltx_lst_identifier">fully</span><span id="lstnumberx92.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.30" class="ltx_text ltx_lst_identifier">grow</span><span id="lstnumberx92.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.32" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx92.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.34" class="ltx_text ltx_lst_identifier">show</span><span id="lstnumberx92.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.36" class="ltx_text ltx_lst_identifier">their</span><span id="lstnumberx92.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx92.38" class="ltx_text ltx_lst_identifier">traits</span>.
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 10: </span>Memory generated after Trial5 in <span id="A2.F10.3.1" class="ltx_text ltx_font_smallcaps">Adapt</span> for a Genetics task.</figcaption>
</figure>
<figure id="A2.F11" class="ltx_figure">
<div id="A2.F11.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,ClxwYXJUYXNrOiBZb3VyIHRhc2sgaXMgdG8gZGV0ZXJtaW5lIHdoaWNoIG9mIHRoZSB0d28gaW5jbGluZWQgcGxhbmVzIChhbHVtaW51bSwgcGxhdGludW0pIGhhcyB0aGUgbW9zdCBmcmljdGlvbi4gQWZ0ZXIgY29tcGxldGluZyB5b3VyIGV4cGVyaW1lbnQsIGZvY3VzIG9uIHRoZSBpbmNsaW5lZCBwbGFuZSB3aXRoIHRoZSBtb3N0IGZyaWN0aW9uLgpccGFyTWV0YS1tZW1vcnkgKEdFTi1FTlYpOgoxLiBNb3ZpbmcgdG8gdGhlIGhhbGx3YXkgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byByZWFjaCB0aGUgd29ya3Nob3AuCjIuIE1vdmluZyB0byB0aGUgd29ya3Nob3AgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBmaW5kIHRoZSBibG9jay4KMy4gUGlja2luZyB1cCB0aGUgYmxvY2sgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBtb3ZlIGl0IHRvIHRoZSBpbmNsaW5lZCBwbGFuZXMuCjQuIFBsYWNpbmcgdGhlIGJsb2NrIG9uIHRoZSBmaXJzdCBpbmNsaW5lZCBwbGFuZSAoZWl0aGVyIGFsdW1pbnVtIG9yIHBsYXRpbnVtKSBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIG1lYXN1cmUgdGhlIGZyaWN0aW9uLgo1LiBBY3RpdmF0aW5nIHRoZSBzdG9wd2F0Y2ggU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byB0aW1lIHRoZSBleHBlcmltZW50Lgo2LiBXYWl0aW5nIGZvciBhIGNlcnRhaW4gcGVyaW9kIE1BWSBDT05UUklCVVRFIHRvIG9ic2VydmluZyB0aGUgZnJpY3Rpb24gZWZmZWN0Lgo3LiBEZWFjdGl2YXRpbmcgdGhlIHN0b3B3YXRjaCBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIHN0b3AgdGltaW5nIHRoZSBleHBlcmltZW50Lgo4LiBNb3ZpbmcgdGhlIGJsb2NrIHRvIHRoZSBzZWNvbmQgaW5jbGluZWQgcGxhbmUgKGVpdGhlciBhbHVtaW51bSBvciBwbGF0aW51bSkgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBjb21wYXJlIHRoZSBmcmljdGlvbi4KOS4gQWN0aXZhdGluZyB0aGUgc3RvcHdhdGNoIGFnYWluIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gdGltZSB0aGUgc2Vjb25kIHBhcnQgb2YgdGhlIGV4cGVyaW1lbnQuCjEwLiBXYWl0aW5nIGZvciBhIGNlcnRhaW4gcGVyaW9kIGFnYWluIE1BWSBCRSBORUNFU1NBUlkgdG8gb2JzZXJ2ZSB0aGUgZnJpY3Rpb24gZWZmZWN0LgoxMS4gRGVhY3RpdmF0aW5nIHRoZSBzdG9wd2F0Y2ggYWdhaW4gU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBzdG9wIHRpbWluZyB0aGUgZXhwZXJpbWVudC4KMTIuIEZvY3VzaW5nIG9uIHRoZSBpbmNsaW5lZCBwbGFuZSB3aXRoIHRoZSBtb3N0IGZyaWN0aW9uIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gY29uY2x1ZGUgdGhlIGV4cGVyaW1lbnQuCjEzLiBSZXBlYXRpbmcgdGhlIGV4cGVyaW1lbnQgbXVsdGlwbGUgdGltZXMgTUFZIEJFIE5FQ0VTU0FSWSBmb3IgbW9yZSBhY2N1cmF0ZSByZXN1bHRzLgoxNC4gTG9va2luZyBhcm91bmQgaW4gdGhlIGluaXRpYWwgcm9vbSBtdWx0aXBsZSB0aW1lcyBET0VTIE5PVCBDT05UUklCVVRFIHRvIHRoZSB0YXNrLgoxNS4gTW92aW5nIHRoZSBibG9jayBiYWNrIGFuZCBmb3J0aCBiZXR3ZWVuIHRoZSB0d28gaW5jbGluZWQgcGxhbmVzIERPRVMgTk9UIENPTlRSSUJVVEUgdG8gdGhlIHRhc2suCg==" download="">⬇</a></div>
<div id="lstnumberx93" class="ltx_listingline">
</div>
<div id="lstnumberx94" class="ltx_listingline">\<span id="lstnumberx94.1" class="ltx_text ltx_lst_identifier">parTask</span>:<span id="lstnumberx94.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.3" class="ltx_text ltx_lst_identifier">Your</span><span id="lstnumberx94.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.5" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx94.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx94.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx94.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.11" class="ltx_text ltx_lst_identifier">determine</span><span id="lstnumberx94.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.13" class="ltx_text ltx_lst_identifier">which</span><span id="lstnumberx94.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.15" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx94.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.17" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx94.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.19" class="ltx_text ltx_lst_identifier">two</span><span id="lstnumberx94.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.21" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx94.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.23" class="ltx_text ltx_lst_identifier">planes</span><span id="lstnumberx94.24" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx94.25" class="ltx_text ltx_lst_identifier">aluminum</span>,<span id="lstnumberx94.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.27" class="ltx_text ltx_lst_identifier">platinum</span>)<span id="lstnumberx94.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.29" class="ltx_text ltx_lst_identifier">has</span><span id="lstnumberx94.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.31" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx94.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.33" class="ltx_text ltx_lst_identifier">most</span><span id="lstnumberx94.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.35" class="ltx_text ltx_lst_identifier">friction</span>.<span id="lstnumberx94.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.37" class="ltx_text ltx_lst_identifier">After</span><span id="lstnumberx94.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.39" class="ltx_text ltx_lst_identifier">completing</span><span id="lstnumberx94.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.41" class="ltx_text ltx_lst_identifier">your</span><span id="lstnumberx94.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.43" class="ltx_text ltx_lst_identifier">experiment</span>,<span id="lstnumberx94.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.45" class="ltx_text ltx_lst_identifier">focus</span><span id="lstnumberx94.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.47" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx94.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.49" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx94.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.51" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx94.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.53" class="ltx_text ltx_lst_identifier">plane</span><span id="lstnumberx94.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.55" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx94.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.57" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx94.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.59" class="ltx_text ltx_lst_identifier">most</span><span id="lstnumberx94.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx94.61" class="ltx_text ltx_lst_identifier">friction</span>.
</div>
<div id="lstnumberx95" class="ltx_listingline">\<span id="lstnumberx95.1" class="ltx_text ltx_lst_identifier">parMeta</span>-<span id="lstnumberx95.2" class="ltx_text ltx_lst_identifier">memory</span><span id="lstnumberx95.3" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx95.4" class="ltx_text ltx_lst_identifier">GEN</span>-<span id="lstnumberx95.5" class="ltx_text ltx_lst_identifier">ENV</span>):
</div>
<div id="lstnumberx96" class="ltx_listingline">1.<span id="lstnumberx96.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx96.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx96.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx96.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.8" class="ltx_text ltx_lst_identifier">hallway</span><span id="lstnumberx96.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx96.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx96.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx96.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx96.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.18" class="ltx_text ltx_lst_identifier">reach</span><span id="lstnumberx96.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx96.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx96.22" class="ltx_text ltx_lst_identifier">workshop</span>.
</div>
<div id="lstnumberx97" class="ltx_listingline">2.<span id="lstnumberx97.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx97.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx97.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx97.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.8" class="ltx_text ltx_lst_identifier">workshop</span><span id="lstnumberx97.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx97.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx97.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx97.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx97.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.18" class="ltx_text ltx_lst_identifier">find</span><span id="lstnumberx97.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx97.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx97.22" class="ltx_text ltx_lst_identifier">block</span>.
</div>
<div id="lstnumberx98" class="ltx_listingline">3.<span id="lstnumberx98.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.2" class="ltx_text ltx_lst_identifier">Picking</span><span id="lstnumberx98.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.4" class="ltx_text ltx_lst_identifier">up</span><span id="lstnumberx98.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx98.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.8" class="ltx_text ltx_lst_identifier">block</span><span id="lstnumberx98.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx98.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx98.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx98.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx98.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.18" class="ltx_text ltx_lst_identifier">move</span><span id="lstnumberx98.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.20" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx98.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.22" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx98.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.24" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx98.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.26" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx98.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx98.28" class="ltx_text ltx_lst_identifier">planes</span>.
</div>
<div id="lstnumberx99" class="ltx_listingline">4.<span id="lstnumberx99.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.2" class="ltx_text ltx_lst_identifier">Placing</span><span id="lstnumberx99.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx99.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.6" class="ltx_text ltx_lst_identifier">block</span><span id="lstnumberx99.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.8" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx99.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.10" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx99.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.12" class="ltx_text ltx_lst_identifier">first</span><span id="lstnumberx99.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.14" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx99.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.16" class="ltx_text ltx_lst_identifier">plane</span><span id="lstnumberx99.17" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx99.18" class="ltx_text ltx_lst_identifier">either</span><span id="lstnumberx99.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.20" class="ltx_text ltx_lst_identifier">aluminum</span><span id="lstnumberx99.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.22" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx99.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.24" class="ltx_text ltx_lst_identifier">platinum</span>)<span id="lstnumberx99.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.26" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx99.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.28" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx99.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.30" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx99.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.32" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx99.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.34" class="ltx_text ltx_lst_identifier">measure</span><span id="lstnumberx99.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.36" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx99.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx99.38" class="ltx_text ltx_lst_identifier">friction</span>.
</div>
<div id="lstnumberx100" class="ltx_listingline">5.<span id="lstnumberx100.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.2" class="ltx_text ltx_lst_identifier">Activating</span><span id="lstnumberx100.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx100.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.6" class="ltx_text ltx_lst_identifier">stopwatch</span><span id="lstnumberx100.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.8" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx100.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.10" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx100.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.12" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx100.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.14" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx100.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.16" class="ltx_text ltx_lst_identifier">time</span><span id="lstnumberx100.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.18" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx100.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx100.20" class="ltx_text ltx_lst_identifier">experiment</span>.
</div>
<div id="lstnumberx101" class="ltx_listingline">6.<span id="lstnumberx101.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.2" class="ltx_text ltx_lst_identifier">Waiting</span><span id="lstnumberx101.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.4" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx101.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.6" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx101.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.8" class="ltx_text ltx_lst_identifier">certain</span><span id="lstnumberx101.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.10" class="ltx_text ltx_lst_identifier">period</span><span id="lstnumberx101.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.12" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx101.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.14" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx101.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx101.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.18" class="ltx_text ltx_lst_identifier">observing</span><span id="lstnumberx101.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx101.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.22" class="ltx_text ltx_lst_identifier">friction</span><span id="lstnumberx101.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx101.24" class="ltx_text ltx_lst_identifier">effect</span>.
</div>
<div id="lstnumberx102" class="ltx_listingline">7.<span id="lstnumberx102.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.2" class="ltx_text ltx_lst_identifier">Deactivating</span><span id="lstnumberx102.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx102.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.6" class="ltx_text ltx_lst_identifier">stopwatch</span><span id="lstnumberx102.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.8" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx102.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.10" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx102.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.12" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx102.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.14" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx102.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.16" class="ltx_text ltx_lst_identifier">stop</span><span id="lstnumberx102.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.18" class="ltx_text ltx_lst_identifier">timing</span><span id="lstnumberx102.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx102.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx102.22" class="ltx_text ltx_lst_identifier">experiment</span>.
</div>
<div id="lstnumberx103" class="ltx_listingline">8.<span id="lstnumberx103.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx103.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx103.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.6" class="ltx_text ltx_lst_identifier">block</span><span id="lstnumberx103.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.8" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx103.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.10" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx103.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.12" class="ltx_text ltx_lst_identifier">second</span><span id="lstnumberx103.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.14" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx103.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.16" class="ltx_text ltx_lst_identifier">plane</span><span id="lstnumberx103.17" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx103.18" class="ltx_text ltx_lst_identifier">either</span><span id="lstnumberx103.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.20" class="ltx_text ltx_lst_identifier">aluminum</span><span id="lstnumberx103.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.22" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx103.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.24" class="ltx_text ltx_lst_identifier">platinum</span>)<span id="lstnumberx103.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.26" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx103.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.28" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx103.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.30" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx103.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.32" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx103.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.34" class="ltx_text ltx_lst_identifier">compare</span><span id="lstnumberx103.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.36" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx103.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx103.38" class="ltx_text ltx_lst_identifier">friction</span>.
</div>
<div id="lstnumberx104" class="ltx_listingline">9.<span id="lstnumberx104.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.2" class="ltx_text ltx_lst_identifier">Activating</span><span id="lstnumberx104.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx104.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.6" class="ltx_text ltx_lst_identifier">stopwatch</span><span id="lstnumberx104.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.8" class="ltx_text ltx_lst_identifier">again</span><span id="lstnumberx104.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx104.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx104.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx104.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx104.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.18" class="ltx_text ltx_lst_identifier">time</span><span id="lstnumberx104.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx104.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.22" class="ltx_text ltx_lst_identifier">second</span><span id="lstnumberx104.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.24" class="ltx_text ltx_lst_identifier">part</span><span id="lstnumberx104.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.26" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx104.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.28" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx104.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx104.30" class="ltx_text ltx_lst_identifier">experiment</span>.
</div>
<div id="lstnumberx105" class="ltx_listingline">10.<span id="lstnumberx105.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.2" class="ltx_text ltx_lst_identifier">Waiting</span><span id="lstnumberx105.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.4" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx105.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.6" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx105.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.8" class="ltx_text ltx_lst_identifier">certain</span><span id="lstnumberx105.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.10" class="ltx_text ltx_lst_identifier">period</span><span id="lstnumberx105.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.12" class="ltx_text ltx_lst_identifier">again</span><span id="lstnumberx105.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.14" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx105.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.16" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx105.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.18" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx105.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.20" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx105.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.22" class="ltx_text ltx_lst_identifier">observe</span><span id="lstnumberx105.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.24" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx105.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.26" class="ltx_text ltx_lst_identifier">friction</span><span id="lstnumberx105.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx105.28" class="ltx_text ltx_lst_identifier">effect</span>.
</div>
<div id="lstnumberx106" class="ltx_listingline">11.<span id="lstnumberx106.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.2" class="ltx_text ltx_lst_identifier">Deactivating</span><span id="lstnumberx106.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx106.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.6" class="ltx_text ltx_lst_identifier">stopwatch</span><span id="lstnumberx106.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.8" class="ltx_text ltx_lst_identifier">again</span><span id="lstnumberx106.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx106.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx106.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx106.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx106.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.18" class="ltx_text ltx_lst_identifier">stop</span><span id="lstnumberx106.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.20" class="ltx_text ltx_lst_identifier">timing</span><span id="lstnumberx106.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.22" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx106.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx106.24" class="ltx_text ltx_lst_identifier">experiment</span>.
</div>
<div id="lstnumberx107" class="ltx_listingline">12.<span id="lstnumberx107.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.2" class="ltx_text ltx_lst_identifier">Focusing</span><span id="lstnumberx107.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.4" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx107.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx107.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.8" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx107.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.10" class="ltx_text ltx_lst_identifier">plane</span><span id="lstnumberx107.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.12" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx107.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.14" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx107.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.16" class="ltx_text ltx_lst_identifier">most</span><span id="lstnumberx107.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.18" class="ltx_text ltx_lst_identifier">friction</span><span id="lstnumberx107.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.20" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx107.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.22" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx107.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.24" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx107.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.26" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx107.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.28" class="ltx_text ltx_lst_identifier">conclude</span><span id="lstnumberx107.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx107.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx107.32" class="ltx_text ltx_lst_identifier">experiment</span>.
</div>
<div id="lstnumberx108" class="ltx_listingline">13.<span id="lstnumberx108.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.2" class="ltx_text ltx_lst_identifier">Repeating</span><span id="lstnumberx108.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx108.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.6" class="ltx_text ltx_lst_identifier">experiment</span><span id="lstnumberx108.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.8" class="ltx_text ltx_lst_identifier">multiple</span><span id="lstnumberx108.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.10" class="ltx_text ltx_lst_identifier">times</span><span id="lstnumberx108.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.12" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx108.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.14" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx108.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.16" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx108.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.18" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx108.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.20" class="ltx_text ltx_lst_identifier">more</span><span id="lstnumberx108.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.22" class="ltx_text ltx_lst_identifier">accurate</span><span id="lstnumberx108.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx108.24" class="ltx_text ltx_lst_identifier">results</span>.
</div>
<div id="lstnumberx109" class="ltx_listingline">14.<span id="lstnumberx109.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.2" class="ltx_text ltx_lst_identifier">Looking</span><span id="lstnumberx109.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.4" class="ltx_text ltx_lst_identifier">around</span><span id="lstnumberx109.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.6" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx109.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.8" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx109.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.10" class="ltx_text ltx_lst_identifier">initial</span><span id="lstnumberx109.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.12" class="ltx_text ltx_lst_identifier">room</span><span id="lstnumberx109.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.14" class="ltx_text ltx_lst_identifier">multiple</span><span id="lstnumberx109.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.16" class="ltx_text ltx_lst_identifier">times</span><span id="lstnumberx109.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.18" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx109.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.20" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx109.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.22" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx109.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.24" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx109.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx109.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx109.28" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx110" class="ltx_listingline">15.<span id="lstnumberx110.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx110.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx110.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.6" class="ltx_text ltx_lst_identifier">block</span><span id="lstnumberx110.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.8" class="ltx_text ltx_lst_identifier">back</span><span id="lstnumberx110.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.10" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx110.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.12" class="ltx_text ltx_lst_identifier">forth</span><span id="lstnumberx110.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.14" class="ltx_text ltx_lst_identifier">between</span><span id="lstnumberx110.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.16" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx110.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.18" class="ltx_text ltx_lst_identifier">two</span><span id="lstnumberx110.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.20" class="ltx_text ltx_lst_identifier">inclined</span><span id="lstnumberx110.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.22" class="ltx_text ltx_lst_identifier">planes</span><span id="lstnumberx110.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.24" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx110.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.26" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx110.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.28" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx110.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.30" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx110.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.32" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx110.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx110.34" class="ltx_text ltx_lst_identifier">task</span>.
</div>
</div>
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 11: </span>Meta-memory used in <span id="A2.F11.3.1" class="ltx_text ltx_font_smallcaps">Gen-Env</span> for a Friction task.</figcaption>
</figure>
<figure id="A2.F12" class="ltx_figure">
<div id="A2.F12.1" class="ltx_listing ltx_lstlisting ltx_listing">
<div class="ltx_listing_data"><a href="data:text/plain;base64,ClxwYXJUYXNrOiBZb3VyIHRhc2sgaXMgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgcm91bmQgc2VlZCBzaGFwZSBpcyBhIGRvbWluYW50IG9yIHJlY2Vzc2l2ZSB0cmFpdCBpbiB0aGUgcGVhIHBsYW50LiBJZiB0aGUgdHJhaXQgaXMgZG9taW5hbnQsIGZvY3VzIG9uIHRoZSBibHVlIGJveC4gSWYgdGhlIHRyYWl0IGlzIHJlY2Vzc2l2ZSwgZm9jdXMgb24gdGhlIG9yYW5nZSBib3guClxwYXJNZXRhLW1lbW9yeSAoR0VOLVRBU0spOgpUYXNrOiBZb3VyIHRhc2sgaXMgdG8gZnJlZXplIG1lcmN1cnkuIEZpcnN0LCBmb2N1cyBvbiB0aGUgc3Vic3RhbmNlLiBUaGVuLCB0YWtlIGFjdGlvbnMgdGhhdCB3aWxsIGNhdXNlIGl0IHRvIGNoYW5nZSBpdHMgc3RhdGUgb2YgbWF0dGVyLgpccGFyTWV0YS1tZW1vcnkgKEdFTi1UQVNLKToKMS4gTG9va2luZyBhcm91bmQgTUFZIEJFIE5FQ0VTU0FSWSB0byBpZGVudGlmeSB0aGUgYXZhaWxhYmxlIHJlc291cmNlcyBhbmQgdGhlIGxheW91dCBvZiB0aGUgZW52aXJvbm1lbnQuCjIuIE1vdmluZyB0byBkaWZmZXJlbnQgcm9vbXMgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBmaW5kIHRoZSB0b29scyBhbmQgbWF0ZXJpYWxzIG5lZWRlZCB0byBjaGFuZ2UgdGhlIHN0YXRlIG9mIHRoZSBzdWJzdGFuY2UuCjMuIFBpY2tpbmcgdXAgaXRlbXMgbGlrZSBnbGFzcyBjdXBzIG9yIG1ldGFsIHBvdHMgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBjb250YWluIHRoZSBzdWJzdGFuY2UgZm9yIGNoYW5naW5nIGl0cyBzdGF0ZS4KNC4gRm9jdXNpbmcgb24gdGhlIHN1YnN0YW5jZSBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIHVuZGVyc3RhbmQgaXRzIHByb3BlcnRpZXMgYW5kIGhvdyB0byBpbnRlcmFjdCB3aXRoIGl0Lgo1LiBQaWNraW5nIHVwIHRoZSB0aGVybW9tZXRlciBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIG1vbml0b3IgdGhlIHRlbXBlcmF0dXJlIG9mIHRoZSBzdWJzdGFuY2UuCjYuIFVzaW5nIHRoZSB0aGVybW9tZXRlciBvbiB0aGUgc3Vic3RhbmNlIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gbW9uaXRvciB0aGUgcHJvZ3Jlc3Mgb2YgdGhlIHRhc2suCjcuIFB1cmluZyB0aGUgc3Vic3RhbmNlIGludG8gdGhlIGNvbnRhaW5lciBTSE9VTEQgQkUgTkVDRVNTQVJZIHRvIHByZXBhcmUgaXQgZm9yIGNvb2xpbmcuCjguIE1vdmluZyB0aGUgY29udGFpbmVyIHRvIGEgY29vbGluZyBkZXZpY2UgU0hPVUxEIEJFIE5FQ0VTU0FSWSB0byBjb29sIHRoZSBzdWJzdGFuY2UuCjkuIFdhaXRpbmcgZm9yIGEgcGVyaW9kIG9mIHRpbWUgYWZ0ZXIgY29vbGluZyB0aGUgc3Vic3RhbmNlIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gYWxsb3cgdGhlIHN1YnN0YW5jZSB0byBjaGFuZ2Ugc3RhdGUuCjEwLiBSZXBlYXRlZGx5IGNoZWNraW5nIHRoZSB0ZW1wZXJhdHVyZSBvZiB0aGUgc3Vic3RhbmNlIFNIT1VMRCBCRSBORUNFU1NBUlkgdG8gbW9uaXRvciB0aGUgcHJvZ3Jlc3Mgb2YgdGhlIHRhc2suCjExLiBBY3RpdmF0aW5nIHRoZSBzdG92ZSBET0VTIE5PVCBDT05UUklCVVRFIHRvIHRoZSB0YXNrIGFzIGl0IGRvZXMgbm90IHByb2dyZXNzIHRoZSB0YXNrLgoxMi4gUGlja2luZyB1cCB1bnJlbGF0ZWQgaXRlbXMgbGlrZSBhIGxpZ2h0ZXIgRE9FUyBOT1QgQ09OVFJJQlVURSB0byB0aGUgdGFzayBhcyBpdCBkb2VzIG5vdCBwcm9ncmVzcyB0aGUgdGFzay4KMTMuIE1vdmluZyB0byB1bnJlbGF0ZWQgcm9vbXMgbGlrZSB0aGUgd29ya3Nob3AgRE9FUyBOT1QgQ09OVFJJQlVURSB0byB0aGUgdGFzayBhcyBpdCBkb2VzIG5vdCBwcm9ncmVzcyB0aGUgdGFzay4KMTQuIFRlbGVwb3J0aW5nIHRvIHRoZSBraXRjaGVuIE1BWSBCRSBORUNFU1NBUlkgZm9yIHRoZSB0YXNrIGFzIGl0IHNwZWVkcyB1cCB0aGUgcHJvY2VzcyBvZiBtb3ZpbmcgYmV0d2VlbiByb29tcy4KMTUuIFVzaW5nIHRoZSB0aGVybW9tZXRlciBtdWx0aXBsZSB0aW1lcyBvbiB0aGUgc3Vic3RhbmNlIGFmdGVyIGl0IHJlYWNoZXMgZnJlZXppbmcgcG9pbnQgRE9FUyBOT1QgQ09OVFJJQlVURSB0byB0aGUgdGFzayBhcyBpdCBkb2VzIG5vdCBwcm9ncmVzcyB0aGUgdGFzay4K" download="">⬇</a></div>
<div id="lstnumberx111" class="ltx_listingline">
</div>
<div id="lstnumberx112" class="ltx_listingline">\<span id="lstnumberx112.1" class="ltx_text ltx_lst_identifier">parTask</span>:<span id="lstnumberx112.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.3" class="ltx_text ltx_lst_identifier">Your</span><span id="lstnumberx112.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.5" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx112.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx112.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx112.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.11" class="ltx_text ltx_lst_identifier">determine</span><span id="lstnumberx112.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.13" class="ltx_text ltx_lst_identifier">whether</span><span id="lstnumberx112.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.15" class="ltx_text ltx_lst_identifier">round</span><span id="lstnumberx112.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.17" class="ltx_text ltx_lst_identifier">seed</span><span id="lstnumberx112.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.19" class="ltx_text ltx_lst_identifier">shape</span><span id="lstnumberx112.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.21" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx112.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.23" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx112.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.25" class="ltx_text ltx_lst_identifier">dominant</span><span id="lstnumberx112.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.27" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx112.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.29" class="ltx_text ltx_lst_identifier">recessive</span><span id="lstnumberx112.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.31" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx112.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.33" class="ltx_text ltx_lst_identifier">in</span><span id="lstnumberx112.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.35" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx112.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.37" class="ltx_text ltx_lst_identifier">pea</span><span id="lstnumberx112.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.39" class="ltx_text ltx_lst_identifier">plant</span>.<span id="lstnumberx112.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.41" class="ltx_text ltx_lst_identifier">If</span><span id="lstnumberx112.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.43" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx112.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.45" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx112.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.47" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx112.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.49" class="ltx_text ltx_lst_identifier">dominant</span>,<span id="lstnumberx112.50" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.51" class="ltx_text ltx_lst_identifier">focus</span><span id="lstnumberx112.52" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.53" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx112.54" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.55" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx112.56" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.57" class="ltx_text ltx_lst_identifier">blue</span><span id="lstnumberx112.58" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.59" class="ltx_text ltx_lst_identifier">box</span>.<span id="lstnumberx112.60" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.61" class="ltx_text ltx_lst_identifier">If</span><span id="lstnumberx112.62" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.63" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx112.64" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.65" class="ltx_text ltx_lst_identifier">trait</span><span id="lstnumberx112.66" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.67" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx112.68" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.69" class="ltx_text ltx_lst_identifier">recessive</span>,<span id="lstnumberx112.70" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.71" class="ltx_text ltx_lst_identifier">focus</span><span id="lstnumberx112.72" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.73" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx112.74" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.75" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx112.76" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.77" class="ltx_text ltx_lst_identifier">orange</span><span id="lstnumberx112.78" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx112.79" class="ltx_text ltx_lst_identifier">box</span>.
</div>
<div id="lstnumberx113" class="ltx_listingline">\<span id="lstnumberx113.1" class="ltx_text ltx_lst_identifier">parMeta</span>-<span id="lstnumberx113.2" class="ltx_text ltx_lst_identifier">memory</span><span id="lstnumberx113.3" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx113.4" class="ltx_text ltx_lst_identifier">GEN</span>-<span id="lstnumberx113.5" class="ltx_text ltx_lst_identifier">TASK</span>):
</div>
<div id="lstnumberx114" class="ltx_listingline">
<span id="lstnumberx114.1" class="ltx_text ltx_lst_identifier">Task</span>:<span id="lstnumberx114.2" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.3" class="ltx_text ltx_lst_identifier">Your</span><span id="lstnumberx114.4" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.5" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx114.6" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.7" class="ltx_text ltx_lst_identifier">is</span><span id="lstnumberx114.8" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.9" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx114.10" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.11" class="ltx_text ltx_lst_identifier">freeze</span><span id="lstnumberx114.12" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.13" class="ltx_text ltx_lst_identifier">mercury</span>.<span id="lstnumberx114.14" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.15" class="ltx_text ltx_lst_identifier">First</span>,<span id="lstnumberx114.16" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.17" class="ltx_text ltx_lst_identifier">focus</span><span id="lstnumberx114.18" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.19" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx114.20" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.21" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx114.22" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.23" class="ltx_text ltx_lst_identifier">substance</span>.<span id="lstnumberx114.24" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.25" class="ltx_text ltx_lst_identifier">Then</span>,<span id="lstnumberx114.26" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.27" class="ltx_text ltx_lst_identifier">take</span><span id="lstnumberx114.28" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.29" class="ltx_text ltx_lst_identifier">actions</span><span id="lstnumberx114.30" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.31" class="ltx_text ltx_lst_identifier">that</span><span id="lstnumberx114.32" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.33" class="ltx_text ltx_lst_identifier">will</span><span id="lstnumberx114.34" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.35" class="ltx_text ltx_lst_identifier">cause</span><span id="lstnumberx114.36" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.37" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx114.38" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.39" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx114.40" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.41" class="ltx_text ltx_lst_identifier">change</span><span id="lstnumberx114.42" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.43" class="ltx_text ltx_lst_identifier">its</span><span id="lstnumberx114.44" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.45" class="ltx_text ltx_lst_identifier">state</span><span id="lstnumberx114.46" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.47" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx114.48" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx114.49" class="ltx_text ltx_lst_identifier">matter</span>.
</div>
<div id="lstnumberx115" class="ltx_listingline">\<span id="lstnumberx115.1" class="ltx_text ltx_lst_identifier">parMeta</span>-<span id="lstnumberx115.2" class="ltx_text ltx_lst_identifier">memory</span><span id="lstnumberx115.3" class="ltx_text ltx_lst_space"> </span>(<span id="lstnumberx115.4" class="ltx_text ltx_lst_identifier">GEN</span>-<span id="lstnumberx115.5" class="ltx_text ltx_lst_identifier">TASK</span>):
</div>
<div id="lstnumberx116" class="ltx_listingline">1.<span id="lstnumberx116.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.2" class="ltx_text ltx_lst_identifier">Looking</span><span id="lstnumberx116.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.4" class="ltx_text ltx_lst_identifier">around</span><span id="lstnumberx116.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.6" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx116.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.8" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx116.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.10" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx116.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.12" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx116.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.14" class="ltx_text ltx_lst_identifier">identify</span><span id="lstnumberx116.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.16" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx116.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.18" class="ltx_text ltx_lst_identifier">available</span><span id="lstnumberx116.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.20" class="ltx_text ltx_lst_identifier">resources</span><span id="lstnumberx116.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.22" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx116.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.24" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx116.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.26" class="ltx_text ltx_lst_identifier">layout</span><span id="lstnumberx116.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.28" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx116.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx116.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx116.32" class="ltx_text ltx_lst_identifier">environment</span>.
</div>
<div id="lstnumberx117" class="ltx_listingline">2.<span id="lstnumberx117.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx117.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx117.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.6" class="ltx_text ltx_lst_identifier">different</span><span id="lstnumberx117.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.8" class="ltx_text ltx_lst_identifier">rooms</span><span id="lstnumberx117.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx117.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx117.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx117.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx117.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.18" class="ltx_text ltx_lst_identifier">find</span><span id="lstnumberx117.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx117.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.22" class="ltx_text ltx_lst_identifier">tools</span><span id="lstnumberx117.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.24" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx117.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.26" class="ltx_text ltx_lst_identifier">materials</span><span id="lstnumberx117.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.28" class="ltx_text ltx_lst_identifier">needed</span><span id="lstnumberx117.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.30" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx117.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.32" class="ltx_text ltx_lst_identifier">change</span><span id="lstnumberx117.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.34" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx117.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.36" class="ltx_text ltx_lst_identifier">state</span><span id="lstnumberx117.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.38" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx117.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.40" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx117.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx117.42" class="ltx_text ltx_lst_identifier">substance</span>.
</div>
<div id="lstnumberx118" class="ltx_listingline">3.<span id="lstnumberx118.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.2" class="ltx_text ltx_lst_identifier">Picking</span><span id="lstnumberx118.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.4" class="ltx_text ltx_lst_identifier">up</span><span id="lstnumberx118.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.6" class="ltx_text ltx_lst_identifier">items</span><span id="lstnumberx118.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.8" class="ltx_text ltx_lst_identifier">like</span><span id="lstnumberx118.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.10" class="ltx_text ltx_lst_identifier">glass</span><span id="lstnumberx118.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.12" class="ltx_text ltx_lst_identifier">cups</span><span id="lstnumberx118.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.14" class="ltx_text ltx_lst_identifier">or</span><span id="lstnumberx118.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.16" class="ltx_text ltx_lst_identifier">metal</span><span id="lstnumberx118.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.18" class="ltx_text ltx_lst_identifier">pots</span><span id="lstnumberx118.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.20" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx118.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.22" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx118.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.24" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx118.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.26" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx118.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.28" class="ltx_text ltx_lst_identifier">contain</span><span id="lstnumberx118.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx118.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.32" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx118.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.34" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx118.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.36" class="ltx_text ltx_lst_identifier">changing</span><span id="lstnumberx118.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.38" class="ltx_text ltx_lst_identifier">its</span><span id="lstnumberx118.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx118.40" class="ltx_text ltx_lst_identifier">state</span>.
</div>
<div id="lstnumberx119" class="ltx_listingline">4.<span id="lstnumberx119.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.2" class="ltx_text ltx_lst_identifier">Focusing</span><span id="lstnumberx119.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.4" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx119.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx119.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.8" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx119.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx119.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx119.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx119.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx119.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.18" class="ltx_text ltx_lst_identifier">understand</span><span id="lstnumberx119.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.20" class="ltx_text ltx_lst_identifier">its</span><span id="lstnumberx119.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.22" class="ltx_text ltx_lst_identifier">properties</span><span id="lstnumberx119.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.24" class="ltx_text ltx_lst_identifier">and</span><span id="lstnumberx119.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.26" class="ltx_text ltx_lst_identifier">how</span><span id="lstnumberx119.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.28" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx119.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.30" class="ltx_text ltx_lst_identifier">interact</span><span id="lstnumberx119.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.32" class="ltx_text ltx_lst_identifier">with</span><span id="lstnumberx119.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx119.34" class="ltx_text ltx_lst_identifier">it</span>.
</div>
<div id="lstnumberx120" class="ltx_listingline">5.<span id="lstnumberx120.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.2" class="ltx_text ltx_lst_identifier">Picking</span><span id="lstnumberx120.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.4" class="ltx_text ltx_lst_identifier">up</span><span id="lstnumberx120.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx120.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.8" class="ltx_text ltx_lst_identifier">thermometer</span><span id="lstnumberx120.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.10" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx120.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx120.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx120.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.16" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx120.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.18" class="ltx_text ltx_lst_identifier">monitor</span><span id="lstnumberx120.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.20" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx120.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.22" class="ltx_text ltx_lst_identifier">temperature</span><span id="lstnumberx120.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.24" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx120.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx120.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx120.28" class="ltx_text ltx_lst_identifier">substance</span>.
</div>
<div id="lstnumberx121" class="ltx_listingline">6.<span id="lstnumberx121.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.2" class="ltx_text ltx_lst_identifier">Using</span><span id="lstnumberx121.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx121.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.6" class="ltx_text ltx_lst_identifier">thermometer</span><span id="lstnumberx121.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.8" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx121.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.10" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx121.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.12" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx121.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.14" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx121.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.16" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx121.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.18" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx121.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.20" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx121.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.22" class="ltx_text ltx_lst_identifier">monitor</span><span id="lstnumberx121.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.24" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx121.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.26" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx121.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.28" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx121.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx121.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx121.32" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx122" class="ltx_listingline">7.<span id="lstnumberx122.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.2" class="ltx_text ltx_lst_identifier">Puring</span><span id="lstnumberx122.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx122.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.6" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx122.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.8" class="ltx_text ltx_lst_identifier">into</span><span id="lstnumberx122.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.10" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx122.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.12" class="ltx_text ltx_lst_identifier">container</span><span id="lstnumberx122.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.14" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx122.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.16" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx122.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.18" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx122.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.20" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx122.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.22" class="ltx_text ltx_lst_identifier">prepare</span><span id="lstnumberx122.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.24" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx122.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.26" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx122.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx122.28" class="ltx_text ltx_lst_identifier">cooling</span>.
</div>
<div id="lstnumberx123" class="ltx_listingline">8.<span id="lstnumberx123.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx123.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx123.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.6" class="ltx_text ltx_lst_identifier">container</span><span id="lstnumberx123.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.8" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx123.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.10" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx123.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.12" class="ltx_text ltx_lst_identifier">cooling</span><span id="lstnumberx123.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.14" class="ltx_text ltx_lst_identifier">device</span><span id="lstnumberx123.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.16" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx123.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.18" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx123.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.20" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx123.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.22" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx123.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.24" class="ltx_text ltx_lst_identifier">cool</span><span id="lstnumberx123.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx123.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx123.28" class="ltx_text ltx_lst_identifier">substance</span>.
</div>
<div id="lstnumberx124" class="ltx_listingline">9.<span id="lstnumberx124.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.2" class="ltx_text ltx_lst_identifier">Waiting</span><span id="lstnumberx124.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.4" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx124.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.6" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx124.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.8" class="ltx_text ltx_lst_identifier">period</span><span id="lstnumberx124.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.10" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx124.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.12" class="ltx_text ltx_lst_identifier">time</span><span id="lstnumberx124.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.14" class="ltx_text ltx_lst_identifier">after</span><span id="lstnumberx124.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.16" class="ltx_text ltx_lst_identifier">cooling</span><span id="lstnumberx124.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.18" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx124.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.20" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx124.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.22" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx124.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.24" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx124.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.26" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx124.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.28" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx124.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.30" class="ltx_text ltx_lst_identifier">allow</span><span id="lstnumberx124.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.32" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx124.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.34" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx124.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.36" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx124.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.38" class="ltx_text ltx_lst_identifier">change</span><span id="lstnumberx124.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx124.40" class="ltx_text ltx_lst_identifier">state</span>.
</div>
<div id="lstnumberx125" class="ltx_listingline">10.<span id="lstnumberx125.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.2" class="ltx_text ltx_lst_identifier">Repeatedly</span><span id="lstnumberx125.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.4" class="ltx_text ltx_lst_identifier">checking</span><span id="lstnumberx125.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx125.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.8" class="ltx_text ltx_lst_identifier">temperature</span><span id="lstnumberx125.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.10" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx125.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.12" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx125.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.14" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx125.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.16" class="ltx_text ltx_lst_identifier">SHOULD</span><span id="lstnumberx125.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.18" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx125.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.20" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx125.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.22" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx125.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.24" class="ltx_text ltx_lst_identifier">monitor</span><span id="lstnumberx125.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.26" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx125.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.28" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx125.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.30" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx125.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.32" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx125.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx125.34" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx126" class="ltx_listingline">11.<span id="lstnumberx126.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.2" class="ltx_text ltx_lst_identifier">Activating</span><span id="lstnumberx126.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx126.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.6" class="ltx_text ltx_lst_identifier">stove</span><span id="lstnumberx126.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.8" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx126.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.10" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx126.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.12" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx126.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.14" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx126.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.16" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx126.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.18" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx126.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.20" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx126.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.22" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx126.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.24" class="ltx_text ltx_lst_identifier">does</span><span id="lstnumberx126.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.26" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx126.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.28" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx126.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx126.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx126.32" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx127" class="ltx_listingline">12.<span id="lstnumberx127.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.2" class="ltx_text ltx_lst_identifier">Picking</span><span id="lstnumberx127.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.4" class="ltx_text ltx_lst_identifier">up</span><span id="lstnumberx127.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.6" class="ltx_text ltx_lst_identifier">unrelated</span><span id="lstnumberx127.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.8" class="ltx_text ltx_lst_identifier">items</span><span id="lstnumberx127.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.10" class="ltx_text ltx_lst_identifier">like</span><span id="lstnumberx127.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.12" class="ltx_text ltx_lst_identifier">a</span><span id="lstnumberx127.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.14" class="ltx_text ltx_lst_identifier">lighter</span><span id="lstnumberx127.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.16" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx127.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.18" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx127.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.20" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx127.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.22" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx127.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.24" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx127.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.26" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx127.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.28" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx127.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.30" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx127.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.32" class="ltx_text ltx_lst_identifier">does</span><span id="lstnumberx127.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.34" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx127.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.36" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx127.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.38" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx127.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx127.40" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx128" class="ltx_listingline">13.<span id="lstnumberx128.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.2" class="ltx_text ltx_lst_identifier">Moving</span><span id="lstnumberx128.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx128.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.6" class="ltx_text ltx_lst_identifier">unrelated</span><span id="lstnumberx128.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.8" class="ltx_text ltx_lst_identifier">rooms</span><span id="lstnumberx128.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.10" class="ltx_text ltx_lst_identifier">like</span><span id="lstnumberx128.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.12" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx128.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.14" class="ltx_text ltx_lst_identifier">workshop</span><span id="lstnumberx128.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.16" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx128.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.18" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx128.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.20" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx128.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.22" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx128.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.24" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx128.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.26" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx128.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.28" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx128.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.30" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx128.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.32" class="ltx_text ltx_lst_identifier">does</span><span id="lstnumberx128.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.34" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx128.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.36" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx128.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.38" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx128.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx128.40" class="ltx_text ltx_lst_identifier">task</span>.
</div>
<div id="lstnumberx129" class="ltx_listingline">14.<span id="lstnumberx129.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.2" class="ltx_text ltx_lst_identifier">Teleporting</span><span id="lstnumberx129.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.4" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx129.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.6" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx129.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.8" class="ltx_text ltx_lst_identifier">kitchen</span><span id="lstnumberx129.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.10" class="ltx_text ltx_lst_identifier">MAY</span><span id="lstnumberx129.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.12" class="ltx_text ltx_lst_identifier">BE</span><span id="lstnumberx129.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.14" class="ltx_text ltx_lst_identifier">NECESSARY</span><span id="lstnumberx129.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.16" class="ltx_text ltx_lst_identifier">for</span><span id="lstnumberx129.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.18" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx129.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.20" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx129.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.22" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx129.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.24" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx129.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.26" class="ltx_text ltx_lst_identifier">speeds</span><span id="lstnumberx129.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.28" class="ltx_text ltx_lst_identifier">up</span><span id="lstnumberx129.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.30" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx129.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.32" class="ltx_text ltx_lst_identifier">process</span><span id="lstnumberx129.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.34" class="ltx_text ltx_lst_identifier">of</span><span id="lstnumberx129.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.36" class="ltx_text ltx_lst_identifier">moving</span><span id="lstnumberx129.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.38" class="ltx_text ltx_lst_identifier">between</span><span id="lstnumberx129.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx129.40" class="ltx_text ltx_lst_identifier">rooms</span>.
</div>
<div id="lstnumberx130" class="ltx_listingline">15.<span id="lstnumberx130.1" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.2" class="ltx_text ltx_lst_identifier">Using</span><span id="lstnumberx130.3" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.4" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx130.5" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.6" class="ltx_text ltx_lst_identifier">thermometer</span><span id="lstnumberx130.7" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.8" class="ltx_text ltx_lst_identifier">multiple</span><span id="lstnumberx130.9" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.10" class="ltx_text ltx_lst_identifier">times</span><span id="lstnumberx130.11" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.12" class="ltx_text ltx_lst_identifier">on</span><span id="lstnumberx130.13" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.14" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx130.15" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.16" class="ltx_text ltx_lst_identifier">substance</span><span id="lstnumberx130.17" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.18" class="ltx_text ltx_lst_identifier">after</span><span id="lstnumberx130.19" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.20" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx130.21" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.22" class="ltx_text ltx_lst_identifier">reaches</span><span id="lstnumberx130.23" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.24" class="ltx_text ltx_lst_identifier">freezing</span><span id="lstnumberx130.25" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.26" class="ltx_text ltx_lst_identifier">point</span><span id="lstnumberx130.27" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.28" class="ltx_text ltx_lst_identifier">DOES</span><span id="lstnumberx130.29" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.30" class="ltx_text ltx_lst_identifier">NOT</span><span id="lstnumberx130.31" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.32" class="ltx_text ltx_lst_identifier">CONTRIBUTE</span><span id="lstnumberx130.33" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.34" class="ltx_text ltx_lst_identifier">to</span><span id="lstnumberx130.35" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.36" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx130.37" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.38" class="ltx_text ltx_lst_identifier">task</span><span id="lstnumberx130.39" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.40" class="ltx_text ltx_lst_identifier">as</span><span id="lstnumberx130.41" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.42" class="ltx_text ltx_lst_identifier">it</span><span id="lstnumberx130.43" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.44" class="ltx_text ltx_lst_identifier">does</span><span id="lstnumberx130.45" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.46" class="ltx_text ltx_lst_identifier">not</span><span id="lstnumberx130.47" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.48" class="ltx_text ltx_lst_identifier">progress</span><span id="lstnumberx130.49" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.50" class="ltx_text ltx_lst_identifier">the</span><span id="lstnumberx130.51" class="ltx_text ltx_lst_space"> </span><span id="lstnumberx130.52" class="ltx_text ltx_lst_identifier">task</span>.
</div>
</div>
<br class="ltx_break ltx_break">
<figcaption class="ltx_caption"><span class="ltx_tag ltx_tag_figure">Figure 12: </span>Meta-memory used in <span id="A2.F12.3.1" class="ltx_text ltx_font_smallcaps">Gen-Task</span> for a Freeze task.</figcaption>
</figure>
</section>
<section id="A3" class="ltx_appendix">
<h2 class="ltx_title ltx_title_appendix">
<span class="ltx_tag ltx_tag_appendix">Appendix C </span>More results</h2>

<div id="A3.p1" class="ltx_para ltx_noindent">
<p id="A3.p1.1" class="ltx_p">Full results for <span id="A3.p1.1.1" class="ltx_text ltx_font_smallcaps">CLIN</span> outperforming Reflexion is in <a href="#A3.T2" title="In Appendix C More results ‣ CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization" class="ltx_ref"><span class="ltx_text ltx_ref_tag">Table</span> <span class="ltx_text ltx_ref_tag">2</span></a>. For ScienceWorld benchmark, we exclude electricity tasks since they deviate from standard electrical conventions, prohibiting us from fairly using LLM agents. We choose the first 10 test variants for each 18 tasks selected. The full list of 18 tasks from the benchmark, with the number of test variants used in parentheses:</p>
</div>
<div id="A3.p2" class="ltx_para ltx_noindent">
<p id="A3.p2.1" class="ltx_p">grow-plant (10), identify-life-stages-1 (5), grow-fruit (10), measure-melting-point-known-substance (10), mendelian-genetics-unknown-plant (10), chemistry-mix-paint-secondary-color (9), freeze (9),
lifespan-longest-lived (10), inclined-plane-determine-angle (10), boil (9), use-thermometer (10), chemistry-mix (8), lifespan-shortest-lived (10), find-plant (10), find-living-thing (10), identify-life-stages-2 (4), mendelian-genetics-known-plant (10), inclined-plane-friction-named-surfaces (10).</p>
</div>
<div id="A3.p3" class="ltx_para ltx_noindent">
<p id="A3.p3.1" class="ltx_p">Short tasks have oracle lengths less than 37 steps (median), and Long tasks have oracle lengths more than equal to 37 steps.</p>
</div>
<div id="A3.p4" class="ltx_para ltx_noindent">
<p id="A3.p4.1" class="ltx_p">The map to the short names used for tasks in the paper:</p>
</div>
<div id="A3.p5" class="ltx_para ltx_noindent">
<p id="A3.p5.1" class="ltx_p">Temp: use-thermometer, measure-melting-point-known-substance; Pick&amp;Place: find-plant, find-living-thing; Chemistry: chemistry-mix, chemistry-mix-paint-secondary-color; Lifespan: lifespan-longest-lived, lifespan-shortest-lived; Biology: identify-life-stages-1, identify-life-stages-2, Boil; Freeze; Grow Plant, Grow Fruit; Force: inclined-plane-determine-angle; Friction: inclined-plane-friction-named-surfaces; Genetics: mendelian-genetics-known-plant, mendelian-genetics-unknown-plant.</p>
</div>
<figure id="A3.T2" class="ltx_table">
<div id="A3.T2.1" class="ltx_inline-block ltx_align_center ltx_transformed_outer" style="width:260.2pt;height:443.1pt;vertical-align:-0.0pt;"><span class="ltx_transformed_inner" style="transform:translate(8.5pt,-14.6pt) scale(1.07030881314069,1.07030881314069) ;">
<table id="A3.T2.1.1" class="ltx_tabular ltx_align_middle">
<tr id="A3.T2.1.1.1" class="ltx_tr">
<td id="A3.T2.1.1.1.1" class="ltx_td ltx_nopad_r ltx_border_tt"></td>
<td id="A3.T2.1.1.1.2" class="ltx_td ltx_nopad_l ltx_border_rr ltx_border_tt"></td>
<td id="A3.T2.1.1.1.3" class="ltx_td ltx_align_center ltx_border_r ltx_border_tt" colspan="2">Generative L. Agents</td>
<td id="A3.T2.1.1.1.4" class="ltx_td ltx_align_center ltx_border_tt" colspan="2">
<span id="A3.T2.1.1.1.4.1" class="ltx_text ltx_font_smallcaps">CLIN</span><span id="A3.T2.1.1.1.4.2" class="ltx_text ltx_font_bold"> (ours)</span>
</td>
</tr>
<tr id="A3.T2.1.1.2" class="ltx_tr">
<td id="A3.T2.1.1.2.1" class="ltx_td ltx_nopad_r ltx_align_left ltx_border_t">Task</td>
<td id="A3.T2.1.1.2.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr ltx_border_t">Type</td>
<td id="A3.T2.1.1.2.3" class="ltx_td ltx_align_center ltx_border_t">ReAct</td>
<td id="A3.T2.1.1.2.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">Reflexion</td>
<td id="A3.T2.1.1.2.5" class="ltx_td ltx_align_center ltx_border_t"><span id="A3.T2.1.1.2.5.1" class="ltx_text ltx_font_smallcaps">Base</span></td>
<td id="A3.T2.1.1.2.6" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t"><span id="A3.T2.1.1.2.6.1" class="ltx_text ltx_font_smallcaps">Adapt</span></td>
</tr>
<tr id="A3.T2.1.1.3" class="ltx_tr">
<td id="A3.T2.1.1.3.1" class="ltx_td ltx_nopad_r ltx_align_left ltx_border_t">Temp</td>
<td id="A3.T2.1.1.3.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr ltx_border_t">S</td>
<td id="A3.T2.1.1.3.3" class="ltx_td ltx_align_center ltx_border_t">7.2</td>
<td id="A3.T2.1.1.3.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">5.9</td>
<td id="A3.T2.1.1.3.5" class="ltx_td ltx_align_center ltx_border_t">25.2</td>
<td id="A3.T2.1.1.3.6" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t">14.3</td>
</tr>
<tr id="A3.T2.1.1.4" class="ltx_tr">
<td id="A3.T2.1.1.4.1" class="ltx_td ltx_nopad_r ltx_align_left">Temp</td>
<td id="A3.T2.1.1.4.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.4.3" class="ltx_td ltx_align_center">6.1</td>
<td id="A3.T2.1.1.4.4" class="ltx_td ltx_align_center ltx_border_r">28.6</td>
<td id="A3.T2.1.1.4.5" class="ltx_td ltx_align_center">53.2</td>
<td id="A3.T2.1.1.4.6" class="ltx_td ltx_nopad_r ltx_align_center">51.8</td>
</tr>
<tr id="A3.T2.1.1.5" class="ltx_tr">
<td id="A3.T2.1.1.5.1" class="ltx_td ltx_nopad_r ltx_align_left">Pick&amp;Place</td>
<td id="A3.T2.1.1.5.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.5.3" class="ltx_td ltx_align_center">26.7</td>
<td id="A3.T2.1.1.5.4" class="ltx_td ltx_align_center ltx_border_r">64.9</td>
<td id="A3.T2.1.1.5.5" class="ltx_td ltx_align_center">92.5</td>
<td id="A3.T2.1.1.5.6" class="ltx_td ltx_nopad_r ltx_align_center">100.0</td>
</tr>
<tr id="A3.T2.1.1.6" class="ltx_tr">
<td id="A3.T2.1.1.6.1" class="ltx_td ltx_nopad_r ltx_align_left">Pick&amp;Place</td>
<td id="A3.T2.1.1.6.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.6.3" class="ltx_td ltx_align_center">53.3</td>
<td id="A3.T2.1.1.6.4" class="ltx_td ltx_align_center ltx_border_r">16.4</td>
<td id="A3.T2.1.1.6.5" class="ltx_td ltx_align_center">55.0</td>
<td id="A3.T2.1.1.6.6" class="ltx_td ltx_nopad_r ltx_align_center">100.0</td>
</tr>
<tr id="A3.T2.1.1.7" class="ltx_tr">
<td id="A3.T2.1.1.7.1" class="ltx_td ltx_nopad_r ltx_align_left">Chemistry</td>
<td id="A3.T2.1.1.7.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.7.3" class="ltx_td ltx_align_center">51.0</td>
<td id="A3.T2.1.1.7.4" class="ltx_td ltx_align_center ltx_border_r">70.4</td>
<td id="A3.T2.1.1.7.5" class="ltx_td ltx_align_center">44.5</td>
<td id="A3.T2.1.1.7.6" class="ltx_td ltx_nopad_r ltx_align_center">44.4</td>
</tr>
<tr id="A3.T2.1.1.8" class="ltx_tr">
<td id="A3.T2.1.1.8.1" class="ltx_td ltx_nopad_r ltx_align_left">Chemistry</td>
<td id="A3.T2.1.1.8.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.8.3" class="ltx_td ltx_align_center">58.9</td>
<td id="A3.T2.1.1.8.4" class="ltx_td ltx_align_center ltx_border_r">70.7</td>
<td id="A3.T2.1.1.8.5" class="ltx_td ltx_align_center">56.7</td>
<td id="A3.T2.1.1.8.6" class="ltx_td ltx_nopad_r ltx_align_center">56.7</td>
</tr>
<tr id="A3.T2.1.1.9" class="ltx_tr">
<td id="A3.T2.1.1.9.1" class="ltx_td ltx_nopad_r ltx_align_left">Lifespan</td>
<td id="A3.T2.1.1.9.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.9.3" class="ltx_td ltx_align_center">60.0</td>
<td id="A3.T2.1.1.9.4" class="ltx_td ltx_align_center ltx_border_r">100.0</td>
<td id="A3.T2.1.1.9.5" class="ltx_td ltx_align_center">85.0</td>
<td id="A3.T2.1.1.9.6" class="ltx_td ltx_nopad_r ltx_align_center">100.0</td>
</tr>
<tr id="A3.T2.1.1.10" class="ltx_tr">
<td id="A3.T2.1.1.10.1" class="ltx_td ltx_nopad_r ltx_align_left">Lifespan</td>
<td id="A3.T2.1.1.10.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.10.3" class="ltx_td ltx_align_center">67.5</td>
<td id="A3.T2.1.1.10.4" class="ltx_td ltx_align_center ltx_border_r">84.4</td>
<td id="A3.T2.1.1.10.5" class="ltx_td ltx_align_center">70.0</td>
<td id="A3.T2.1.1.10.6" class="ltx_td ltx_nopad_r ltx_align_center">90.0</td>
</tr>
<tr id="A3.T2.1.1.11" class="ltx_tr">
<td id="A3.T2.1.1.11.1" class="ltx_td ltx_nopad_r ltx_align_left">Biology</td>
<td id="A3.T2.1.1.11.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">S</td>
<td id="A3.T2.1.1.11.3" class="ltx_td ltx_align_center">8.0</td>
<td id="A3.T2.1.1.11.4" class="ltx_td ltx_align_center ltx_border_r">8.0</td>
<td id="A3.T2.1.1.11.5" class="ltx_td ltx_align_center">10.0</td>
<td id="A3.T2.1.1.11.6" class="ltx_td ltx_nopad_r ltx_align_center">8.0</td>
</tr>
<tr id="A3.T2.1.1.12" class="ltx_tr">
<td id="A3.T2.1.1.12.1" class="ltx_td ltx_nopad_r ltx_align_left">Boil</td>
<td id="A3.T2.1.1.12.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.12.3" class="ltx_td ltx_align_center">3.5</td>
<td id="A3.T2.1.1.12.4" class="ltx_td ltx_align_center ltx_border_r">4.2</td>
<td id="A3.T2.1.1.12.5" class="ltx_td ltx_align_center">7.0</td>
<td id="A3.T2.1.1.12.6" class="ltx_td ltx_nopad_r ltx_align_center">15.2</td>
</tr>
<tr id="A3.T2.1.1.13" class="ltx_tr">
<td id="A3.T2.1.1.13.1" class="ltx_td ltx_nopad_r ltx_align_left">Freeze</td>
<td id="A3.T2.1.1.13.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.13.3" class="ltx_td ltx_align_center">7.8</td>
<td id="A3.T2.1.1.13.4" class="ltx_td ltx_align_center ltx_border_r">7.8</td>
<td id="A3.T2.1.1.13.5" class="ltx_td ltx_align_center">10.0</td>
<td id="A3.T2.1.1.13.6" class="ltx_td ltx_nopad_r ltx_align_center">10.0</td>
</tr>
<tr id="A3.T2.1.1.14" class="ltx_tr">
<td id="A3.T2.1.1.14.1" class="ltx_td ltx_nopad_r ltx_align_left">GrowPlant</td>
<td id="A3.T2.1.1.14.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.14.3" class="ltx_td ltx_align_center">9.1</td>
<td id="A3.T2.1.1.14.4" class="ltx_td ltx_align_center ltx_border_r">7.3</td>
<td id="A3.T2.1.1.14.5" class="ltx_td ltx_align_center">10.2</td>
<td id="A3.T2.1.1.14.6" class="ltx_td ltx_nopad_r ltx_align_center">11.1</td>
</tr>
<tr id="A3.T2.1.1.15" class="ltx_tr">
<td id="A3.T2.1.1.15.1" class="ltx_td ltx_nopad_r ltx_align_left">GrowFruit</td>
<td id="A3.T2.1.1.15.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.15.3" class="ltx_td ltx_align_center">18.6</td>
<td id="A3.T2.1.1.15.4" class="ltx_td ltx_align_center ltx_border_r">13.0</td>
<td id="A3.T2.1.1.15.5" class="ltx_td ltx_align_center">35.9</td>
<td id="A3.T2.1.1.15.6" class="ltx_td ltx_nopad_r ltx_align_center">71.6</td>
</tr>
<tr id="A3.T2.1.1.16" class="ltx_tr">
<td id="A3.T2.1.1.16.1" class="ltx_td ltx_nopad_r ltx_align_left">Biology</td>
<td id="A3.T2.1.1.16.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.16.3" class="ltx_td ltx_align_center">27.7</td>
<td id="A3.T2.1.1.16.4" class="ltx_td ltx_align_center ltx_border_r">2.6</td>
<td id="A3.T2.1.1.16.5" class="ltx_td ltx_align_center">70.0</td>
<td id="A3.T2.1.1.16.6" class="ltx_td ltx_nopad_r ltx_align_center">81.0</td>
</tr>
<tr id="A3.T2.1.1.17" class="ltx_tr">
<td id="A3.T2.1.1.17.1" class="ltx_td ltx_nopad_r ltx_align_left">Force</td>
<td id="A3.T2.1.1.17.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.17.3" class="ltx_td ltx_align_center">40.5</td>
<td id="A3.T2.1.1.17.4" class="ltx_td ltx_align_center ltx_border_r">50.6</td>
<td id="A3.T2.1.1.17.5" class="ltx_td ltx_align_center">53.5</td>
<td id="A3.T2.1.1.17.6" class="ltx_td ltx_nopad_r ltx_align_center">100.0</td>
</tr>
<tr id="A3.T2.1.1.18" class="ltx_tr">
<td id="A3.T2.1.1.18.1" class="ltx_td ltx_nopad_r ltx_align_left">Friction</td>
<td id="A3.T2.1.1.18.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.18.3" class="ltx_td ltx_align_center">44.0</td>
<td id="A3.T2.1.1.18.4" class="ltx_td ltx_align_center ltx_border_r">100.0</td>
<td id="A3.T2.1.1.18.5" class="ltx_td ltx_align_center">56.5</td>
<td id="A3.T2.1.1.18.6" class="ltx_td ltx_nopad_r ltx_align_center">72.5</td>
</tr>
<tr id="A3.T2.1.1.19" class="ltx_tr">
<td id="A3.T2.1.1.19.1" class="ltx_td ltx_nopad_r ltx_align_left">Genetics</td>
<td id="A3.T2.1.1.19.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.19.3" class="ltx_td ltx_align_center">25.7</td>
<td id="A3.T2.1.1.19.4" class="ltx_td ltx_align_center ltx_border_r">50.9</td>
<td id="A3.T2.1.1.19.5" class="ltx_td ltx_align_center">77.4</td>
<td id="A3.T2.1.1.19.6" class="ltx_td ltx_nopad_r ltx_align_center">100.0</td>
</tr>
<tr id="A3.T2.1.1.20" class="ltx_tr">
<td id="A3.T2.1.1.20.1" class="ltx_td ltx_nopad_r ltx_align_left">Genetics</td>
<td id="A3.T2.1.1.20.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.20.3" class="ltx_td ltx_align_center">16.8</td>
<td id="A3.T2.1.1.20.4" class="ltx_td ltx_align_center ltx_border_r">23.7</td>
<td id="A3.T2.1.1.20.5" class="ltx_td ltx_align_center">62.3</td>
<td id="A3.T2.1.1.20.6" class="ltx_td ltx_nopad_r ltx_align_center">92.6</td>
</tr>
<tr id="A3.T2.1.1.21" class="ltx_tr">
<td id="A3.T2.1.1.21.1" class="ltx_td ltx_nopad_r ltx_border_t"></td>
<td id="A3.T2.1.1.21.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr ltx_border_t">S</td>
<td id="A3.T2.1.1.21.3" class="ltx_td ltx_align_center ltx_border_t">37.6</td>
<td id="A3.T2.1.1.21.4" class="ltx_td ltx_align_center ltx_border_r ltx_border_t">49.9</td>
<td id="A3.T2.1.1.21.5" class="ltx_td ltx_align_center ltx_border_t">54.7</td>
<td id="A3.T2.1.1.21.6" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_t"><span id="A3.T2.1.1.21.6.1" class="ltx_text ltx_font_bold">62.8</span></td>
</tr>
<tr id="A3.T2.1.1.22" class="ltx_tr">
<td id="A3.T2.1.1.22.1" class="ltx_td ltx_nopad_r"></td>
<td id="A3.T2.1.1.22.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_rr">L</td>
<td id="A3.T2.1.1.22.3" class="ltx_td ltx_align_center">21.5</td>
<td id="A3.T2.1.1.22.4" class="ltx_td ltx_align_center ltx_border_r">28.9</td>
<td id="A3.T2.1.1.22.5" class="ltx_td ltx_align_center">42.5</td>
<td id="A3.T2.1.1.22.6" class="ltx_td ltx_nopad_r ltx_align_center"><span id="A3.T2.1.1.22.6.1" class="ltx_text ltx_font_bold">61.6</span></td>
</tr>
<tr id="A3.T2.1.1.23" class="ltx_tr">
<td id="A3.T2.1.1.23.1" class="ltx_td ltx_nopad_r ltx_border_bb"></td>
<td id="A3.T2.1.1.23.2" class="ltx_td ltx_nopad_l ltx_align_center ltx_border_bb ltx_border_rr">All</td>
<td id="A3.T2.1.1.23.3" class="ltx_td ltx_align_center ltx_border_bb">29.6</td>
<td id="A3.T2.1.1.23.4" class="ltx_td ltx_align_center ltx_border_bb ltx_border_r">39.4</td>
<td id="A3.T2.1.1.23.5" class="ltx_td ltx_align_center ltx_border_bb">48.6</td>
<td id="A3.T2.1.1.23.6" class="ltx_td ltx_nopad_r ltx_align_center ltx_border_bb"><span id="A3.T2.1.1.23.6.1" class="ltx_text ltx_font_bold">62.2</span></td>
</tr>
</table>
</span></div>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 2: </span>Comparing <span id="A3.T2.4.1" class="ltx_text ltx_font_smallcaps">CLIN</span> with baselines for <span id="A3.T2.5.2" class="ltx_text ltx_font_bold">adaptation</span> </figcaption>
</figure>
<div class="ltx_pagination ltx_role_newpage"></div>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2310.10133" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2310.10134" class="ar5iv-text-button ar5iv-severity-warning">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2310.10134">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2310.10134" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2310.10135" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Wed Feb 28 00:40:41 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2310.10134
Language agents have shown some ability to interact with an external environment, e.g., a virtual
world such as ScienceWorld, to perform complex tasks, e.g., growing a plant, without the startup
costs of reinforcement learning. However, despite their zero-shot capabilities, these agents
to date do not continually improve over time, beyond performance refinement on a specific task.
Here we present CLIN, the first language-based agent to achieve this,
so that it continually improves over multiple trials, including when both the
environment and task are varied, and without requiring parameter updates.
Our approach is to use a persistent, dynamic, textual memory,
centered oncausal abstractions(rather than general “helpful hints”) ,
that is regularly updated after each trial so that the agent gradually learns useful
knowledge for new trials. In the ScienceWorld benchmark, CLIN is able to continually improve on repeated trials on the same task and environment,
outperforming state-of-the-art reflective language agents like Reflexion by 23 absolute points.
CLIN can also transfer its learning to new environments (or new tasks) , improving its zero-shot performance
by 4 points (13 for new tasks)  and can further improve performance there through continual memory updates, enhancing
performance by an additional 17 points (7 for new tasks) .
This suggests a new architecture for agents built on frozen models that can still
continually and rapidly improve over time.
abstract
CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization
Abstract
2310.10134
1IntroductionLarge language models (LLMs)  have been increasingly used to interact with external
environments (e.g., simulated worlds)  as goal-driven agents. However, it has been
challenging for these language agents to efficiently learn from trial-and-error
as traditional reinforcement learning methods require extensive training samples and
expensive model fine-tuning. More recently, new techniques have appeared in which
an agent reflects on its own past experience solving a task in a particular environment,
and generates language-based insights to help it retry the task, e.g., Reflexion.
Such methods have the advantage of not requiring parameter updates (particularly useful given
the growing popularity of frozen language models) . However, the style of such insights plays
a crucial role in performance, and not all insights improve generalization performance. For example,
a specific insight such as "In the next trial, I will go to desk 1 and find the lamp"may have limited value (or even hurt)  for a different environment
or task.Figure 1:CLIN creates (Trial1)  or adapts (Trial2+)  amemory of causal abstractionsto help in future trials by reflecting on the last trial and current memory. It does this using a suitably prompted LLM to generate the updated memory (Section3.2) . Here, reflecting on Trial1, CLIN notes in memory that going to the kitchen helped with finding seeds, enabling it to find the seeds faster in Trial2. From there, it also learns that moving the seeds to the pot helped plant the seeds. To further generalize across episodes (sequences of trials, right figure)  for use in new environments, CLIN generates a summary (“meta-memory”)  of the best (starred)  memories from each prior episode, here generating the generalization that moving to different rooms helps finding objects (Section3.3) Our goal is a system that will continually improve over time, both while attempting
the same task in the same environment, and across different tasks and environments.
Our approach builds on prior work on reflection in two ways: First, we conjecture that a
specificstyleof insight will be useful, namely one that capturescausal abstractionsabout agent’s actions, e.g., “opening doors may be necessary for
movement between rooms”. Causal abstractions can potentially help the agent decide
which action to take in the future, and can be viewed as a kind of action model
learning, but placed in the modern context of language models.
Second, we maintain these abstractions in acontinually
evolving, dynamic memory, which is regularly updated as the agent gains experience,
allowing useful causal knowledge to persist (and unhelpful knowledge to be dropped) 
over time and between tasks and environments, as illustrated in Figure1.We operationalize and evaluate this approach in a memory-augmented language agent called
CLIN (continuallearning frominteractions) . CLIN is an agent that operates in
ScienceWorld, a virtual, text-based environment in which an agent is
tasked with science-based goals, e.g., boiling a liquid, growing a plant. We find that
CLIN is able to rapidly learn about the environment and its action vocabulary and continually improve on repeated trials on the same task and environment,
outperforming state-of-the-art (SOTA)  reflective language agents like Reflexion by 23 points.
CLIN can also transfer its learning to new environments (or tasks) , improving its zero-shot performance
by 4 (13 for new tasks)  points and can further improve performance through continual memory updates, enhancing
performance by an additional 17 (7 for new tasks)  points. Our contributions are as follows:•For memory-based language agents, we show that memory of
causal abstractions is effective at helping the agents learn over an extended
period and in varying conditions.•We describe and evaluate CLIN, an architecture for a novel nonparametric learning paradigm. We
show that CLIN learns faster than prior systems and generalizes better
to new tasks and new environments, achieving state-of-the-art.Overall, this work suggests that a dynamically maintained memory, centered
around causal knowledge, is a promising way forward for agents built
on frozen models to continually improve over time.
introduction
CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization
1Introduction
2310.10134
2Related WorkThere is a long literature of work on agents that can navigate complex environments.
A common approach is to use reinforcement learning (RL) , e.g., DRRN,
KG-A2C,
CALM,
where agents learn a task over repeated trials. However, while effective,
such agents typically require a large number of trials to learn and have trouble
adapting to unexpected changes in the test environment. More recently,demonstrated AdA, an agent that could rapidly adapt to open-ended
novel 3D problems, using meta-reinforcement learning, essentially being able
to change its policy on the fly. However, AdA required vast amounts of
pretraining, and this skill was still limited to the style of environments and
problems seen in pretraining.Recently, LLMs have provided a new tool for building
goal-directed agents. Given a linguistic description of the world state,
a task, and a history, the LLM can be prompted to suggest next actions
to take to achieve a goal, exploiting their wealth of semantic knowledge
about the world and requiring little training, e.g., SayCan,
ReAct, and more recently SwiftSage, which
combines a supervised agent and a deliberative agent together. However, while
performing reasonably with little training data, such agents are unable
to learn and adapt from experience.Two recent systems have demonstrated how a frozen-model-based agent could
improve at a task. Voyageroperates in the world of Minecraft,
growing a (code-based)  skill library from rich feedback of its failures.
Reflexionimproves at a task byreflectingon a failed attempt at that task and devise a new plan that accounted for
that mistake, used in the subsequent prompt to retry the task. While Reflexion did not have a long-term memory,
and its reflections were task- and environment-specific,
e.g., “In the next trial, I will go to desk 1 and find the lamp.”, we take
inspiration from it to build an agent,CLIN, which continually maintains and
adapts a long-term, persistent memory of reflections, useful across
different trials, tasks, and environments.More generally, others have found that a memory of useful learnings can be
used to improve frozen LLM behavior, e.g., in QA, or for modeling social behavior. We apply this finding to
goal-directed agents.Finally, we note that thecontentof experiential memory is also important. Specifically,CLINlearns a memory ofcausal abstractions, which
can be seen as learning a linguistic form of action model, describing the causal
effects of actions. While there has been substantial work in the
planning community of learning action models in a fully formalized context,CLINloosely applies this idea in the linguistic world of LLM agents.
body
CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization
2Related Work
2310.10134
3ApproachProblem Formulation.Sequential decision-making tasks require agents to repeatedly observe and then act in an environment until they accomplish a specific goal. At a high level, this can be accomplished by developing beliefs about the world, acting on the environment based on those beliefs, and then updating one’s beliefs based on the observed outcome. Here, we investigate constructing an agent that can continually update its beliefs through interaction and observation while exploiting its past experience toward solving unseen parametric variations of tasks.Setup.We investigate our continual learning agents in simulated environments. Our environments are modeled in a high-fidelity text-based simulator, where both actions and observations are expressed in natural language. Let’s define the task space to be, a collection of partially observable Markov Decision Processes (POMDPs)  that can be executed in a set of environment configurations. Each taskhas an initial state and a desired winning state, which vary depending on the environment.Our setup allows an agent to attempt a task several times; each time is denoted as atrial,, which consists of a total ofsteps. Each step comprises an action by the agent () , and in response, the simulator returns the result of that action in the form of an observation ()  and a reward () . A collection oftrials is called anepisode. The environment gets reset when the task reaches an end state (such as completing, failing, or timing out) . In our continual learning setup, the agent retains its memory across trials/episodes, reaping the benefits of continued interaction with the environment.3.1CLIN: A Generative AgentContinuallyLearning fromINteractionsTo act in the world, CLIN uses three modules: amemory, acontroller, and anexecutor, illustrated in Figure2and which we now describe.
Learning then occurs using a fourth module, amemory generator, to generate an updated memory after each trial.Figure 2:The architecture of CLIN. Acontrollertakes the current task, retrievals from memory, and the trial so far, to generate the next goal to achieve. Theexecutorthen converts this to a valid action to perform towards that goal. The simulator then performs the action and returns an observation of that action’s effect. Memory is updated at the end of each trial by thememory generator(Section3.2) .Memory.CLIN’s memory ()  is a persistent, dynamic collection of NL sentences that express causal abstractions, generated by reflecting on past experiences in order to help the agent perform better in the future. This generation process is described shortly in Section3.2.
For example, having an explicit causal insight, “opening the fridge isnecessaryto access apple juice”, learned from past experiences, can reduce the action search space forCLINwhile looking for “apple juice” in the same environment in future trials. To aid in continual learning, the memory also capturesmistakesmade by the agent in previous trials, similar to reflective agents explored in recent work, noting actions that failed to contribute to a task.Controller.The role of the controller is to generate the next goal to pursue in service of the task.
In CLIN it is a frozen LLM, whose prompt includes the currenttask, e.g., “convert water into steam”, retrievals from thecurrent memory, and thetrial so far(the sequence of goal-action-observation triples {}) , and is prompted to output the nextgoalto pursue, e.g., “find water”. Memory items are retrieved using both the task instruction and the trial history. The controller first selects one or more memory items given the current state and if they are useful for the next action to progress in the task. After that, it appends the learning, if selected, in context to generate a goal, otherwise the goal is generated based on the trial history (see full prompt inFigure6) .Executor.The role of the executor is to convert the generated goalinto a validactionthat can be executed in the environment in pursuit of that goal.
Again a (frozen)  LLM is used, whose prompt includes the goal, the trial so far, and all the possible actions that can be performed in the
current state (provided by the simulator, as is standard practice in current generative agent research) .
The list of possible actions is expressed as possible action templates and available objects that can instantiate them, rather than
a combinatorially large enumeration of possible actions. The model is then prompted to generate a candidate action to perform (see prompt inFigure6) . Finally, CLIN checks this
candidate action is one of the valid actions. If it is not, it finds the most similar valid action using the pre-trained embeddings from thesentence-transformermodel. If the top-ranked valid action has a similarity score greater than a threshold (here, 0.9, chosen as a hyperparameter) , the action is selected. Otherwise, we perform iterative refinementby suffixing the context with feedback that the generated candidate action is not executable.
This allows the executor to retry the generation for up to a maximum number of tries (here, 5) .Finally, upon executing the action,CLINreceives a partial next state, as anobservation, from the simulator and the reward () . Rewards are nominally given by the simulator for achieving either major task subgoals (e.g., finding water, for the boiling task) , or minor and optional subgoals (e.g., being in the kitchen, for the boiling task) . Rewards are sparse and generally only supplied after the completion of a task subgoal. A snapshot of a full trial is given in lines 4-10 in Algorithm1.Note that CLIN does not make use of any gold data to identify goals and memories. Rather, we expectCLINto perform a balanced act of exploration-exploitation by interacting, learning, and adapting to unseen tasks or environment configurations—a key difference from few-shot generative agents by previous work.1:procedureAdaptation(Task:, Env:, Memory:) :2:Initialize Memory:3:fordo:4:Intialize Trial,5:whilemax. steps or task not completedo:6:Controller(,,,) 7:Executor(, admissible actions) 8:Simulator(,) 9:+ (,,,) 10:Final reward=11:memory-generator(,,) 12:13:procedureGeneralization(Task:, Env:, past/) 14:crucial-memories(past/) 15:meta-memory(,) 16:Adaptation(,,) Algorithm 1Continual Learning with CLINFigure 3:(left) CLIN’s continual learning algorithm. (right)  Example causal abstractions.3.2Memory: A collection of Adaptive Causal AbstractionsAt the end of each trial (completion or failure) , CLIN uses amemory generatorto create or update its memory.
The memory generator is a (frozen)  LLM prompted to reflect on the current trial and memory, and generate a new memory
of insights in the form of (English sentences expressing)  usefulcausal abstractions, as we now describe.Learning about state transitions is essential for sequential decision-making tasks, which can be manifested by knowing 1)  actions enabling desired state transitions, 2)  actions producing undesired or no change in states, and 3)  state transitions that contribute to the task progress. To generate these kinds of knowledge, the generator is prompted to generate insights in a particular syntax (see prompt inFigure7) . To capture good actions enabling desired changes and helpful state transitions, we use the template “X is NECESSARY to Y”, and to capture contrastive examples of unsuitable actions and state transitions, we employ “X DOES NOT CONTRIBUTE to Y”, as depicted inFigure5, whereX,Yare related to actions. These abstractions are functionally analogous to hindsight experience replay,
obtained fromCLIN’s past self-explorations.While useful insights can be abstracted from the trials,CLIN’s exploration can be limited, especially in the early trials, given an incredibly large action space. Hence, incomplete exploration can pose varying degrees of uncertainty on extracted insights. To capture this, we also include a measure of uncertainty in each abstraction by either of the two linguistic variations in their surface forms: “X may ...” to denote moderate to high uncertainty, and “X should ...” to indicate low uncertainty (SeeFigure5) . In the course of continual learning, asCLINgathers experience, we expect the level of uncertainty to change depending on the frequency of their use and their fitness to the task.Updating Memory Across Trials.CLINcontinually attempts to solve a task in an environment for multiple trials (in sum, an episode) . To update the memory after each trial within an episode, the memory generator is prompted with the most recent trial (a sequence of (,,)  tuples and the final reward111The reward is converted to NL feedback for a LLM using 7 simple rules, e.g., “if score ¿= 0 and score ¡ 20 then feedback =”The agent performed poorly and made some progress but not enough to solve the task.”) ,
and the memories from the three most recent trials. It is then prompted to generate an updated memory, namely a new list of semi-structured causal abstractions in the forms described above, for use in the next trial. Although we do not specify a maximum size for the memory, we observe that size of the generated memory (i.e., the number of causal abstractions generated)  is far less than the number of actions executed in the trial, indicating the memory-generator additionally performs a saliency-based pruning to keep only important insights based on the success of the trial, as indicated by the final rewardat the end of the trial.3.3Meta-Memory for Better GeneralizationUpdating memory based on past insights and the current trial to influence future trials for the same task in the current environment configuration during test-time adaptation. However, to generalize across tasks or environment configurations, the memory needs to contain more generalized causal abstractions than memories used across trials in an episode. We call this asmeta-memory, abstracted across multiple episodes of solving different tasks in different environment configurations to be applicable in future episodes.Auto-curriculum Selection.Before we generate the meta-memory, it is important to choose memories extracted from the best trials from previous episodes because random sampling may not benefitCLINin zero-shot generalization. Following the prioritized level replay scheme, we choose the most successful trial per episode and retrieve memories abstracted from those trials with a fixed archive of size 10, a hyperparameter.Generating Meta-Memory.The goal of the meta-memory is to helpCLINgeneralize to unseen tasks and/or environments. While we keep the format of the causal abstractions the same as memories generated across trials, the prompt to generate the meta-memory is different than those used for generating per-trial memory. When the new memory is to be used for thesametask but in adifferentenvironment, the prompt instructs for a meta-memory helpful “to solve the same task in a new environment configuration” given the target task description with an expectation that meta-memory abstractions will entail generic causal insights about the task irrespective of environment configurations (seeFigure8) . Similarly, when the new memory is to be used for adifferenttask, the prompt is modified accordingly to reflect this (Figure9) . Along with the target task description for better memory generation, each past memory selected to generate the meta-memory is attached to the final rewards for the associated trials, allowing the generator to combine insights across episodes and assign the levels of uncertainty using the evidence of success.
body
CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization
3Approach
2310.10134
4Results and AnalysisExperimental Setup.Test-time adaptation and generalization via continual learning require a variety of complex tasks and environment configurations to allow an agent to explore, learn latent causal insights from interactions, and exploit them in the future. We choose ScienceWorld, a text-based interactive environment requiring complex interactive reasoning processes to solve a plethora of science-theory-based tasks spanning several diverse classes (e.g., thermodynamics, genetics, friction, etc.) . The virtual space presents 10 sub-places:
foundry, greenhouse, outside area, an art studio, workshop, kitchen, living room, bedroom, bathroom, and a hallway connecting inside rooms. The presence of several objects, their individual states, and action templates renders the search space intractable for any agent. ScienceWorld presents strikingly different environment configurations across task types, making it a rich testbed for evaluating adaptation and generalization. ScienceWorld tasks are partitioned into Short (S) , e.g.,pick & placeand Long (L) , e.g.,grow plant, tasks based on the number of required actions to succeed.Here, we define our setups for zero-shot adaptation (Adapt)  and generalization (Gen-EnvandGen-Task) . For all setups, we testCLINand competing baselines on 18 tasks (two task instances from 9 classes)  in several environment configurations from the test split of the ScienceWorld benchmark resulting in a total of 164 task-environment combinations unless stated otherwise. We evaluate based on the final reward score provided by the ScienceWorld simulator.Adapt: This setup focuses onCLIN’s ability to adapt to a task by attempting it for several trials in the same environment configuration. Most importantly,CLINinitializes with an empty memory at the beginning of the first trial and generates memory at the end of each trial. While the environment gets reset at the trial boundary,CLIN’s memory continues to be updated, capturing informative causal abstractions pertaining to both successful and failed actions. Here, we compare with Reflexion, a SOTA, however,CLINdiffers from Reflexion by how the memory is abstracted.Gen-Env: In this setup, we focus onCLIN’s ability to transfer its learning from past experiences to solve tasks in an unseen environment. For a task, we runCLINfor 10 different (train)  environment settings (with varying objects and starting locations)  and then create meta-memories from its exploration to solve the same task in an unseen (test)  environment. Here, we compareCLINwith RL methods
DRRN, KG-A2C, and CALMtrained on all (large)  training variations with simulator reward and Generative Language agents, SayCan, ReAct,
and Reflexion, prompted with few-shot demonstrations.Gen-Task: In this setup, we focus onCLIN’s ability to transfer its learning from past experiences to solve a new task in the same environment. For an environment, we runCLINfor to solve a taskand then condense its learning to solve a novel taskin the environment. We took all test examples where we have a different task defined in the same environment configuration.suggests that transferring learning from a random task can be very hard; hence we couple tasks that are related (revolve around overlapping task-critical objects/locations such water, kitchen) , such asboilandfreezeto measure transfer learning from one to the other. This is a novel setup where we do not have any off-the-shelf baselines. However, here, we compare againstCLIN-Base, a strong baseline agent.Gen-Adapt(G+A) : IfCLIN, inGen-EnvorGen-Tasksetting, does not successfully complete the new task, it can continue learning and retrying that task. We refer to this setup asGen-Adapt.CLINcan use any instruction-tuned LLMas part of the controller, executor, and memory generator. In this paper, we usegpt-4, the same as our generative agent baselines.CLIN’sAdaptimprovementsType#trials to%ep.success () improv.S3.329.2L3.237.2All3.333.2(c) Figure 4:Rapid task adaptation withCLIN.(a) Example tasks whereCLINimproves scores across trials. ForCLIN, Trial-0 is theBase, Trial-4 is theAdapt.(b) Comparison ofCLINwith Reflexion.(c) CLINimproves fromBasetoAdapt(full results inAppendixC) .4.1CLINExhibits Rapid Task AdaptationFigure4a demonstrates two example trends whereCLINlearns from its own prior attempts (Adapt)  and gets better at solving a given task. Apart from length, the difficulty level of a task also depends on the environment configuration (hence, variance across environment configurations for each task) .CLINquickly adapts to a short task,Pick & Place, solving it in its 4th attempt, whereas for a longer task,Grow Fruit, it is not solved after 5th (max)  attempts.
Furthermore,Figure4a depicts,CLINbecomes more efficient in later trials by solving the tasks with a lower number of (average)  steps. Figure4c shows an average number of attempts222DuringAdapt,CLINtries up to 5 trials. If it solves a task with a score of 100, it stops retrying.forCLINto solve a task and % episodes per task where scores improved compared to its own first trial.Next, we compareCLINwith Reflexion, the reflective SOTA agent, inFigure4b.CLINalready starts off with a stronger base performance (see discussion in4.3) , however,CLIN’s  relative improvement inAdaptis significantly stronger than Reflexion’s gain from its base agent ReAct. Furthermore,CLIN’s relative improvement is higher for longer tasks.
This can be attributed toCLIN’s persistent memory, which gets refined over past trials, whereas Reflexion may fall short of collecting useful learnings from earlier trials as it only focuses on the current trial for its reflections (hence not long-term) . Furthermore,CLINaccumulates both useful (for the task)  and harmful (for the task)  causal learnings, whereas Reflexion only learns from its mistakes, lacking comprehensive learning.RL MethodsGenerative Language AgentsCLIN(ours) TaskTypeDRRNKGA2CCALMSayCanReActReflexionBaseGen-EnvG+ATemp1S6.66.01.026.47.25.925.215.713.8Temp2S5.511.01.08.06.128.653.249.758.2Pick&Place1S15.018.010.022.926.764.992.559.2100.0Pick&Place2S21.716.010.020.953.316.455.0100.0100.0Chemistry1S15.817.03.047.851.070.444.542.251.7Chemistry2S26.719.06.039.358.970.756.785.693.3Lifespan1S50.043.06.080.060.0100.085.065.0100.0Lifespan2S50.032.010.067.567.584.470.075.090.0Biology1S8.010.00.016.08.08.010.032.032.0BoilL3.50.00.033.13.54.27.04.416.3FreezeL0.04.00.03.97.87.810.08.910.0GrowPlantL8.06.02.09.99.17.310.210.911.2GrowFruitL14.311.04.013.918.613.035.970.894.5Biology2L21.05.04.020.927.72.670.042.885.6ForceL10.04.00.021.940.550.653.570.0100.0FrictionL10.04.03.032.344.0100.056.570.094.0Genetics1L16.811.02.067.525.750.977.484.5100.0Genetics2L17.011.02.059.516.823.762.361.4100.0S22.119.15.236.537.649.954.758.371.0L11.26.21.929.221.528.942.547.168.0All16.712.73.632.929.639.448.652.769.5Table 1:ComparingCLINwith baselines forgeneralization across unseen environments4.2CLINOutperforms SOTA, Generalizing to Novel Environments and TasksGeneralizing to new environments.Table1comparesCLINwith baselines that learn from training environmental variants for a task to improve its performance in a novel environment333Baseline numbers are derived from
Table 1 in. Language agents (includingCLIN)  that use NL feedback from the ScienceWorld (e.g., “Door to the kitchen is closed”)  perform significantly better compared to RL methods that purely rely on (sparse)  numeric rewards from the environment to learn a policy.
We observe a positive generalization effect inGen-Env(average 4 point gain)  compared toBasewhereCLINtries to solve the tasks zero-shot. With a strongBaseperformance,CLINbeats all baselines in generalization performance. Furthermore, inG+A,CLINshows a substantial 16 additional improvement, beating the SOTA reflective agent by 23 points.
Figure5a additionally shows trend of improvement compared to whenCLINdoes not start with a meta-memory. Meta-memory helpsCLINwith a stronger start thanBase(52.7 vs. 48.6) , with a continued gain in scores till the end of Trial-4 (G+A: 69.5 vs.Adapt: 62.2) . The stronger start forCLINwith meta-memory also results in fewer steps to solve a task.
Unlike imitation learning-based agents, TDTand SwiftSage,CLIN(and most baselines)  does not use any gold trajectories. Refining its memory only from self-generated trajectories,CLINoutperforms TDT on all 18 tasks and SwiftSage on 8/18 (mostly long)  tasks.Generalizing to new tasks.Mirroring trends fromGen-Env,CLINdemonstrates strong transfer learning to new tasks with 13-point improvement over itsBaseperformance, being better at 38.8% of times (Figure5c) . The improvement attributes to critical learning about the environment (“apple juice is in the fridge”, required for both boiling and freezing it) , leading to improvement in previously low-performing tasks in bothAdaptandGen-Envsetups. This transfer learning inGen-TaskandG+AhelpsCLINto solve the tasks with a lesser number of steps444# steps inFigure5a,b are normalized between 0-1, 1 being maximum # steps allowed for a task.and achieve higher rewards.4.3DiscussionImportance of memory structure.CLINextracts causal abstractions structured around ‘necessary’ and ‘does not contribute’ relations. As an ablation study, we modified our memory generator to generate free-form advice for future trials (without any constraint on their formats) .
We find that the average reward drops by 6 points (in 10% cases compared toCLIN)  when using the unstructured memory, indicating the usefulness of causal abstractions, as shown inFigure5d.SuperiorBaseperformance.Figure4depicts a superiorBaseperformance forCLINthan the final performance of both ReAct and Reflexion despite using the same underlying LLM (here,gpt-4) . We find if we ablate for the controller module inCLIN, responsible for generating a goal before outputting the next action,CLIN’sBaseperformance drops in 44% cases. With an 18 point drop in average reward (seeFigure5d) , Abl-Contoller-Baseversion ofCLINbecomes equivalent to ReAct, the base agent for Reflexion, demonstrating the importance of controller even inBasesetup.A qualitative example.Figure3depicts how memory items get refined during task adaptation and for generalization for a taskboil.Env2has a working stove, whereas inEnv1, the stove is broken, but a lighter is available as an alternative. With a number of trials in these environments,CLINlearns how to use these two devices to generate heat. In an unseen environment with a broken stove,CLINquickly receives a positive reward by using a lighter to heat a substance. While insights within an episode are often specific, e.g., “Using thelighteron the metal pot should be necessary to heat thewaterin the pot”,CLINcompiles these insights for a new target environment (as meta-memory) , e.g., “Using aheat source(stove, lighter)  on the container should be necessary to heat asubstance.”AppendixBcontains examples of memories generated during adaptation and generalization.CLIN’sGen-TaskimprovementsTypeGen-TaskG + Aavg%ep.avg%ep.scoreimprov.scoreimprov.S14.640.04.95.7L10.336.79.215.6All13.038.86.59.3(c) Ablations forCLINAblation Setupavg%ep.score () drop. () Abl-Causal-Memory-6.210Abl-Controller-Base-18.144.8(d) Figure 5:Reward and #steps trends forCLINin(a) Gen-Envand(b) Gen-Task.(c) % episode improvements and score change thanCLINwithout meta-memory (Gen-Task) .(d) CLINablations.Limitation: Lack of exploration.CLIN’s learnings are dependent on its own past experience. IfCLINnever explores a location in the environment or does not perform an action, an insight related to the unobserved activity can never be generated. Hence, exploration becomes important when task-critical location or action in unknown toCLINfrom past trials. For example, in task of creating an orange paint, the agent is supposed to find red and yellow paints from the art studio. However, art studio is not visible whenCLINstarts from location ‘outside’. Unless theCLINknows that there exists an art studio, it tries alternative method to create orange paints from other irrelevant objects (e.g., an orange)  and remains unsuccessful. When a memory related art-studio appears from past exploration,CLINis able to successfully complete the task. Similarly, in boil or freeze tasks,CLINis unable to perform well which requires it to consistently measure the temperature of the substance to know its boiling/freezing point—an act it could never perform successfully in past trials resulting into less useful memory insights and subsequent lower performance in future trials.Limitation: Poor memory retrieval.For a task of boiling gallium,CLINis supposed to use oven/blast furnace and not a stove. In the meta-memory for boiling tasks, there are two insights regarding the act of boiling: “Activating stove should be necessary to boil a substance” and “Using an alternative heat source (e.g., oven or fire pit)  may be necessary if the initial heat source is insufficient.” However,CLINrepeatedly retrieves the former and hence failing at the task despite performing other actions (e.g., finding gallium)  correctly. This problem intensifies at the initial trial during generalization due to the presence of insights with varied initial conditions for them to be applied.
This can be circumvented by improved memory representation, which we leave as a future work.
body
CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization
4Results and Analysis
2310.10134
5ConclusionOur goal is a system that can continually improve over time, both while rapidly adapting to a task by multiple retries and efficiently generalizing to novel tasks and environments.
We proposeCLIN, an architecture for language agents that constructs a persistent, dynamic memory of causal abstractions, refines it over time and uses it effectively to improve its performance on future tasks, achieving state-of-the-art performance. Our work systematically evaluates a novel nonparametric learning paradigm, promising never-ending learning abilities to frozen language agents.AcknowledgementWe sincerely thank Aristo team members Tushar Khot, Ashish Sabharwal, Shashank Gupta, Nathaniel Weir, Kyle Richardson, Jiangjie Chen, Archiki Prasad, and other members such as Faeze Brahman, Alexander Koller at the Allen Institute of AI for their generous feedback.
conclusion
CLIN: Continual Learning in Language Agents for Rapid Task Adaptation and Generalization
5Conclusion