<p dir="auto"><strong>Describe the bug</strong><br>
I want to wait until a subprocess(like a web server) is ready.<br>
So I'm tried to write the code that reads the stdout of the process in async iteration and when I get a specific string, break it.</p>
<p dir="auto">But it seems to be not releasing a TextDecoder if it hasn't completed the async iteration loop.</p>
<p dir="auto">Example:</p>
<p dir="auto"><code class="notranslate">readablestream_test.ts</code></p>
<div class="highlight highlight-source-ts notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import { TextLineStream, } from &quot;https://deno.land/std@0.129.0/streams/delimiter.ts&quot;;

Deno.test(&quot;ReadableStream leak memory&quot;, async () =&gt; {
  const process = Deno.run({
    cmd: [&quot;ls&quot;],
    stdout: &quot;piped&quot;,
  });

  const lines = process.stdout.readable
    .pipeThrough(new TextDecoderStream())
    .pipeThrough(new TextLineStream());

  for await (const _line of lines) {
    break;
  }
  await lines.cancel();

  process.kill(&quot;SIGTERM&quot;);
  process.close();

  console.log(Deno.resources());
});"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-kos">{</span> <span class="pl-smi">TextLineStream</span><span class="pl-kos">,</span> <span class="pl-kos">}</span> <span class="pl-k">from</span> <span class="pl-s">"https://deno.land/std@0.129.0/streams/delimiter.ts"</span><span class="pl-kos">;</span>

<span class="pl-smi">Deno</span><span class="pl-kos">.</span><span class="pl-en">test</span><span class="pl-kos">(</span><span class="pl-s">"ReadableStream leak memory"</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">)</span> <span class="pl-c1">=&gt;</span> <span class="pl-kos">{</span>
  <span class="pl-k">const</span> <span class="pl-s1">process</span> <span class="pl-c1">=</span> <span class="pl-smi">Deno</span><span class="pl-kos">.</span><span class="pl-en">run</span><span class="pl-kos">(</span><span class="pl-kos">{</span>
    <span class="pl-c1">cmd</span>: <span class="pl-kos">[</span><span class="pl-s">"ls"</span><span class="pl-kos">]</span><span class="pl-kos">,</span>
    <span class="pl-c1">stdout</span>: <span class="pl-s">"piped"</span><span class="pl-kos">,</span>
  <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span>

  <span class="pl-k">const</span> <span class="pl-s1">lines</span> <span class="pl-c1">=</span> <span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-c1">stdout</span><span class="pl-kos">.</span><span class="pl-c1">readable</span>
    <span class="pl-kos">.</span><span class="pl-en">pipeThrough</span><span class="pl-kos">(</span><span class="pl-k">new</span> <span class="pl-smi">TextDecoderStream</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">)</span>
    <span class="pl-kos">.</span><span class="pl-en">pipeThrough</span><span class="pl-kos">(</span><span class="pl-k">new</span> <span class="pl-smi">TextLineStream</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">)</span><span class="pl-kos">;</span>

  <span class="pl-k">for</span> <span class="pl-k">await</span> <span class="pl-kos">(</span><span class="pl-k">const</span> <span class="pl-s1">_line</span> <span class="pl-k">of</span> <span class="pl-s1">lines</span><span class="pl-kos">)</span> <span class="pl-kos">{</span>
    <span class="pl-k">break</span><span class="pl-kos">;</span>
  <span class="pl-kos">}</span>
  <span class="pl-k">await</span> <span class="pl-s1">lines</span><span class="pl-kos">.</span><span class="pl-en">cancel</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span>

  <span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-en">kill</span><span class="pl-kos">(</span><span class="pl-s">"SIGTERM"</span><span class="pl-kos">)</span><span class="pl-kos">;</span>
  <span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-en">close</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span>

  <span class="pl-smi">console</span><span class="pl-kos">.</span><span class="pl-en">log</span><span class="pl-kos">(</span><span class="pl-smi">Deno</span><span class="pl-kos">.</span><span class="pl-en">resources</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">)</span><span class="pl-kos">;</span>
<span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span></pre></div>
<p dir="auto">We can get the following result.</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="$ deno run -A readablestream_test.ts
Check file:///Users/ioriikeda/Projects/ts-scraps/leak-error/readablestream_test.ts
running 1 test from file:///Users/ioriikeda/Projects/ts-scraps/leak-error/readablestream_test.ts
test ReadableStream leak memory ...{ &quot;0&quot;: &quot;stdin&quot;, &quot;1&quot;: &quot;stdout&quot;, &quot;2&quot;: &quot;stderr&quot;, &quot;5&quot;: &quot;textDecoder&quot; }

 FAILED (45ms)

failures:

ReadableStream leak memory
AssertionError: Test case is leaking 1 resource:

 - A text decoder (rid 5) was created during the test, but not finsihed during the test. Close the text decoder by calling `textDecoder.decode('')` or `await textDecoderStream.readable.cancel()`.

    at assert (deno:runtime/js/06_util.js:46:13)
    at resourceSanitizer (deno:runtime/js/40_testing.js:406:7)
    at async Object.exitSanitizer [as fn] (deno:runtime/js/40_testing.js:424:9)
    at async runTest (deno:runtime/js/40_testing.js:784:7)
    at async Object.runTests (deno:runtime/js/40_testing.js:982:22)

failures:

        ReadableStream leak memory

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out (75ms)

error: Test failed"><pre class="notranslate">$ deno run -A readablestream_test.ts
Check file:///Users/ioriikeda/Projects/ts-scraps/leak-error/readablestream_test.ts
running 1 <span class="pl-c1">test</span> from file:///Users/ioriikeda/Projects/ts-scraps/leak-error/readablestream_test.ts
<span class="pl-c1">test</span> ReadableStream leak memory ...{ <span class="pl-s"><span class="pl-pds">"</span>0<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>stdin<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>1<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>stdout<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>2<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>stderr<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>5<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>textDecoder<span class="pl-pds">"</span></span> }

 FAILED (45ms)

failures:

ReadableStream leak memory
AssertionError: Test <span class="pl-k">case</span> is leaking 1 resource:

 - A text decoder (rid 5) was created during the test, but not finsihed during the test. Close the text decoder by calling <span class="pl-s"><span class="pl-pds">`</span>textDecoder.decode(<span class="pl-s"><span class="pl-pds">'</span><span class="pl-pds">'</span></span>)<span class="pl-pds">`</span></span> or <span class="pl-s"><span class="pl-pds">`</span>await <span class="pl-en">textDecoderStream.readable.cancel</span>()<span class="pl-s"><span class="pl-pds">`</span>.</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">    at assert (deno:runtime/js/06_util.js:46:13)</span></span>
<span class="pl-s"><span class="pl-s">    at resourceSanitizer (deno:runtime/js/40_testing.js:406:7)</span></span>
<span class="pl-s"><span class="pl-s">    at async Object.exitSanitizer [as fn] (deno:runtime/js/40_testing.js:424:9)</span></span>
<span class="pl-s"><span class="pl-s">    at async runTest (deno:runtime/js/40_testing.js:784:7)</span></span>
<span class="pl-s"><span class="pl-s">    at async Object.runTests (deno:runtime/js/40_testing.js:982:22)</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">failures:</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">        ReadableStream leak memory</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s"><span class="pl-c1">test</span> result: FAILED. 0 passed<span class="pl-k">;</span> 1 failed<span class="pl-k">;</span> 0 ignored<span class="pl-k">;</span> 0 measured<span class="pl-k">;</span> 0 filtered out (75ms)</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">error: Test failed</span></span></pre></div>
<p dir="auto"><strong>Expected behavior</strong><br>
Release the resource even if an async iteration has not completed when we break the iteration.</p>
<p dir="auto"><strong>Environment</strong></p>
<ul dir="auto">
<li>OS: MacOS 12.1</li>
<li>deno: deno 1.20.1 (release, x86_64-apple-darwin)</li>
<li>std: 0.129.0</li>
</ul>
<p dir="auto">I've come across what seems to be a bug in <code class="notranslate">TextDecoderStream</code> which allows it to leak the native decoder used by its <code class="notranslate">TextDecoder</code>.</p>
<p dir="auto">I've made a test module to demonstrate the issue (repro code is at the bottom of that page, under the output): <a href="https://gist.github.com/h4l/0199ab7cc24dd13536e01c5ea98b3ae7">https://gist.github.com/h4l/0199ab7cc24dd13536e01c5ea98b3ae7</a><br>
The 3 tests trigger the test runner's resource leak detection (a really nice feature!).</p>
<p dir="auto">What seems to be happening is:</p>
<ul dir="auto">
<li><code class="notranslate">TextDecoderStream</code> uses a <code class="notranslate">TextDecoder</code> to decode its chunks</li>
<li><code class="notranslate">TextDecoder.decode()</code> creates a native decoder resource, which it holds open when used in streaming mode. It closes the resource when a non-streaming <code class="notranslate">decode()</code> call is made.</li>
<li><code class="notranslate">TextDecoderStream</code> makes streaming <code class="notranslate">decode()</code> calls in its <code class="notranslate">transform()</code> method, and makes a final non-streaming <code class="notranslate">decode()</code> call in its <code class="notranslate">flush()</code> method.</li>
<li>When a stream pipeline is errored, the <code class="notranslate">flush()</code> method of any <code class="notranslate">Transformer</code> in a <code class="notranslate">TransformStream</code> is not called, so in the case of <code class="notranslate">TextDecoderStream</code> it has no way to know its no longer in use, and keeps open its decoder.</li>
</ul>
<p dir="auto">I was looking through the <a href="https://streams.spec.whatwg.org/" rel="nofollow">streams spec</a> when I encountered the leak (before I worked out the cause) to try to work out if I was misusing the streams in some way. It seems to me like an oversight in the spec that <code class="notranslate">Transformer</code>s have no way to be told to close/clean up when a stream doesn't close cleanly. <code class="notranslate">flush()</code> is only called when the stream closes normally if I've not missed something, and there are no other lifecycle methods available for <code class="notranslate">Transformer</code>s.</p>
<p dir="auto">I can't see any idiomatic way to tell the <code class="notranslate">Transformer</code> to close, but one approach could be to wrap the readable and writable streams of the <code class="notranslate">TransformerStream</code> to watch for <code class="notranslate">close</code>/<code class="notranslate">cancel</code>/<code class="notranslate">abort</code> calls.</p>
<p dir="auto">I've not made any previous contributions, but I'd be happy to help with a PR to fix this if it'd be useful.</p>