<p dir="auto">The following shows only 1 failed test:</p>
<div class="highlight highlight-source-julia notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="julia&gt; @testset &quot;foo $i&quot; for i=1:3
         @test 2==3+i
       end
foo 1: Test Failed
  Expression: 2 == 3 + i
   Evaluated: 2 == 4
Stacktrace:
 [1] macro expansion at .\REPL[44]:2 [inlined]
 [2] macro expansion at .\test.jl:921 [inlined]
 [3] anonymous at .\&lt;missing&gt;:?
Test Summary: | Fail  Total
foo 1         |    1      1
Test Summary: | Fail  Total
foo 1         |    1      1
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken."><pre class="notranslate">julia<span class="pl-k">&gt;</span> <span class="pl-c1">@testset</span> <span class="pl-s"><span class="pl-pds">"</span>foo <span class="pl-v">$i</span><span class="pl-pds">"</span></span> <span class="pl-k">for</span> i<span class="pl-k">=</span><span class="pl-c1">1</span><span class="pl-k">:</span><span class="pl-c1">3</span>
         <span class="pl-c1">@test</span> <span class="pl-c1">2</span><span class="pl-k">==</span><span class="pl-c1">3</span><span class="pl-k">+</span>i
       <span class="pl-k">end</span>
foo <span class="pl-c1">1</span><span class="pl-k">:</span> Test Failed
  Expression<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">3</span> <span class="pl-k">+</span> i
   Evaluated<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">4</span>
Stacktrace<span class="pl-k">:</span>
 [<span class="pl-c1">1</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>REPL[<span class="pl-c1">44</span>]<span class="pl-k">:</span><span class="pl-c1">2</span> [inlined]
 [<span class="pl-c1">2</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">921</span> [inlined]
 [<span class="pl-c1">3</span>] anonymous at <span class="pl-k">.\</span><span class="pl-k">&lt;</span><span class="pl-c1">missing</span><span class="pl-k">&gt;:</span>?
Test Summary<span class="pl-k">:</span> <span class="pl-k">|</span> Fail  Total
foo <span class="pl-c1">1</span>         <span class="pl-k">|</span>    <span class="pl-c1">1</span>      <span class="pl-c1">1</span>
Test Summary<span class="pl-k">:</span> <span class="pl-k">|</span> Fail  Total
foo <span class="pl-c1">1</span>         <span class="pl-k">|</span>    <span class="pl-c1">1</span>      <span class="pl-c1">1</span>
ERROR<span class="pl-k">:</span> Some tests did not pass<span class="pl-k">:</span> <span class="pl-c1">0</span> passed, <span class="pl-c1">1</span> failed, <span class="pl-c1">0</span> errored, <span class="pl-c1">0</span> broken.</pre></div>
<p dir="auto">However, if you manually write out the loop, all three tests are run:</p>
<div class="highlight highlight-source-julia notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="julia&gt; @testset &quot;foo&quot; begin
         @test 2==3+1
         @test 2==3+2
         @test 2==3+3
       end
foo: Test Failed
  Expression: 2 == 3 + 1
   Evaluated: 2 == 4
Stacktrace:
 [1] macro expansion at .\REPL[46]:2 [inlined]
 [2] macro expansion at .\test.jl:860 [inlined]
 [3] anonymous at .\&lt;missing&gt;:?
foo: Test Failed
  Expression: 2 == 3 + 2
   Evaluated: 2 == 5
Stacktrace:
 [1] macro expansion at .\REPL[46]:3 [inlined]
 [2] macro expansion at .\test.jl:860 [inlined]
 [3] anonymous at .\&lt;missing&gt;:?
foo: Test Failed
  Expression: 2 == 3 + 3
   Evaluated: 2 == 6
Stacktrace:
 [1] macro expansion at .\REPL[46]:4 [inlined]
 [2] macro expansion at .\test.jl:860 [inlined]
 [3] anonymous at .\&lt;missing&gt;:?
Test Summary: | Fail  Total
foo           |    3      3
ERROR: Some tests did not pass: 0 passed, 3 failed, 0 errored, 0 broken."><pre class="notranslate">julia<span class="pl-k">&gt;</span> <span class="pl-c1">@testset</span> <span class="pl-s"><span class="pl-pds">"</span>foo<span class="pl-pds">"</span></span> <span class="pl-k">begin</span>
         <span class="pl-c1">@test</span> <span class="pl-c1">2</span><span class="pl-k">==</span><span class="pl-c1">3</span><span class="pl-k">+</span><span class="pl-c1">1</span>
         <span class="pl-c1">@test</span> <span class="pl-c1">2</span><span class="pl-k">==</span><span class="pl-c1">3</span><span class="pl-k">+</span><span class="pl-c1">2</span>
         <span class="pl-c1">@test</span> <span class="pl-c1">2</span><span class="pl-k">==</span><span class="pl-c1">3</span><span class="pl-k">+</span><span class="pl-c1">3</span>
       <span class="pl-k">end</span>
foo<span class="pl-k">:</span> Test Failed
  Expression<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">3</span> <span class="pl-k">+</span> <span class="pl-c1">1</span>
   Evaluated<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">4</span>
Stacktrace<span class="pl-k">:</span>
 [<span class="pl-c1">1</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>REPL[<span class="pl-c1">46</span>]<span class="pl-k">:</span><span class="pl-c1">2</span> [inlined]
 [<span class="pl-c1">2</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">860</span> [inlined]
 [<span class="pl-c1">3</span>] anonymous at <span class="pl-k">.\</span><span class="pl-k">&lt;</span><span class="pl-c1">missing</span><span class="pl-k">&gt;:</span>?
foo<span class="pl-k">:</span> Test Failed
  Expression<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">3</span> <span class="pl-k">+</span> <span class="pl-c1">2</span>
   Evaluated<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">5</span>
Stacktrace<span class="pl-k">:</span>
 [<span class="pl-c1">1</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>REPL[<span class="pl-c1">46</span>]<span class="pl-k">:</span><span class="pl-c1">3</span> [inlined]
 [<span class="pl-c1">2</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">860</span> [inlined]
 [<span class="pl-c1">3</span>] anonymous at <span class="pl-k">.\</span><span class="pl-k">&lt;</span><span class="pl-c1">missing</span><span class="pl-k">&gt;:</span>?
foo<span class="pl-k">:</span> Test Failed
  Expression<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">3</span> <span class="pl-k">+</span> <span class="pl-c1">3</span>
   Evaluated<span class="pl-k">:</span> <span class="pl-c1">2</span> <span class="pl-k">==</span> <span class="pl-c1">6</span>
Stacktrace<span class="pl-k">:</span>
 [<span class="pl-c1">1</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>REPL[<span class="pl-c1">46</span>]<span class="pl-k">:</span><span class="pl-c1">4</span> [inlined]
 [<span class="pl-c1">2</span>] <span class="pl-k">macro</span> expansion at <span class="pl-k">.\</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">860</span> [inlined]
 [<span class="pl-c1">3</span>] anonymous at <span class="pl-k">.\</span><span class="pl-k">&lt;</span><span class="pl-c1">missing</span><span class="pl-k">&gt;:</span>?
Test Summary<span class="pl-k">:</span> <span class="pl-k">|</span> Fail  Total
foo           <span class="pl-k">|</span>    <span class="pl-c1">3</span>      <span class="pl-c1">3</span>
ERROR<span class="pl-k">:</span> Some tests did not pass<span class="pl-k">:</span> <span class="pl-c1">0</span> passed, <span class="pl-c1">3</span> failed, <span class="pl-c1">0</span> errored, <span class="pl-c1">0</span> broken.</pre></div>
<p dir="auto">I don't think this is intended (see the discussion in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="224877995" data-permission-text="Title is private" data-url="https://github.com/JuliaLang/julia/issues/21594" data-hovercard-type="issue" data-hovercard-url="/JuliaLang/julia/issues/21594/hovercard" href="https://github.com/JuliaLang/julia/issues/21594">#21594</a>). If this is intended behavior, it should probably be better documented.</p>
<p dir="auto"><code class="notranslate">@testset</code> should constitute rollback points for exceptions (execution should continue on the next testset), and indeed it's the case, but only for nested testsets:</p>
<div class="highlight highlight-source-julia notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="using Base.Test
@testset &quot;outer&quot; begin
    @testset &quot;inner&quot; begin
        @test true
        @assert false
    end
    @testset &quot;inner2&quot; begin
        @test true
    end
end
@testset &quot;next&quot; begin
    @test true
end"><pre class="notranslate"><span class="pl-k">using</span> Base<span class="pl-k">.</span>Test
<span class="pl-c1">@testset</span> <span class="pl-s"><span class="pl-pds">"</span>outer<span class="pl-pds">"</span></span> <span class="pl-k">begin</span>
    <span class="pl-c1">@testset</span> <span class="pl-s"><span class="pl-pds">"</span>inner<span class="pl-pds">"</span></span> <span class="pl-k">begin</span>
        <span class="pl-c1">@test</span> <span class="pl-c1">true</span>
        <span class="pl-c1">@assert</span> <span class="pl-c1">false</span>
    <span class="pl-k">end</span>
    <span class="pl-c1">@testset</span> <span class="pl-s"><span class="pl-pds">"</span>inner2<span class="pl-pds">"</span></span> <span class="pl-k">begin</span>
        <span class="pl-c1">@test</span> <span class="pl-c1">true</span>
    <span class="pl-k">end</span>
<span class="pl-k">end</span>
<span class="pl-c1">@testset</span> <span class="pl-s"><span class="pl-pds">"</span>next<span class="pl-pds">"</span></span> <span class="pl-k">begin</span>
    <span class="pl-c1">@test</span> <span class="pl-c1">true</span>
<span class="pl-k">end</span></pre></div>
<p dir="auto">Summary shows:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Test Summary: | Pass  Error  Total
outer         |    2      1      3
  inner       |    1      1      2
  inner2      |    1             1"><pre class="notranslate"><code class="notranslate">Test Summary: | Pass  Error  Total
outer         |    2      1      3
  inner       |    1      1      2
  inner2      |    1             1
</code></pre></div>
<p dir="auto">inner2 is run, but "next" is not. It seems this is triggered by finalize(), which stops on error at the top level. Is this intended, and if so, why?</p>