<p dir="auto">Currently we print multiple exception stacks on a test failure. It will be good to</p>
<ul dir="auto">
<li>Print only the first exception from a failing test only</li>
<li>In the event of multiple exceptions from different testsets, print all of them only if an env variable is set. Default is to print only one.</li>
<li>Don't print known exceptions raised in the testsystem machinery</li>
<li>Limit stacktrace to last 30 lines?</li>
</ul>
<p dir="auto">I inserted one <code class="notranslate">@test 1 == 2</code> in a <code class="notranslate">@testset</code> loop in my package and this is the results:</p>
<div class="highlight highlight-source-julia notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="tree type: Test Failed
  Expression: 1 == 2
   Evaluated: 1 == 2
 in record(::Base.Test.DefaultTestSet, ::Base.Test.Fail) at ./test.jl:428
 in do_test(::Base.Test.Returned, ::Expr) at ./test.jl:281
 in macro expansion; at /home/kristoffer/.julia/v0.6/NearestNeighbors/test/test_inrange.jl:41 [inlined]
 in macro expansion; at ./test.jl:733 [inlined]
 in macro expansion; at /home/kristoffer/.julia/v0.6/NearestNeighbors/test/test_inrange.jl:0 [inlined]
 in macro expansion; at ./test.jl:733 [inlined]
 in macro expansion; at /home/kristoffer/.julia/v0.6/NearestNeighbors/test/test_inrange.jl:0 [inlined]
 in macro expansion; at ./test.jl:672 [inlined]
 in anonymous at ./&lt;missing&gt;:?
 in include_from_node1(::String) at ./loading.jl:532
 in include(::String) at ./sysimg.jl:14
 in include_from_node1(::String) at ./loading.jl:532
 in include(::String) at ./sysimg.jl:14
 in process_options(::Base.JLOptions) at ./client.jl:289
 in _start() at ./client.jl:353
Test Summary: | Pass  Fail  Total
  inrange     |   36     3     39
  metric      |   36     3     39
  tree type   |   12     1     13
  tree type   |   12     1     13
  tree type   |   12     1     13
ERROR: LoadError: LoadError: Some tests did not pass: 36 passed, 3 failed, 0 errored, 0 broken.
 in finish(::Base.Test.DefaultTestSet) at ./test.jl:495
 in macro expansion; at ./test.jl:679 [inlined]
 in anonymous at ./&lt;missing&gt;:?
 in include_from_node1(::String) at ./loading.jl:532
 in include(::String) at ./sysimg.jl:14
 in include_from_node1(::String) at ./loading.jl:532
 in include(::String) at ./sysimg.jl:14
 in process_options(::Base.JLOptions) at ./client.jl:289
 in _start() at ./client.jl:353
while loading /home/kristoffer/.julia/v0.6/NearestNeighbors/test/test_inrange.jl, in expression starting on line 2
while loading /home/kristoffer/.julia/v0.6/NearestNeighbors/test/runtests.jl, in expression starting on line 35
======================================[ ERROR: NearestNeighbors ]=======================================

failed process: Process(`/home/kristoffer/julia/usr/bin/julia -Cnative -J/home/kristoffer/julia/usr/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /home/kristoffer/.julia/v0.6/NearestNeighbors/test/runtests.jl`, ProcessExited(1)) [1]

========================================================================================================
INFO: No packages to install, update or remove
ERROR: NearestNeighbors had test errors
 in #test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:749
 in (::Base.Pkg.Entry.#kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./&lt;missing&gt;:0
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:69
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31
 in (::Base.Pkg.Dir.#kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./&lt;missing&gt;:0
 in #test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
 in test(::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258"><pre class="notranslate">tree type<span class="pl-k">:</span> Test Failed
  Expression<span class="pl-k">:</span> <span class="pl-c1">1</span> <span class="pl-k">==</span> <span class="pl-c1">2</span>
   Evaluated<span class="pl-k">:</span> <span class="pl-c1">1</span> <span class="pl-k">==</span> <span class="pl-c1">2</span>
 <span class="pl-k">in</span> <span class="pl-c1">record</span>(<span class="pl-k">::</span><span class="pl-c1">Base.Test.DefaultTestSet</span>, <span class="pl-k">::</span><span class="pl-c1">Base.Test.Fail</span>) at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">428</span>
 <span class="pl-k">in</span> <span class="pl-c1">do_test</span>(<span class="pl-k">::</span><span class="pl-c1">Base.Test.Returned</span>, <span class="pl-k">::</span><span class="pl-c1">Expr</span>) at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">281</span>
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">/</span>home<span class="pl-k">/</span>kristoffer<span class="pl-k">/</span><span class="pl-k">.</span>julia<span class="pl-k">/</span>v0.<span class="pl-c1">6</span><span class="pl-k">/</span>NearestNeighbors<span class="pl-k">/</span>test<span class="pl-k">/</span>test_inrange<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">41</span> [inlined]
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">733</span> [inlined]
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">/</span>home<span class="pl-k">/</span>kristoffer<span class="pl-k">/</span><span class="pl-k">.</span>julia<span class="pl-k">/</span>v0.<span class="pl-c1">6</span><span class="pl-k">/</span>NearestNeighbors<span class="pl-k">/</span>test<span class="pl-k">/</span>test_inrange<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">0</span> [inlined]
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">733</span> [inlined]
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">/</span>home<span class="pl-k">/</span>kristoffer<span class="pl-k">/</span><span class="pl-k">.</span>julia<span class="pl-k">/</span>v0.<span class="pl-c1">6</span><span class="pl-k">/</span>NearestNeighbors<span class="pl-k">/</span>test<span class="pl-k">/</span>test_inrange<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">0</span> [inlined]
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">672</span> [inlined]
 <span class="pl-k">in</span> anonymous at <span class="pl-k">./</span><span class="pl-k">&lt;</span><span class="pl-c1">missing</span><span class="pl-k">&gt;:</span>?
 <span class="pl-k">in</span> <span class="pl-c1">include_from_node1</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>loading<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">532</span>
 <span class="pl-k">in</span> <span class="pl-c1">include</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>sysimg<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">14</span>
 <span class="pl-k">in</span> <span class="pl-c1">include_from_node1</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>loading<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">532</span>
 <span class="pl-k">in</span> <span class="pl-c1">include</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>sysimg<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">14</span>
 <span class="pl-k">in</span> <span class="pl-c1">process_options</span>(<span class="pl-k">::</span><span class="pl-c1">Base.JLOptions</span>) at <span class="pl-k">./</span>client<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">289</span>
 <span class="pl-k">in</span> <span class="pl-c1">_start</span>() at <span class="pl-k">./</span>client<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">353</span>
Test Summary<span class="pl-k">:</span> <span class="pl-k">|</span> Pass  Fail  Total
  inrange     <span class="pl-k">|</span>   <span class="pl-c1">36</span>     <span class="pl-c1">3</span>     <span class="pl-c1">39</span>
  metric      <span class="pl-k">|</span>   <span class="pl-c1">36</span>     <span class="pl-c1">3</span>     <span class="pl-c1">39</span>
  tree type   <span class="pl-k">|</span>   <span class="pl-c1">12</span>     <span class="pl-c1">1</span>     <span class="pl-c1">13</span>
  tree type   <span class="pl-k">|</span>   <span class="pl-c1">12</span>     <span class="pl-c1">1</span>     <span class="pl-c1">13</span>
  tree type   <span class="pl-k">|</span>   <span class="pl-c1">12</span>     <span class="pl-c1">1</span>     <span class="pl-c1">13</span>
ERROR<span class="pl-k">:</span> LoadError<span class="pl-k">:</span> LoadError<span class="pl-k">:</span> Some tests did not pass<span class="pl-k">:</span> <span class="pl-c1">36</span> passed, <span class="pl-c1">3</span> failed, <span class="pl-c1">0</span> errored, <span class="pl-c1">0</span> broken.
 <span class="pl-k">in</span> <span class="pl-c1">finish</span>(<span class="pl-k">::</span><span class="pl-c1">Base.Test.DefaultTestSet</span>) at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">495</span>
 <span class="pl-k">in</span> <span class="pl-k">macro</span> expansion; at <span class="pl-k">./</span>test<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">679</span> [inlined]
 <span class="pl-k">in</span> anonymous at <span class="pl-k">./</span><span class="pl-k">&lt;</span><span class="pl-c1">missing</span><span class="pl-k">&gt;:</span>?
 <span class="pl-k">in</span> <span class="pl-c1">include_from_node1</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>loading<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">532</span>
 <span class="pl-k">in</span> <span class="pl-c1">include</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>sysimg<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">14</span>
 <span class="pl-k">in</span> <span class="pl-c1">include_from_node1</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>loading<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">532</span>
 <span class="pl-k">in</span> <span class="pl-c1">include</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>) at <span class="pl-k">./</span>sysimg<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">14</span>
 <span class="pl-k">in</span> <span class="pl-c1">process_options</span>(<span class="pl-k">::</span><span class="pl-c1">Base.JLOptions</span>) at <span class="pl-k">./</span>client<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">289</span>
 <span class="pl-k">in</span> <span class="pl-c1">_start</span>() at <span class="pl-k">./</span>client<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">353</span>
<span class="pl-k">while</span> loading <span class="pl-k">/</span>home<span class="pl-k">/</span>kristoffer<span class="pl-k">/</span><span class="pl-k">.</span>julia<span class="pl-k">/</span>v0.<span class="pl-c1">6</span><span class="pl-k">/</span>NearestNeighbors<span class="pl-k">/</span>test<span class="pl-k">/</span>test_inrange<span class="pl-k">.</span>jl, <span class="pl-k">in</span> expression starting on line <span class="pl-c1">2</span>
<span class="pl-k">while</span> loading <span class="pl-k">/</span>home<span class="pl-k">/</span>kristoffer<span class="pl-k">/</span><span class="pl-k">.</span>julia<span class="pl-k">/</span>v0.<span class="pl-c1">6</span><span class="pl-k">/</span>NearestNeighbors<span class="pl-k">/</span>test<span class="pl-k">/</span>runtests<span class="pl-k">.</span>jl, <span class="pl-k">in</span> expression starting on line <span class="pl-c1">35</span>
<span class="pl-k">======================================</span>[ ERROR<span class="pl-k">:</span> NearestNeighbors ]<span class="pl-k">=======================================</span>

failed process<span class="pl-k">:</span> <span class="pl-c1">Process</span>(<span class="pl-s"><span class="pl-pds">`</span>/home/kristoffer/julia/usr/bin/julia -Cnative -J/home/kristoffer/julia/usr/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /home/kristoffer/.julia/v0.6/NearestNeighbors/test/runtests.jl<span class="pl-pds">`</span></span>, <span class="pl-c1">ProcessExited</span>(<span class="pl-c1">1</span>)) [<span class="pl-c1">1</span>]

<span class="pl-k">========================================================================================================</span>
INFO<span class="pl-k">:</span> No packages to install, update or remove
ERROR<span class="pl-k">:</span> NearestNeighbors had test errors
 <span class="pl-k">in</span> <span class="pl-c"><span class="pl-c">#</span>test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:749</span>
 <span class="pl-k">in</span> (<span class="pl-k">::</span><span class="pl-c1">Base.Pkg.Entry.</span><span class="pl-c"><span class="pl-c">#</span>kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./&lt;missing&gt;:0</span>
 <span class="pl-k">in</span> (<span class="pl-k">::</span><span class="pl-c1">Base.Pkg.Dir.</span><span class="pl-c"><span class="pl-c">#</span>#2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31</span>
 <span class="pl-k">in</span> <span class="pl-c1">cd</span>(<span class="pl-k">::</span><span class="pl-c1">Base.Pkg.Dir.</span><span class="pl-c"><span class="pl-c">#</span>#2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:69</span>
 <span class="pl-k">in</span> <span class="pl-c"><span class="pl-c">#</span>cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31</span>
 <span class="pl-k">in</span> (<span class="pl-k">::</span><span class="pl-c1">Base.Pkg.Dir.</span><span class="pl-c"><span class="pl-c">#</span>kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./&lt;missing&gt;:0</span>
 <span class="pl-k">in</span> <span class="pl-c"><span class="pl-c">#</span>test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258</span>
 <span class="pl-k">in</span> <span class="pl-c1">test</span>(<span class="pl-k">::</span><span class="pl-c1">String</span>, <span class="pl-k">::</span><span class="pl-c1">Vararg{String,N}</span>) at <span class="pl-k">./</span>pkg<span class="pl-k">/</span>pkg<span class="pl-k">.</span>jl<span class="pl-k">:</span><span class="pl-c1">258</span></pre></div>
<p dir="auto">Is it possible that we could scrub away some of the stackframes here to make things more readable? The backtrace for the "Some tests did not pass" exception seems unnecessary and doesn't include anything related to the user. The backtrace for the actual test failure could maybe be changed so that all frames after the <code class="notranslate">test.jl</code> file gets removed?</p>