title: How can I get MStest to return a non-zero exit code when failing?
body: <p>I am trying to test some .NET Core code with MSTest, and I am discovering that the test suite returns an exit code of zero even when there are failing tests. I am able to reproduce the problem using the code samples from <a href="https://blogs.msdn.microsoft.com/visualstudioalm/2016/05/30/announcing-mstest-framework-support-for-net-core-rc2-asp-net-core-rc2/" rel="nofollow noreferrer">the blog post announcing the technology</a>. The code to reproduce the problem is <a href="https://github.com/johnnymo87/exit-code-problems" rel="nofollow noreferrer">on github</a>, and the important parts are reprinted below.</p>
  
  <p><code>project.json</code></p>
  
  <pre><code>{
    "version": "1.0.0-*",
  
    "testRunner": "mstest",
  
    "dependencies": {
      "dotnet-test-mstest": "1.0.1-preview",
      "MSTest.TestFramework": "1.0.0-preview"
    },
  
    "frameworks": {
      "netcoreapp1.0": {
        "imports": [
          "dnxcore50",
          "portable-net45+win8"
        ],
  
        "dependencies": {
          "Microsoft.NETCore.App": {
            "version": "1.0.0",
            "type": "platform"
          }
        }
      }
    }
  }
  </code></pre>
  
  <p><code>ExitCodeTests.cs</code></p>
  
  <pre><code>using Microsoft.VisualStudio.TestTools.UnitTesting;
  
  namespace SampleNetCoreUnitTests
  {
    [TestClass]
    public class TestClass
    {
      [TestMethod]
      public void TestMethodPassing()
      {
        Assert.IsTrue(true);
      }
  
      [TestMethod]
      public void TestMethodFailing()
      {
        Assert.IsTrue(false);
      }
    }
  }
  </code></pre>
  
  <p>When I run the tests with <code>dotnet test &amp;&amp; echo THIS SHOULD NOT BE ECHOED</code>, I get the following output:</p>
  
  <pre><code>Discovering Tests ...
  Executing Tests ...
  Passed   TestMethodPassing
  Failed   TestMethodFailing
  Error Message:
     Assert.IsTrue failed.
  Stack Trace:
     at SampleNetCoreUnitTests.TestClass.TestMethodFailing() in /app/ExitCodeTests.cs:line 17
  ============ Test Run Summary ============
  Total tests: 2. Passed: 1. Failed: 1. Skipped: 0
  Test Run Failed.
  SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.
  THIS SHOULD NOT BE ECHOED
  </code></pre>
  
  <p>Even though I have a failing test, the test suite is returning an exit code of zero, and thus echo command is being executed.</p>
  
  <p>Short of switching to an alternative such as xUnit, how do I get my test suite to return a non-zero exit code when failing?</p>
  
  <p><strong>Edit:</strong> <a href="https://stackoverflow.com/a/15391046/2197402">The author of this stackoverflow answer</a> seems to believe MSTest should return a zero exit code even when failing. Maybe it's a MSTest "feature". Maybe I should consider some alternative test runners.</p>