title: What's a good way to find the difference between two nose tests runs?
body: <p>I am trying to prepare a pull request for my changes to <code>matplotlib</code> here: <a href="https://github.com/shmuller/matplotlib.git" rel="nofollow">https://github.com/shmuller/matplotlib.git</a>. After merging with upstream/master (<a href="https://github.com/matplotlib/matplotlib.git" rel="nofollow">https://github.com/matplotlib/matplotlib.git</a>), I wanted to find out if I broke anything, so I run the test suite (<code>python tests.py -v -a</code>) on upstream/master. I get:</p>
  
  <pre><code>Ran 4688 tests in 555.109s
  FAILED(KNOWNFAIL=330, SKIP=9, errors=197, failures=16)
  </code></pre>
  
  <p>Now on my merged branch:</p>
  
  <pre><code>Ran 4682 tests in 555.070s
  FAILED(KNOWNFAIL=330, SKIP=9, errors=200, failures=18)
  </code></pre>
  
  <p>Darn! Quite close, but not the same! So I did break something that wasn't broken before. Since there are thousands of tests, and lots of errors and failures to begin with, it does not appear obvious to find out what I broke.</p>
  
  <p>So my question is: What's a good way to find out which tests broke that weren't broken before? </p>
  
  <p><code>tests.py</code> essentially does:</p>
  
  <pre><code>import nose
  nose.main()
  </code></pre>
  
  <p>so I am hoping for a feature in <code>nose</code> that helps me figure out what I broke, but couldn't find anything in the help (<code>nosetests --help</code>). I obviously could log and diff the whole output, but I'm hoping for a more elegant solution.</p>