title: Find commit that broke a test without running every test on every commit
body: <p>I am trying to find a tool that can solve the following problem: </p>
  
  <p>Our entire test suite takes hours to runs which often makes it difficult or at least very time consuming to find out which commit broke a specific test since there may be 50 to 200 commits in between test runs. At any given time there are only very few broken tests, so rerunning only the broken tests is very fast compared to running the entire test suite. </p>
  
  <p>Is there a tool, e.g. a continuous integration server, that can rerun failed tests with a couple of revisions in between the last revision where the test was OK and the first revision where the test was not OK and therefore automatically find out on which specific commit a test switched from successful to broken. </p>
  
  <p>For example: </p>
  
  <p>Test A and B are ok in revision 100. Test A and B are broken in revision 200.</p>
  
  <p>The tool should now run both tests with revision 150.
  Then if e.g. test A was broken and test B was OK in revision 150, it could continue to check test A with revision 125 and test B with revision 175 and so on until every broken test can be explained by some specific commit. </p>
  
  <p>For a single test, I could probably hack something together with git bisect. But for multiple failed test this is probably not sufficient, since we need to search in both directions for many revisions. </p>