title: How do you handle unit/regression tests which are expected to fail during development?
body: <p>During software development, there may be bugs in the codebase which are known issues.  These bugs will cause the regression/unit tests to fail, if the tests have been written well.</p>
  
  <p>There is constant debate in our teams about how failing tests should be managed:</p>
  
  <ol>
  <li><p>Comment out failing test cases with a REVISIT or TODO comment.</p>
  
  <ul>
  <li><strong>Advantage</strong>: We will always know when a <em>new</em> defect has been introduced, and not one we are already aware of.</li>
  <li><strong>Disadvantage</strong>: May forget to REVISIT the commented-out test case, meaning that the defect could slip through the cracks.</li>
  </ul></li>
  <li><p>Leave the test cases failing.</p>
  
  <ul>
  <li><strong>Advantage</strong>: Will not forget to fix the defects, as the script failures will constantly reminding you that a defect is present.</li>
  <li><strong>Disadvantage</strong>: Difficult to detect when a <em>new</em> defect is introduced, due to failure noise.</li>
  </ul></li>
  </ol>
  
  <p>I'd like to explore what the best practices are in this regard.  Personally, I think a tri-state solution is the best for determining whether a script is passing.  For example when you run a script, you could see the following:</p>
  
  <ul>
  <li>Percentage passed: 75%</li>
  <li>Percentage failed (expected): 20%</li>
  <li>Percentage failed (unexpected): 5%</li>
  </ul>
  
  <p>You would basically mark any test cases which you <em>expect</em> to fail (due to some defect) with some metadata.  This ensures you still see the failure result at the end of the test, but immediately know if there is a <em>new</em> failure which you weren't expecting.  This appears to take the best parts of the 2 proposals above.</p>
  
  <p>Does anyone have any best practices for managing this?</p>