codecov-io: # [Codecov](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=h1) Report
> Merging [#92](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=desc) into [master](https://codecov.io/gh/python-tap/tappy/commit/3d73aad083d56a888495f7056f4aff2b94dc4af2?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `16.66%`.

[![Impacted file tree graph](https://codecov.io/gh/python-tap/tappy/pull/92/graphs/tree.svg?width=650&token=ULVfoGuR2p&height=150&src=pr)](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master      #92      +/-   ##
==========================================
- Coverage     100%   99.13%   -0.87%     
==========================================
  Files          12       12              
  Lines         569      575       +6     
==========================================
+ Hits          569      570       +1     
- Misses          0        5       +5
```


| [Impacted Files](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [tap/runner.py](https://codecov.io/gh/python-tap/tappy/pull/92/diff?src=pr&el=tree#diff-dGFwL3J1bm5lci5weQ==) | `92.75% <16.66%> (-7.25%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=footer). Last update [3d73aad...70ab2cd](https://codecov.io/gh/python-tap/tappy/pull/92?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

ecederstrand: I think coverage may be run on Python 2 where subtests aren't supported. I'm definitely hitting the new `addSubTest()` method in my local tests. Should I make the exposure of that method conditional? E.g.:

```python
class TAPTestResult(TextTestResult):
    if sys.version_info[0] >= 3:
        def addSubTest(self, test, subtest, err):
            ...
```
ecederstrand: Aslo, I believe I'm innocent of this test failure: https://travis-ci.org/python-tap/tappy/jobs/622716225?utm_medium=notification&utm_source=github_status
mblayman: Hi, @ecederstrand, thanks for the contribution! tappy is generally pretty stable so it's been a while since I kicked CI's tires.

Since Python 2 is EOL at the end of this month, this PR prompted me to start up another PR that will drop Python 2 support before the next release. I should get that merged tonight and get the macOS build working so you can merge master back into your PR and get a green build.

I'll give the code a review after Travis CI is happy. Thanks!
mblayman: Oh! I forgot to add that I pushed a couple of commits to your branch to save you the trouble of merging master. You'll need to pull locally (my apologies if you're a Git pro and know all this already. It's hard with first time contributor PRs to know ability level).
ecederstrand: No problem, and thanks for the help with the Python 2 cleanup :-)

Subtests are a way to specify that parts of a test case should be a test case in its own. It's usually done for a single assert in a larger test case:

```python
# test.py
import unittest

from tap import TAPTestRunner


class TestCase(unittest.TestCase):
    def test_subs(self):
        for i in range(4):
            with self.subTest(i=i):
                self.assertEqual(i % 2, 0)


if __name__ == '__main__':
    runner = TAPTestRunner()
    runner.set_stream(True)
    unittest.main(testRunner=runner)
```

And here's an example of the output with this PR:

```bash
$ python test.py 
# TAP results for TestCase
ok 1 test_subs (__main__.TestCase) (i=0)
not ok 2 test_subs (__main__.TestCase) (i=1)
# Traceback (most recent call last):
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 59, in testPartExecutor
#     yield
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 523, in subTest
#     yield
#   File "test.py", line 10, in test_subs
#     self.assertEqual(i % 2, 0)
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 829, in assertEqual
#     assertion_func(first, second, msg=msg)
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 822, in _baseAssertEqual
#     raise self.failureException(msg)
# AssertionError: 1 != 0
ok 3 test_subs (__main__.TestCase) (i=2)
not ok 4 test_subs (__main__.TestCase) (i=3)
# Traceback (most recent call last):
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 59, in testPartExecutor
#     yield
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 523, in subTest
#     yield
#   File "test.py", line 10, in test_subs
#     self.assertEqual(i % 2, 0)
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 829, in assertEqual
#     assertion_func(first, second, msg=msg)
#   File "/opt/python35/lib/python3.5/unittest/case.py", line 822, in _baseAssertEqual
#     raise self.failureException(msg)
# AssertionError: 1 != 0
1..4
```
ecederstrand: I believe all requested changes are done now.

I noticed one more usage of `sys.version_info` in tap/tests/test_parser.py but left it untouched to not pollute this PR.
ecederstrand: Oh, and Python 3.4 is also EOL: https://devguide.python.org/devcycle/#end-of-life-branches

You may want to remove it from travis.yml
diff --git a/AUTHORS b/AUTHORS
index 566404e..06d46c5 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -5,6 +5,7 @@ Contributors
 
 * Andrew McNamara
 * Chris Clarke
+* Erik Cederstrand
 * Marc Abramowitz
 * Mark E. Hamilton
 * Matt Layman
diff --git a/docs/releases.rst b/docs/releases.rst
index b04ab19..ee84c7e 100644
--- a/docs/releases.rst
+++ b/docs/releases.rst
@@ -5,6 +5,7 @@ Version 3.0, To Be Released
 ---------------------------
 
 * Drop support for Python 2 (it is end-of-life).
+* Add support for subtests.
 
 Version 2.6.2, Released October 20, 2019
 ----------------------------------------
diff --git a/tap/runner.py b/tap/runner.py
index 94134e1..10e8510 100644
--- a/tap/runner.py
+++ b/tap/runner.py
@@ -17,6 +17,18 @@ class TAPTestResult(TextTestResult):
     def __init__(self, stream, descriptions, verbosity):
         super(TAPTestResult, self).__init__(stream, descriptions, verbosity)
 
+    def addSubTest(self, test, subtest, err):
+        super(TAPTestResult, self).addSubTest(test, subtest, err)
+        if err is not None:
+            diagnostics = formatter.format_exception(err)
+            self.tracker.add_not_ok(
+                self._cls_name(test),
+                self._description(subtest),
+                diagnostics=diagnostics,
+            )
+        else:
+            self.tracker.add_ok(self._cls_name(test), self._description(subtest))
+
     def stopTestRun(self):
         """Once the test run is complete, generate each of the TAP files."""
         super(TAPTestResult, self).stopTestRun()

Sub tests are not included
If the unittests include [subtests](https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests) the results of these subtests are not picked up.

Also, if the test class contains only failing subtests, the tap file may not be generated at all.

diff --git a/tap/tests/test_result.py b/tap/tests/test_result.py
index 06fb7e1..b5c1c9a 100644
--- a/tap/tests/test_result.py
+++ b/tap/tests/test_result.py
@@ -1,7 +1,9 @@
 # Copyright (c) 2019, Matt Layman and contributors
 
+import contextlib
 import os
 import unittest
+import unittest.case
 
 from tap.i18n import _
 from tap.runner import TAPTestResult
@@ -13,6 +15,14 @@ class FakeTestCase(unittest.TestCase):
     def runTest(self):
         pass
 
+    @contextlib.contextmanager
+    def subTest(self, *args, **kwargs):
+        try:
+            self._subtest = unittest.case._SubTest(self, object(), {})
+            yield
+        finally:
+            self._subtest = None
+
     def __call__(self, result):
         pass
 
@@ -70,3 +80,25 @@ class TestTAPTestResult(TestCase):
         self.assertEqual(
             line.directive.text, "TODO {}".format(_("(unexpected success)"))
         )
+
+    def test_adds_subtest_success(self):
+        """Test that the runner handles subtest success results."""
+        result = self._make_one()
+        test = FakeTestCase()
+        with test.subTest():
+            result.addSubTest(test, test._subtest, None)
+        line = result.tracker._test_cases["FakeTestCase"][0]
+        self.assertTrue(line.ok)
+
+    def test_adds_subtest_failure(self):
+        """Test that the runner handles subtest failure results."""
+        result = self._make_one()
+        # Python 3 does some extra testing in unittest on exceptions so fake
+        # the cause as if it were raised.
+        ex = Exception()
+        ex.__cause__ = None
+        test = FakeTestCase()
+        with test.subTest():
+            result.addSubTest(test, test._subtest, (ex.__class__, ex, None))
+        line = result.tracker._test_cases["FakeTestCase"][0]
+        self.assertFalse(line.ok)

pip install -e .[dev]
parse_log_pytest
requirements.txt
pytest pytest-tap
apt-get update apt-get install -y gcc
3.9
requirements/base.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
exceptiongroup==1.2.2
iniconfig==2.1.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.5
pytest-tap==3.5
-e git+https://github.com/python-tap/tappy.git@1c079e9ad6aaecaefcfc65d306417b28bce2ce06#egg=tap.py
tomli==2.2.1

name: tappy
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - pip=25.0=py39h06a4308_0
  - python=3.9.21=he870216_1
  - readline=8.2=h5eee18b_0
  - setuptools=75.8.0=py39h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - tzdata=2025a=h04d1e81_0
  - wheel=0.45.1=py39h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - exceptiongroup==1.2.2
      - iniconfig==2.1.0
      - packaging==24.2
      - pluggy==1.5.0
      - pytest==8.3.5
      - pytest-tap==3.5
      - tomli==2.2.1
prefix: /opt/conda/envs/tappy

tap/tests/test_result.py::TestTAPTestResult::test_adds_subtest_failure tap/tests/test_result.py::TestTAPTestResult::test_adds_subtest_success
tap/tests/test_result.py::FakeTestCase::runTest tap/tests/test_result.py::TestTAPTestResult::test_adds_error tap/tests/test_result.py::TestTAPTestResult::test_adds_expected_failure tap/tests/test_result.py::TestTAPTestResult::test_adds_failure tap/tests/test_result.py::TestTAPTestResult::test_adds_skip tap/tests/test_result.py::TestTAPTestResult::test_adds_success tap/tests/test_result.py::TestTAPTestResult::test_adds_unexpected_success
docs/releases.rst tap/runner.py AUTHORS
docs/releases.rst tap/runner.py AUTHORS