python unittest.TestCase.assertEquals() on complex data structures
Question: I'm unit testing a function that returns a _very_ complex data structure (dict
of lists of lists of sets etc.). I validated the output manually, and now I
want to make sure it doesn't change without me noticing.

Right now I have:

    
    
       self.assertEquals(data,
            {'Instr1': {'COUPON_LIST': '0 % SEMI 30/360',
                        'LEGAL_ENTITY': 'LE_XS0181523803',
                        'MATURITY_DATE': '2014/12/31',
                        'scenarios': {'Base': {'Spread Over Yield': -1.9/100,
                                               'THEO/PV01': -1500.15,
                                               'THEO/Value': 0.333,
                                               'THEO/Yield': 3.3/100},
                                      'UP': {'Spread Over Yield': -2.2/100,
                                             'THEO/PV01': -1000.1,
                                             'THEO/Value': 0.111,
                                             'THEO/Yield': 5.5/100}}},
             'Instr2': {'COUPON_LIST': None,
                        'LEGAL_ENTITY': 'LE_US059512AJ22',
                        'MATURITY_DATE': '2014/12/31',
                        'scenarios': {'Base': {'Spread Over Yield': None,
                                               'THEO/PV01': None,
                                               'THEO/Value': 1.0,
                                               'THEO/Yield': 0.0},
                                      'UP': {'Spread Over Yield': None,
                                             'THEO/PV01': -15.15,
                                             'THEO/Value': 4055.344,
                                             'THEO/Yield': 4.4/100}}},
             'Instr3': {'COUPON_LIST': '0 % SEMI 30/360',
                        'LEGAL_ENTITY': 'LE_XS0181523803',
                        'MATURITY_DATE': '2014/12/31',
                        'scenarios': {'Base': {'Spread Over Yield': -1.9/100,
                                               'THEO/PV01': -1500.15,
                                               'THEO/Value': 0.333,
                                               'THEO/Yield': 3.3/100},
                                      'UP': {'Spread Over Yield': -2.2/100,
                                             'THEO/PV01': -1000.1,
                                             'THEO/Value': 0.111,
                                             'THEO/Yield': 5.5/100}}},
             'Instr4': {'COUPON_LIST': None,
                        'LEGAL_ENTITY': 'LE_US059512AJ22',
                        'MATURITY_DATE': '2014/12/31',
                        'scenarios': {'Base': {'Spread Over Yield': None,
                                               'THEO/PV01': None,
                                               'THEO/Value': 1.0,
                                               'THEO/Yield': 0.0},
                                      'UP': {'Spread Over Yield': None,
                                             'THEO/PV01': -15.15,
                                             'THEO/Value': 4055.344,
                                             'THEO/Yield': 4.4/100}}}}
    

I have two problems:

  1. the tested class is not stable and the data CAN change. In that case, I want to quickly pinpoint where the output changed, and only validate the differences. E.g. I'm looking for a nice output saying

data['Instr1']['MATURITY_DATE']: '2014/12/31' != '31/12/2014'

data['Instr5']: node not found in lhs

but at the same time I do not want to manually test every single node of the
structure.

  2. as you can see some elements are float, and 4.4/100 != 0.044. I need logic that runs AssertAlmostEqual on float nodes, and AssertEqual on everything else.

Is there any library that does this, or do I have to write my own?


Answer: A _quick and dirty_ solution is to compare the pretty-print representations of
the data using `difflib`. However this solution is absolutely _not_ robust:

    
    
    In [22]: import copy
        ...: import difflib
        ...: import pprint
        ...:
    In [23]: data = {'Instr1': {'COUPON_LIST': '0 % SEMI 30/360',
        ...:                     'LEGAL_ENTITY': 'LE_XS0181523803',
        ...:                     'MATURITY_DATE': '2014/12/31',
        ...:                     'scenarios': {'Base': {'Spread Over Yield': -1.9/100,
        ...:                                            'THEO/PV01': -1500.15,
        ...:                                            'THEO/Value': 0.333,
        ...:                                            'THEO/Yield': 3.3/100},
        ...:                                   'UP': {'Spread Over Yield': -2.2/100,
        ...:                                          'THEO/PV01': -1000.1,
        ...:                                          'THEO/Value': 0.111,
        ...:                                          'THEO/Yield': 5.5/100}}},
        ...:          'Instr2': {'COUPON_LIST': None,
        ...:                     'LEGAL_ENTITY': 'LE_US059512AJ22',
        ...:                     'MATURITY_DATE': '2014/12/31',
        ...:                     'scenarios': {'Base': {'Spread Over Yield': None,
        ...:                                            'THEO/PV01': None,
        ...:                                            'THEO/Value': 1.0,
        ...:                                            'THEO/Yield': 0.0},
        ...:                                   'UP': {'Spread Over Yield': None,
        ...:                                          'THEO/PV01': -15.15,
        ...:                                          'THEO/Value': 4055.344,
        ...:                                          'THEO/Yield': 4.4/100}}},
        ...:          'Instr3': {'COUPON_LIST': '0 % SEMI 30/360',
        ...:                     'LEGAL_ENTITY': 'LE_XS0181523803',
        ...:                     'MATURITY_DATE': '2014/12/31',
        ...:                     'scenarios': {'Base': {'Spread Over Yield': -1.9/100,
        ...:                                            'THEO/PV01': -1500.15,
        ...:                                            'THEO/Value': 0.333,
        ...:                                            'THEO/Yield': 3.3/100},
        ...:                                   'UP': {'Spread Over Yield': -2.2/100,
        ...:                                          'THEO/PV01': -1000.1,
        ...:                                          'THEO/Value': 0.111,
        ...:                                          'THEO/Yield': 5.5/100}}},
        ...:          'Instr4': {'COUPON_LIST': None,
        ...:                     'LEGAL_ENTITY': 'LE_US059512AJ22',
        ...:                     'MATURITY_DATE': '2014/12/31',
        ...:                     'scenarios': {'Base': {'Spread Over Yield': None,
        ...:                                            'THEO/PV01': None,
        ...:                                            'THEO/Value': 1.0,
        ...:                                            'THEO/Yield': 0.0},
        ...:                                   'UP': {'Spread Over Yield': None,
        ...:                                          'THEO/PV01': -15.15,
        ...:                                          'THEO/Value': 4055.344,
        ...:                                          'THEO/Yield': 4.4/100}}}}
    
    In [24]: data_repr = pprint.pformat(data)
    
    In [25]: data2 = copy.deepcopy(data)
    
    In [26]: data2['Instr1']['MATURITY_DATE'] = '31/12/2014'
    
    In [27]: data2_repr = pprint.pformat(data2)
    
    In [28]: def get_diff(a, b):
        ...:     differ = difflib.unified_diff(a.splitlines(True), b.splitlines(True))
        ...:     return ''.join(line for line in differ if not line.startswith(' '))
    
    In [29]: print(get_diff(data_repr, data2_repr))
    --- 
    +++ 
    @@ -1,6 +1,6 @@
    -            'MATURITY_DATE': '2014/12/31',
    +            'MATURITY_DATE': '31/12/2014',
    

However this doesn't solve the problem with floating point numbers. You could
solve this by first replacing `float`ing points values with `round`ed values
to some significant digit, using a simple recursive function.

As far as I know there is no such library that allows this level of fine
control over comparisons, so if you want a robust solution you'd better write
the whole code yourself.

I'd also point out that maybe you should refactor this data structure into a
more structured class, which would make things easier.

Last but not least: you can use `unittest`'s
[`addTypeEqualityFunc`](http://docs.python.org/3.3/library/unittest.html#unittest.TestCase.addTypeEqualityFunc)
to make sure the `TestCase` calls `assertAlmostEqual` when comparing `float`s,
without doing it by hand.

* * *

Now that I think about it you may be able to use `addTypeEqualityFunc` to
perform a custom comparison of `dict`s which could add more information on the
mismatch. To find _all_ mismatches You'd have to use some `except
AssertionError as e:` blocks into your custom function, always check all sub-
elements and then "join" the error messages somehow. But I don't think the
solution would be so clean.

