Running test script scenarios in python
Question: I'm developing a script to test another program. Basically I replicated a
class with all the functions that the program I am testing have. To make
things simple, assume I have only 2 functions: set(value) and add(value).
set(value) sets an accumulator value and add(value) adds value to it; and I'm
testing a program that should do the same thing.

I also have a check() function that verifies that the accumulator value in
which set(value) and add(value) operates have the same value as the value
retrieved by the program I'm testing; so, after EACH operation, I want to run
this check() function and make sure they match.

To try to make this clear, here's a (non-tested) example:

    
    
    def set(self, value):
        self.acc = value
        sendCommandToMyProgram('set', value)
    
    def add(self, value):
        self.acc += value
        sendCommandToMyProgram('add', value)
    
    def check(self):
        return self.acc == sendCommandToMyProgram('check', value)
    

and I want to do "test scenarios" like:

    
    
    set(1)
    check()
    add(2)
    check()
    add(-2)
    check()
    add(3)
    

and many others

What would be the best way to make this "test scenario" script? It would be
nice if it could work with functions that takes different parameters (in this
example, both set() and add() received just one parameter - value -, but this
isn't always the case).

thanks


Answer: Assuming that you're not wanting to check the value _all_ the time during
normal operation, the thing that pops to mind first would be the built-in
[unit test module](http://docs.python.org/2/library/unittest.html).

If you are needing to do more complex things that you cannot cleanly isolate
the test as shown below, check out a some of [my
issues](http://stackoverflow.com/q/5834872/157744) I had to work around. It's
worth noting that I ended up rolling my own test runner that behaves very
similar to unit-test, but with greater control. I'll be pushing it up to py-pi
in the next month or so.

**Working Unit Test Examples**

    
    
    import unittest
    
    class RealAccumlator(object):
        def __init__(self):
            self.acc = 0
    
        def add(self, val):
            self.acc += val
    
        def set(self, val):
            self.acc = val
    
        def get(self):
            return self.acc
    
    
    # Using a global variable as I dont know how "sendCommandToMyProgram"
    # is used
    real_acc = RealAccumlator()
    
    
    # Test Script
    
    class MyAccumlator(object):
        def __init__(self):
            self.acc = 0
    
        def add(self, val):
            self.acc += val
            real_acc.add(val)
    
        def set(self, val):
            self.acc = val
            real_acc.set(val)
    
        def check(self):
            return self.acc == real_acc.get()
    
    class MockedAccumlatorTests(unittest.TestCase):
    
        def setUp(self):
            self.acc = MyAccumlator()
    
        def test_SetFunction(self):
            test_values = range(-10,10)
    
            # Test the set commands
            for val in test_values:
                self.acc.set(val)
                self.assertTrue(self.acc.check())
    
        def test_AddFunction(self):
            test_values = range(-10,10)
    
            # Set the acc to a known value and to a quick test
            self.acc.set(0) 
            self.assertTrue(self.acc.check())
    
            # Test the add commands
            for val in test_values:
                self.acc.add(val)
                self.assertTrue(self.acc.check())
    
    
    class RealAccumlatorTests(unittest.TestCase):
    
        def test_SetFunction(self):
            test_values = range(-10,10)
    
            # Test the set commands
            for val in test_values:
                real_acc.set(val)
                self.assertEqual(val, real_acc.get())
    
        def test_AddFunction(self):
            test_values = range(-10,10)
    
            # Set the acc to a known value and to a quick test
            real_acc.set(0) 
            self.assertEqual(0, real_acc.get())
    
            # Test the add commands
            expected_value = 0
            for val in test_values:
                expected_value += val
                real_acc.add(val)
                self.assertEqual(expected_value, real_acc.get())
    
    if __name__ == '__main__':
        unittest.main()
    

**Update based on accepted answer**

If this is only going to be a test script and your mock accumulator isn't used
out side this tests consider the following mod. _I still believe writing the
unittests will serve you better in the long run_

    
    
    class MyAccumlator(object):
        def __init__(self):
            self.acc = 0
    
        def add(self, val):
            self.acc += val
            real_acc.add(val)
            assert(self.check())
            return self.check()
    
        def set(self, val):
            self.acc = val
            real_acc.set(val)
            assert(self.check())
            return self.check()
    
        def check(self):
            return self.acc == real_acc.get()
    

This would allow you to iterate over whatever lists you want without having to
think about calling the check function. There are two methods provided that
you could check.

  1. Leaving the `assert` call would throw an exception if they ever didn't match (only suggesting under the assumption this will live as a test script). 

  2. Removing the `assert` and checking the return status from your calling script (rather than an explicit call to `check()` might clean things up as well.

