Automated testing during development helps ensure that software works according to the test suite. Traditional test suites verify a few well-picked scenarios or example inputs. However, such example-based testing does not uncover errors in legal inputs that the test writer overlooked. We propose theory-based testing as an adjunct to example-based testing. A theory generalizes a (possibly infinite) set of example-based tests. A theory is an assertion that should be true for any data, and it can be exercised by human-chosen data or by automatic data generation. A theory is expressed in an ordinary programming language, it is easy for developers to use (often even easier than example-based testing), and it serves as a lightweight form of specification. Six case studies demonstrate the utility of theories that generalize existing tests to prevent bugs, clarify intentions, and reveal design problems.
For ten years now, test-infected Java programmers have looked forward to that green bar to know that things are as they should be. My four-year-old son knows to bug me to make sure that the bar is green.
However, also think of the green bar as a place where we can meet, share a drink*, and talk about what we want to see happen in software, and the world. The sign on the back wall says “All that’s made is first named.” At the bar, we’ll name it, and then go out to make it.
P.S. I’ve blogged before. Those posts will stay up where they are, and new material will be added here.
* Actually, only talking about software will happen here. For the drinking, and talking about the world, you’ll have to drop me a mail.