numfail + open(outpathbad'wb'write(outdataconsider both failure can get status+stderr save output to view elif not os path exists(outpathor forcegenprint('generating:'outpathopen(outpath'wb'write(outdatacreate first output elsepriorout open(outpath'rb'read(if priorout =outdataprint('passed:'testnameelsenumfail + print('failed output:'testnameoutpathbadopen(outpathbad'wb'write(outdataor compare to prior print('finished:'time asctime()print('% tests were run% tests failed (len(testfiles)numfail)we've seen the tools used by this script earlier in this part of the book--subprocessos pathglobfilesand the like this example largely just pulls these tools together to solve useful purpose its core operation is comparing new outputs to oldin order to spot changes ("regressions"along the wayit also manages command-line argumentserror messagesstatus codesand files this script is also larger than most we've seen so farbut it' realistic and representative system administration tool (in factit' derived from similar tool actually used in the past to detect changes in compilerprobably the best way to understand how it works is to demonstrate what it does the next section steps through testing session to be read in conjunction with studying the test script' code running the test driver much of the magic behind the test driver script in example - has to do with its directory structure when you run it for the first time in test directory (or force it to start from scratch there by passing second command-line argument)itcollects scripts to be run in the scripts subdirectory fetches any associated script input and command-line arguments from the inputs and args subdirectories generates initial stdout output files for tests that exit normally in the outputs subdirectory reports tests that fail either by exit status code or by error messages appearing in stderr on all failuresthe script also saves any stderr error message textas well as any stdout data generated up to the point of failurestandard error text is saved to file in the errors subdirectoryand standard output of failed tests is saved with special regression test script