Run several Python programs and check for errors
allitems = open('allitems.txt').read().splitlines()



You're putting all of those lines into a list, but you don't need them as a list.  All that you do with them later is iterate, so you should just say allitems = open('allitems.txt').  That will mean that the newline characters will be at the end of each line, but you can fix that by adding item = item.strip() in your loop. You should also always use a with statement when opening files.


kind, chapter_c, number_c = itemparser.match(item).groups()



What if the item doesn't match?  You'll have a very ugly error that won't make sense to whoever uses your program.  You should have some error handling added:

item = item.strip()
match = itemparser.match(item):
if not match or len(match.groups()) != 3:
    print("Invalid item: {}".format(item), file=sys.stderr)
    continue


If you want, you could throw an error instead, but your error would hopefully be more helpful to the user.


number = int(number_c)



Again, no error handling.


status = 'Success'
...
status = 'Failed'
...
status = 'Not implemented'



Those are hard-coded values.  What would happen if you had just one slight typo?  It could throw you off with something completely unexpected and it would be very difficult to debug.  You should define constants at the beginning of the file for the different status codes:

SUCCESS = 'Success'
FAILED = 'Failed'


Well, you don't really need to define one for 'Not implemented' because NotImplemented already exists as a built-in constant.  You could even use True, False, and NotImplemented.  In that case, the variable might be success instead of status.


sys.exit(statuscounter['Failed'])



I like that.  You exit with success if nothing failed, but the error exit codes make sense.