Python Pre-testing for exceptions when coverage fails
Question: I recently came across a simple but nasty bug. I had a list and I wanted to
find the smallest member in it. I used Python's built-in min(). Everything
worked great until in some strange scenario the list was empty (due to strange
user input I could not have anticipated). My application crashed with a
ValueError (BTW - not documented in the official docs).

I have very extensive unit tests and I regularly check coverage to avoid
surprises like this. I also use Pylint (everything is integrated in PyDev) and
I never ignore warnings, yet I failed to catch this bug before my users did.

Is there anything I can change **in my methodology** to avoid these kind of
runtime errors? (which would have been caught at compile time in Java / C#?).

I'm looking for something more than wrapping my code with a big try-except.
What else can I do? How many other build in Python functions are hiding nasty
surprises like this???


Answer: The problem here is that malformed external input crashed your program. The
solution is to exhaustively unit test possible input scenarios at the
boundaries of your code. You say your unit tests are 'extensive', but you
clearly hadn't tested for this possibility. Code coverage is a useful tool,
but it's important to remember that covering code is **not** the same as
thoroughly testing it. Thorough testing is a combination of covering usage
scenarios as well as lines of code.

The methodology I use is to trust internal callers, but never to trust
external callers or input. So I explicitly **don't** unit test for the empty
list case in any code beyond the first function that receives the external
input. But that input function should be **exhaustively** covered.

In this case I think the library's exception is reasonable behaviour - it
makes no sense to ask for the `min` of an empty list. The library can't
legitimately set a value such as 0 for you, since you may be dealing with
negative numbers, for example.

I think the empty list should never have reached the code that asks for the
`min` \- it should have been identified at input, and either raised an
exception there, or set it to 0 if that works for you, or whatever else it is
that does work for you.

