why is "any()" running slower than using loops?
Question: I've been working in a project that manage big lists and pass the lists trough
a lot of tests in order to validate or not each word of the list. The funny
thing is that each time that I've used the "faster" tools or generators (like
the `itertools` module) and I make some tests, they seem to be slower.

Finally I decided to ask the question because it is possible that I be doing
something wrong. The following code will try to test the performance of the
`any()` function vs `loops`.

    
    
    #!/usr/bin/python3
    #
    
    import time
    from unicodedata import normalize
    
    """
    
        Import a large list of tests (like 300Mb). The list contains words each one separated in a line.
    
    
    """
    PATH='./tests'
    
    
    start=time.time()
    with open(PATH, encoding='utf-8', mode='rt') as f:
        tests_list=f.read()
    print('File reading done in {} seconds'.format(time.time() - start))
    
    start=time.time()
    tests_list=[line.strip() for line in normalize('NFC',tests_list).splitlines()]
    print('String formalization, and list strip done in {} seconds'.format(time.time()-start))
    print('{} strings'.format(len(tests_list)))
    
    
    
    """
        test to check if  "any()" is faster.
    
    """
    print('Testing the performance of any()')
    
    unallowed_combinations=['ab','ac','ad','ae','af','ag','ah','ai','af','ax','ae','rt','rz','bt','du','iz','ip','uy','io','ik','il','iw','ww','wp']
    
    
    def combination_is_valid(string):
        if any(combination in string for combination in unallowed_combinations):
            return False
    
        return True
    
    def combination_is_valid2(string):
        for combination in unallowed_combinations:
            if combination in string:
                return False
    
        return True
    
    
    start=time.time()
    for string in tests_list:
        combination_is_valid(string)
    print('combination_is_valid ended in {} seconds'.format(time.time()-start))
    
    
    start=time.time()
    for string in tests_list:
        combination_is_valid2(string)
    print('combination_is_valid2 ended in {} seconds'.format(time.time()-start))  
    

The code I posted is pretty representative of the kind of tests I do with my
program. And if we take a look to the results:

    
    
    File reading done in 0.22988605499267578 seconds
    String formalization, and list strip done in 6.803032875061035 seconds
    38709922 strings
    Testing the performance of any()
    combination_is_valid ended in 80.74802565574646 seconds
    combination_is_valid2 ended in 41.69514226913452 seconds
    
    
    File reading done in 0.24268722534179688 seconds
    String formalization, and list strip done in 6.720442771911621 seconds
    38709922 strings
    Testing the performance of any()
    combination_is_valid ended in 79.05265760421753 seconds
    combination_is_valid2 ended in 42.24800777435303 seconds
    

I find kinda amazing that using the function with loops its half faster than
using `any()`. What would be the explanation to this? Am I doing something
wrong?

I interpret the results as if `any()` generated a list of all the results and
check if they are True or False at the end. In the other hand the function
that uses loops, after a result is created it is tested so it can exit earlier
without generating UN-usefull tests.

What do you think?

(I used python3.4 under GNU-Linux)


Answer: Actually the
[`any()`](https://docs.python.org/3/library/functions.html?highlight=enumerate#any)
function is equal to following function :

    
    
    def any(iterable):
        for element in iterable:
            if element:
                return True
        return False
    

which is like your second function, but since the `any()` returns a boolean
value by itself, you don't need to check for the result and then return a new
value, So the difference of performance is because of that you are actually
use a redundant return and `if` conditions,also calling the `any` inside
another function.

So the advantage of `any` here is that you don't need to wrap it with another
function because it does all the things for you.

Also as @interjay mentioned in comment it seems that the most important reason
which I missed is that you are passing a generator expression to `any()` which
doesn't provide the results at once and since it produce the result on demand
it does an extra job.

Based on [PEP 0289 -- Generator
Expressions](https://www.python.org/dev/peps/pep-0289/)

> The semantics of a generator expression are equivalent to creating an
> anonymous generator function and calling it. For example:
    
    
    g = (x**2 for x in range(10))
    print g.next()
    

is equivalent to:

    
    
    def __gen(exp):
        for x in exp:
            yield x**2
    g = __gen(iter(range(10)))
    print g.next()
    

So as you can see each time that python want to access the next item it calls
the `iter` function and the `next` method of a generator.And finally the
result is that it's overkill to use `any()` in such cases.

