In vs re.search python
Question: I have a python 2.7 script which usually runs for hours, and I am now trying
to optimize it. It has a lot of searches for strings, which represents the
heavy part of computation. At the moment I am currently using
`re.search('stringToFind', haystack)` to find substrings in longer strings. I
am considering rewriting all expressions that do not contain regular
expressions and use `in` instead. Almost all strings that are searched for are
normal strings, ie contains no regular expressions.

I know that `in` uses a `contains` method to check substrings, but I have no
clue how re.search works for strings that are non-re. How does re.search
actually work? Would it really be more efficient to use in rather than
re.search for substring in string comparision?

EDIT: To clarify I can give an example of currently running code:

    
    
    if re.search('extern', variable):
        # Do something...
    

Could be replace by:

    
    
    if 'extern' in variable:
        # Do  something...
    


Answer: As others have commented, executing a regular expression search is inherently
more complex than a simple substring search. Here's a very rough benchmark:

    
    
    import timeit
    import re
    s = '0' * 1000 + 'foo' + '0' * 1000
    rgx = re.compile(r'foo')
    setup = 'from __main__ import s, rgx'
    print timeit.timeit('"foo" in s', setup = setup)             # 0.745168924332
    print timeit.timeit('rgx.search(s).group(0)', setup = setup) # 1.14249396324
    

But if your program runs for hours, I doubt that the source of the problem
lies in such areas.

