Python text processing and parsing
Question: I have a file in `gran/config.py` AND I cannot import this file (not an
option).

Inside this config.py, there is the following code

    
    
    ...<more code>
    
        animal = dict(
            bear = r'^bear4x',
            tiger = r'^.*\tiger\b.*$'
        )
    
    ...<more code>
    

I want to be able parse `r'^bear4x'` or `r'^.*\tiger\b.*$'` based on bear or
tiger.

I started out with

    
    
    try:
        text = open('gran/config.py','r')
        tline = filter('not sure', text.readlines())
        text.close()
    except IOError, str:
        pass
    

I was hoping to grab the whole animal dict by  
`grab = re.compile("^animal\s*=\s*('.*')")` or something like that  
and maybe change `tline` to `tline = filter(grab.search,text.readlines())`

but it only grabs `animal = dict(` and not the following lines of dict.

how can i grab multiple lines?  
look for animal then confirm the first '(' then continue to look until ')' ??

Note: the size of animal dict may change so anything static approach (like
grab 4 extra lines after animal is found) wouldnt work


Answer: I am not getting what exactly are you trying to do.

If you want to process **each line** with regular expression - you have `^` in
regular expression `re.compile("^animal\s*=\s*('.*')")`. It matches only when
`animal` is at the start of line, not after some spaces. Also of course it
does not match `bear` or `tiger` \- use something like
`re.compile("^\s*([a-z]+)\s*=\s*('.*')")`.

If you want to process **multiple lines** with single regular expression, read
about re.DOTALL and re.MULTILINE and how they affect matching newline
characters:

<http://docs.python.org/2/library/re.html#re.MULTILINE>

Also note that text.readlines() reads lines, so the filter function in
`filter('not sure', text.readlines())` is run on each line, not on whole file.
You cannot pass regular expression in this `filter(<re here>,
text.readlines())` and hope it will match multiple lines.

BTW processing Python files (and HTML, XML, JSON... files) using regular
expressions is not wise. For every regular expression you write there are
cases where it will not work. Use parser designed for given format - for
Python source code it's [ast](http://docs.python.org/2/library/ast.html). But
for your use case `ast` is too complex.

Maybe it would be better to use classic config files and
[configparser](http://docs.python.org/2/library/configparser.html). More
structured data like lists and dicts can be easily stored in
[JSON](http://docs.python.org/2/library/json.html) or
[YAML](https://bitbucket.org/xi/pyyaml) files.

