Removing lines from a text file using python and regular expressions
Question: I have some text files, and I want to remove all lines that begin with the
asterisk (“*”).

Made-up example:

    
    
    words
    *remove me
    words
    words
    *remove me 
    

My current code fails. It follows below:

    
    
    import re
    
    program = open(program_path, "r")
    program_contents = program.readlines()
    program.close() 
    
    new_contents = []
    pattern = r"[^*.]"
    for line in program_contents:
        match = re.findall(pattern, line, re.DOTALL)
        if match.group(0):
            new_contents.append(re.sub(pattern, "", line, re.DOTALL))
        else:
            new_contents.append(line)
    
    print new_contents
    

This produces ['', '', '', '', '', '', '_', '', '_ ', '', '*', ''], which is
no goo.

I’m very much a python novice, but I’m eager to learn. And I’ll eventually
bundle this into a function (right now I’m just trying to figure it out in an
ipython notebook).

Thanks for the help!


Answer: You _don't_ want to use a `[^...]` negative character class; you are matching
_all_ characters except for the `*` or `.` characters now.

`*` is a meta character, you want to escape that to `\*`. The `.` 'match any
character' syntax needs a multiplier to match more than one. Don't use
`re.DOTALL` here; you are operating on a line-by-line basis but don't want to
erase the newline.

There is no need to test first; if there is nothing to replace the original
line is returned.

    
    
    pattern = r"^\*.*"
    for line in program_contents:
        new_contents.append(re.sub(pattern, "", line))
    

Demo:

    
    
    >>> import re
    >>> program_contents = '''\
    ... words
    ... *remove me
    ... words
    ... words
    ... *remove me 
    ... '''.splitlines(True)
    >>> new_contents = []
    >>> pattern = r"^\*.*"
    >>> for line in program_contents:
    ...     new_contents.append(re.sub(pattern, "", line))
    ... 
    >>> new_contents
    ['words\n', '\n', 'words\n', 'words\n', '\n']
    

