grep/zgrep within python using subprocess
Question: I have a set of tsvs that are zipped in *.tsv.gz format and some that are not
zipped, ie, *.tsv in a directory.

I want to grep for a string from these files and print the grep results each
in a new line.

I have a function that looks that takes in the input directory in which tsvs
and *.tsv.gz are stored and the string to be searched.

    
    
    import sys, os, traceback,subprocess,gzip,glob
    def filter_from_tsvs(input_dir,string):
    
        tsvs = glob.glob(os.path.join(input_dir,'*.tsv*'))
        open_cmd=open
        for tsvfile in tsvs:
            print os.path.splitext
            extension = os.path.splitext(tsvfile)[1]
            if extension == ".gz":
              open_cmd = gzip.open
        print open_cmd
        try:
            print subprocess.check_output('grep string tsvfile', shell=True)
    
        except Exception as e:
            print "%s" %e
            print "%s" %traceback.format_exc()
    return
    

I have also tried to use:

    
    
             `try:
                 fname = open_cmd(tsvfile,"r")
                 print "opened"
                 print subprocess.check_output('grep string fname', shell=True)`
    

I get this error: `gzip: tsvfile.gz: No such file or directory Command 'zgrep
pbuf tsvfile' returned non-zero exit status 2 Traceback (most recent call
last): File "ex.py", line 23, in filter_from_maintsvs print
subprocess.check_output('zgrep pbuf tsvfile', shell=True) File
"/datateam/tools/opt/lib/python2.7/subprocess.py", line 544, in check_output
raise CalledProcessError(retcode, cmd, output=output) CalledProcessError:
Command 'zgrep pbuf tsvfile' returned non-zero exit status 2`

How can use grep/zgrep within python?  
Thanks in advance for the help.


Answer: I got the following solution after going through a
[blog](https://blog.nelhage.com/2010/02/a-very-subtle-bug/) and it worked for
me :)

    
    
    import subprocess
    import signal
    
    output = subprocess.check_output('grep string tsvfile', shell=True, preexec_fn=lambda: signal.signal(signal.SIGPIPE, signal.SIG_DFL))
    
    print output  
    

