connecting to a remote server and downloading and extracting binary tar file using python
Question: I would like to connect to a remote server, download and extract binary tar
file into a specific directory on that host. I am using python 2.6.8

  1. What would be a simple way to ssh to that server?
  2. I see the below errors on my script to download the tar file and extract it

Traceback (most recent call last): File "./wgetscript.py", line 16, in tar =
tarfile.open(file_tmp) File "/usr/lib64/python2.6/tarfile.py", line 1653, in
open return func(name, "r", fileobj, **kwargs) File
"/usr/lib64/python2.6/tarfile.py", line 1715, in gzopen fileobj =
bltn_open(name, mode + "b") TypeError: coercing to Unicode: need string or
buffer, tuple found

    
    
    #!/usr/bin/env python
    
    import os
    import tarfile
    import urllib
    
    url = 'http://**************/Lintel/Mongodb/mongodb-linux-x86_64-enterprise-suse12-3.2.6.tgz'
    
    fullfilename = os.path.join('/tmp/demo1','file.tgz')
    file_tmp = urllib.urlretrieve(url,fullfilename)
    print file_tmp
    base_name = os.path.basename(url)
    print base_name
    file_name, file_extension = os.path.splitext(base_name)
    print file_name, file_extension
    tar = tarfile.open(file_tmp)
    nameoffile = os.path.join('/tmp/demo1','file')
    tar.extractall(file_name,nameoffile)
    tar.close()


Answer: There are 2 errors here:

  * `urllib.urlretrieve` (or `urllib.requests.urlretrieve` in Python 3) returns a `tuple`: filename, httprequest. You have to unpack the result in 2 values (or in the original `fullfilename`)
  * download is OK but the `tarfile` module doesn't work the way to think it works: `tar.extractall` takes 2 arguments: path of the .tar/.tgz file and optional `list` of members (that you can get with `tar.getmembers()` in your case). For this example, I propose that we drop that filter and extract all the contents in the temp directory.

Fixed code:

    
    
    url = 'http://**************/Lintel/Mongodb/mongodb-linux-x86_64-enterprise-suse12-3.2.6.tgz'
    temp_dir = '/tmp/demo1'
    
    fullfilename = os.path.join(temp_dir,'file.tgz')
    # urllib.request in Python 3
    file_tmp,http_message = urllib.urlretrieve(url,fullfilename)
    base_name = os.path.basename(url)
    file_name, file_extension = os.path.splitext(base_name)
    tar = tarfile.open(file_tmp)
    
    #print(tar.getmembers()[0:5])  # would print 5 first members of the archive
    
    tar.extractall(os.path.join(temp_dir,file_name))
    tar.close()
    

