Python 2.7 - Having trouble outputting a command line percentage bar for large file download (I want output like: 0%...25%...50%...75%...100%)?
Question: Apologies if this seems a basic question, I have been trying to figure out how
to do this for the past hour with no progress. I have this method in Python
for downloading big (10GB+) files:

    
    
    def downloadChunks(url, destination):
        """Helper to download large files
           the file will be downloaded
           in chunks and print out how much remains
        modified slightly from: https://gist.github.com/gourneau/1430932
        """
    
        baseFile = os.path.basename(url)
    
        #move the file to a more uniq path
        os.umask(0002)
        try:
            file = os.path.join(destination,baseFile)
    
            req = urllib2.urlopen(url)
            total_size = int(req.info().getheader('Content-Length').strip())
            downloaded = 0
            CHUNK = 256 * 10240 # 256 * 10240 == 2560 KB
            with open(file, 'wb') as fp:
                while True:
                    chunk = req.read(CHUNK)
                    downloaded += len(chunk)
                    #print math.floor( (downloaded / total_size) * 100 ),
            if math.floor( (downloaded / total_size) * 100 ) % 100000000 ==0:
                #print "downloaded ="+str(downloaded)+" bytes"
                #print "total_size ="+str(total_size)+" bytes ("+str( (float(downloaded) / float (total_size)) )+"%)"
                #print ""
    
                    if not chunk: break
                    fp.write(chunk)
    
        except urllib2.HTTPError, e:
            print "HTTP Error:",e.code , url
            return False
        except urllib2.URLError, e:
            print "URL Error:",e.reason , url
            return False
    
        return file
    

**The problem:**

As this method runs I basically want output like so:
0%...25%...50%...75%...100% or 10%...20%... etc. (As they are big files I want
the granularity to be adjustable by a constant.)

I know to use print(message,end="") but I have been having trouble with the
algorithm.

**What I've tried:**

You can see some of my past experiments commented out, they gave this output,
which is obviously too long for a log file.

    
    
    downloaded =2621440 bytes
    total_size =611886812 bytes (0.00428419104414%)
    
    downloaded =5242880 bytes
    total_size =611886812 bytes (0.00856838208829%)
    
    downloaded =7864320 bytes
    total_size =611886812 bytes (0.0128525731324%)
    
    downloaded =10485760 bytes
    total_size =611886812 bytes (0.0171367641766%)
    
    downloaded =13107200 bytes
    total_size =611886812 bytes (0.0214209552207%)
    
    downloaded =15728640 bytes
    total_size =611886812 bytes (0.0257051462649%)
    
    downloaded =18350080 bytes
    total_size =611886812 bytes (0.029989337309%)
    
    downloaded =20971520 bytes
    total_size =611886812 bytes (0.0342735283531%)
    
    downloaded =23592960 bytes
    total_size =611886812 bytes (0.0385577193973%)
    
    downloaded =26214400 bytes
    total_size =611886812 bytes (0.0428419104414%)
    
    downloaded =28835840 bytes
    total_size =611886812 bytes (0.0471261014856%)
    
    downloaded =31457280 bytes
    total_size =611886812 bytes (0.0514102925297%)
    
    downloaded =34078720 bytes
    total_size =611886812 bytes (0.0556944835739%)
    
    downloaded =36700160 bytes
    total_size =611886812 bytes (0.059978674618%)
    

Just wondering if anyone can help me do this? Thanks very much.

**EDIT:**

I also wrote this small program, which gives incorrect output:

program:

    
    
    #!/usr/bin/python
    import os
    import math
    scale = 100.0
    CHUNK_SIZE=77   
    FILE_SIZE_TOTAL =611886812
    count =0.0
    
    printnum = 25
    numIncrememnts = math.floor( scale /printnum)
    incrementsize = FILE_SIZE_TOTAL / numIncrememnts
    
    output =0
    
    
    while True:
            if math.floor(count) % math.floor(incrementsize) ==0:
                    print str(output)+"%"
                    output= output+printnum
            count = count+CHUNK_SIZE
            if count >= FILE_SIZE_TOTAL:
                    break
    

output:

    
    
    khennessy@organization-9758:~/Desktop/VM_deployer$ ./test3.py 
    0%
    khennessy@organization-9758:~/Desktop/VM_deployer$ 
    

When CHUNK_SIZE ==1 it works fine though. Here is the output for that:

    
    
    khennessy@organization-9758:~/Desktop/VM_deployer$ ./test3.py 
    0%
    25%
    50%
    75%
    khennessy@organization-9758:~/Desktop/VM_deployer$ 
    

**EDIT - My final answer:**

code:

    
    
    def downloadChunks(url, destination):
        """Helper to download large files
           the file will be downloaded
           in chunks and print out how much remains
        modified slightly from: https://gist.github.com/gourneau/1430932
        """
    
        baseFile = os.path.basename(url)
    
        #move the file to a more uniq path
        os.umask(0002)
        try:
            file = os.path.join(destination,baseFile)
    
            req = urllib2.urlopen(url)
            total_size = int(req.info().getheader('Content-Length').strip())
            downloaded = 0
            CHUNK = 256 * 10240 # 256 * 10240 == 2560 KB
        val = -1.0
        savedval=0.0
            with open(file, 'wb') as fp:
                while True:
                    chunk = req.read(CHUNK)
                    downloaded += len(chunk)
            savedval = val      
            val = 5 * (20 * downloaded / total_size)
            if val != savedval:
                printInfo(str(val)+"% downloaded. ("+str(downloaded)+" bytes / "+str(total_size)+" bytes)") 
                    if not chunk: break
                    fp.write(chunk)
    
        except urllib2.HTTPError, e:
            print "HTTP Error:",e.code , url
            return False
        except urllib2.URLError, e:
            print "URL Error:",e.reason , url
            return False
    
        return file
    

output:

    
    
    [INFO] - 0% downloaded. (2621440 bytes / 611886812 bytes)
    [INFO] - 5% downloaded. (31457280 bytes / 611886812 bytes)
    [INFO] - 10% downloaded. (62914560 bytes / 611886812 bytes)
    

Thanks JuniorCompressor!


Answer: It seems you don't multiply your progress by 100 to get a percentage.

Also you can round this percentage like this:

    
    
    print "total_size =%d bytes (%.0f%%)" % (total_size, 100. * downloaded / total_size)
    

If you want your percentages to be multiples of 25 you can use the following
code:

    
    
    print "total_size =%d bytes (%d%%)" % (total_size, 25 * (4 * downloaded / total_size))
    

`4 * downloaded / total_size` because of the integer division can have `0, 1,
2, 3, 4` as possible values, depending on which range the progress belongs
(0%-25%, 25%-50%, 50%-75%, 75%-100%).

Similarly, you use `10 * (10 * downloaded / total_size)` to get multiples of
10.

