title: Get output continuously: rsync info=progress2 call in python script
body: <p>I'm calling rsync with popen and the output is not printing out continuously in my python script for my web application as it does in just normal linux. I'm trying to copy over all the files on one directory to another (massive copy). I want to use the progress numbers received from the output changing to eventually create/update a progress-bar I have in my web application. All I want is the total progress of the overall copy so I would use --info=progress2 in my rsync command.
   I also tried :</p>
  
  <pre><code>while True:
          line = self.proc.stdout.readline()
          if line != '':
              # the real code does filtering here
              print("test:", line.rstrip())
          else:
              break
  </code></pre>
  
  <p>but that waited till the end to just print test: b''
  I think the issues is either with the while loop extracting data or how I print it to my console using a different class too.</p>
  
  <p>There is not much information of using this --info=progress2
  since it's a relatively new update.</p>
  
  <p>Here's my code.</p>
  
  <pre><code>import subprocess
  import logging
  import sys
  import os
  import replicator.dfp.rep_utils as ru
  
  
  class SyncProcessor(object):
      def __init__(self, src, dest):
          self.src = src
          self.dest = dest
          self.proc = None
          self.callback = None
          log_file = "sync-{}-{}.log".format(self.src, self.dest)
          self.sync_logger = ru.setup_logger(__file__, log_file, level=logging.DEBUG)
  
      def add_sync_callback(self, cb):
          self.callback = cb
  
      def run(self):
          print("Syncing Drive "+ str(self.src.driveNum) + " to Drive " + str(self.dest.driveNum))
          rsync_cmd = "sudo rsync -ah --info=progress2 --delete --stats /media/drive{}/ /media/drive{}".format(self.src.driveNum, self.dest.driveNum)
          self.proc = subprocess.Popen(rsync_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  
          while self.proc.poll() is None:
              output = self.proc.stdout.readline()
              if output == '':
                  break
              if output:
                  print("OUTPUT DECODE: " + output.decode("utf-8")
                  #self.sync_logger.info(output.decode())
                  self.callback.update(output.decode())
          print("&lt;&lt; Finished Syncing &gt;&gt;")
          #self.sync_logger.debug("&lt;&lt; Finished Syncing &gt;&gt;")
          rc = self.proc.poll()
          #self.sync_logger.debug("Return code: {}".format(rc))
          os.system("sync")
          return rc
  
      def communicate(self):
          return self.proc.communicate()
  
  class Progress(object):
      """Callback to report progress of a SyncProcessor"""
      def __init__(self, src, dest, out=sys.stdout):
          self.src = src
          self.dest = dest
          self.out = out
  
      def update(self, data):
          line = "From Progress({}-{}) -&gt; {}"
      self.out.write(line.format(self.src, self.dest, data))
  </code></pre>