How to count number of files available in a directory recursively in Python using rsync?
Question: I want to count number of files recursively in a remote server path using
rsync in Python? I tried it doing like this:

    
    
    def find_remote_files(source, password):
        cmdline = ['sshpass', '-p', password, 'rsync', '--recursive', source]
        with open(os.devnull, "w") as devnull:
            proc = subprocess.Popen(cmdline, stdout=subprocess.PIPE, stderr=devnull)
            try:
                for entry in proc.stdout:
                    items = entry.strip().split(None, 4)
                    if not items[0].startswith("d"):
                        yield lent(items[4])
                proc.wait()
            except:
                # On any exception, terminate process and re-raise exception.
                proc.terminate()
                proc.wait()
                raise
    

It works in a situation where I have less files. But in case if I have more
than 3000 files the rsync will take a long time to store it in a list and
again calculate the length. That's why, I wanted to know if there is a `rsync`
command just to count the file.


Answer: I would use a different approach using fabric, which is a great tool for
executing remote commands.

    
    
    from fabric.api import run, env  
    env.host_string = 'example.org'
    output = run('find /tmp -type f | wc -l')
    num_files = int(output)
    

Now you have the number of files in your variable `num_files`. I was just
using the `find` command to search for files recursively beginning at
directory `/tmp`, and counted the returned lines with `wc -l`.

