title: Remote rsync in parallel
body: <p>I'm trying to run rsync over ssh in parallel to transfer files between two machines for evaluation purposes. I wanna see how faster can I get compared to a single rsync process.</p>
  
  <p>I tried these two solutions:
  <a href="https://wiki.ncsa.illinois.edu/display/~wglick/Parallel+Rsync" rel="nofollow">https://wiki.ncsa.illinois.edu/display/~wglick/Parallel+Rsync</a> but with no great success.
  <a href="https://gist.github.com/rcoup/5358786" rel="nofollow">https://gist.github.com/rcoup/5358786</a> (I couldn't make it work)</p>
  
  <p>Based on the <a href="https://wiki.ncsa.illinois.edu/display/~wglick/Parallel+Rsync" rel="nofollow">first link</a> I run a command like this: </p>
  
  <pre><code>ssh HOST "mkdir -p ~/destdir/basefolder"
  cd ./basefolder; ls | xargs -n1 -P 4 -I% rsync -arvuz -e ssh % HOST:~/destdir/basefolder/.
  </code></pre>
  
  <p>and I get the files transfered, but it doesn't seem to work well... In this case, It will run a process for every file and folder in the basefolder, but when it finds a folder, it will transfer everything inside that folder using only 1 process.</p>
  
  <p>I tried to use find -type f, but I got problems because I loose the file hierarchy.</p>
  
  <p>Does anyone how some methods to do what I want? (Use rsync in parallel over ssh while keeping files and folders hierarchy).</p>