title: How to download thirteen million small files faster with wget?
body: <p>I have a file list total 13 million files, about 140GB.
  NOTE: Need maintain directory structure. Each file is approximately 100KB-2MB. </p>
  
  <p>I use the <code>wget -x -i file_list.txt</code></p>
  
  <p>this is work will, but too slow.</p>
  
  <p>NOTE: all is url in file list.</p>
  
  <p>My server have 100M bandwidth, should be get 10M/s speed.
  but <code>wget -x -i file_list.txt</code> just give me 1M/s.</p>
  
  <p>How to fix it?</p>