title: How can I improve ftp upload speed for multiple files [Java]
body: <p>I implemented java code to upload files to server with org.apache.commons.net.ftp.FTPClient
  For multiple files the ftp upload speed is very slow.
  How can I improve the speed.</p>
  
  <p>-Change library? 
  What is the powerful FTP client class library for uploading multiple files?</p>
  
  <p>-Use multiple threads?
  How can I implement ftp upload function with multiple thread? 
  Could someone show me an example?
  I am a new for multiple threading programming.</p>
  
  <hr>
  
  <p>After I read all answer, I try to change my code and test it.</p>
  
  <p><strong>Following is a sample FTPClient code:</strong></p>
  
  <pre><code>// create instance of FTPClient
  FTPClient ftp = new FTPClient();
  
  ftp.setControlEncoding("UTF-8");
  ftp.setDefaultTimeout(30000);
  
  
  // connect to server
  try
  {
      ftp.connect("10.1.1.1", 990);
  }
  catch(Exception e)
  {
      System.out.println("Cannot connect to server");
      return;
  }
  
  // login to server
  if (!ftp.login("username", "password"))
      {
          ftp.logout();
          System.out.println("Cannot login to server");
          return;
      }
  
  try
  {
      ftp.setFileTransferMode(FTP.BINARY_FILE_TYPE);
      ftp.enterLocalPassiveMode();
      // ftp.setBufferSize(0); &lt;-- someone suggest me to set buffer size to 0, but it throw error sometime.
  }
  catch(Exception e)
  {
  }
  // create directory on server
  // dirs is list of required directories on server
  for (String dir : dirs)
      {
          try
          {
              ftp.makeDirectory(dir);
          }
          catch(IOException e)
          {
          }
      }
  
  
  // files is a map of local file and string of remote file
  // such as 
  // file on client is "C://test/a.txt"
  // location on server is "/test/a.txt"
  for (Map.Entry&lt;File, String&gt; entry : files.entrySet())
      {
          File localFile = entry.getKey();
          String remoteFile = entry.getValue();
  
          FileInputStream input = null;
          try
          {
              input= new FileInputStream(localFile);
              ftp.storeFile(remoteFile, input);
          }
          catch (Exception e)
          {
              try
              {
                  ftp.deleteFile(remoteFile);
              }
              catch (IOException e1)
              {
              }
          }
          finally
          {
              if (input != null)
              {
                  try
                  {
                      input.close();
                  }
                  catch (IOException e)
                  {
                  }
              }
          }
      }
  
  // disconnect
  if (ftp != null &amp;&amp; ftp.isConnected())
      {
          try
          {
              ftp.disconnect();
          }
          catch (IOException f)
          {
              // do nothing
          }
      }
  </code></pre>
  
  <p>When I uploaded 1050 files (each file is about 1-20 KB), it took about 49406 - 51000 millisec (This is upload time only). 
  I would like to improve the speed. </p>
  
  <p>Some people suggest me to use ftp4j, but when I test the library with 1050 files, the upload speed of ftp4j is slowly than FTPClient about 10000 millisec. 
  it took about  60000 millisec.</p>
  
  <p><strong>Following is sample ftp4j code:</strong></p>
  
  <pre><code>// create instance of FTPClient 
  FTPClient ftp = new FTPClient();
  
  ftp.setCharset("UTF-8");
  
  // connect to server
  try
  {
      ftp.connect("10.1.1.1", 990);
  }
  catch(Exception e)
  {
      System.out.println("Cannot connect to server")
      return;
  }
  
  // login to server  
  try
  {
      ftp.login("username", "password");
  }
  catch (Exception e)
  {
      try
      {
        ftp.logout();
      }
      catch (Exception e1)
      {
      }
      System.out.println("Cannot login to server")
      return;
  }
  
  try
  {
      ftp.setType(FTPClient.TYPE_BINARY);
      ftp.setPassive(true);
  }
  catch(Exception e)
  {   
  }
  
  // create directory on server  
  // dirs is list of required directories on server  
  for (String dir : dirs)
  {
      try
      {
        ftp.createDirectory(dir);
      }
      catch (Exception e)
      {
      }
  }
  
  // files is a map of local file and string of remote file  
  // such as   
  // file on client is "C://test/a.txt"  
  // location on server is "/test/a.txt" 
  for (Map.Entry&lt;File, String&gt; entry : files.entrySet())
  {
      final File localFile  = entry.getKey();
      final String remoteFile  = entry.getValue();
  
      BufferedInputStream input = null;
      boolean success = false;
      try
      {
        input = new BufferedInputStream(new FileInputStream(localFile));
  
         // ftp.upload(localFile); &lt;-- if I use ftp.upload(File), it will took more time. 
         ftp.upload(remoteFile, input, 0, 2048, new MyTransferListener());
         success = true;
      }
      catch (Exception e)
      {
      }
      finally
      {
          if (input != null)
          {
              try
              {
                  input.close();
              }
              catch (IOException e)
              {
              }
          }
          if (!success)
          {
              try
              {
                ftp.deleteFile(remoteFile);
              }
              catch (Exception e)
              {
              }
          }
      }
  }
  
  // disconnect
  if (ftp != null &amp;&amp; ftp.isConnected())  
  {  
      try  
      {  
        ftp.disconnect();  
      }  
      catch (IOException f)  
      {  
        // do nothing  
      }  
  } 
  </code></pre>
  
  <p>I try to use multiple threads.</p>
  
  <p><strong>Following is multiple threading code:</strong> </p>
  
  <pre><code>final CountDownLatch latch = new CountDownLatch(files.size());
  ExecutorService pool = Executors.newFixedThreadPool(10);
  for (Map.Entry&lt;File, String&gt; entry : files.entrySet())
  {
      final File localFile = entry.getKey();                  
      final String remoteFile = entry.getValue();
  
      pool.execute(new Runnable() {
          public void run()
          {
              FileInputStream input = null;
              try
              {
                  input= new FileInputStream(localFile);
                  ftp.storeFile(remoteFile, input);
              }
              catch (Exception e)
              {
                  try
                  {
                      ftp.deleteFile(remoteFile);
                  }
                  catch (IOException e1)
                  {
                  }
              }
              finally
              {
                  if (input != null)
                  {
                      try
                      {
                          input.close();
                      }
                      catch (IOException e)
                      {
                      }
                  }
                 latch.countDown();
              }
          }
      });
  }
  try
  {
    // waiting for all threads finish
    // see: http://stackoverflow.com/questions/1250643/how-to-wait-for-all-threads-to-finish-using-executorservice
    latch.await();
  }
  catch(Exception e)
  {
  }
  </code></pre>
  
  <p>Is it correct? It work correctly but it cannot improve speed. 
  it took about 49000 - 51000 millisec same as the code without thread.</p>
  
  <p>I test the speed with intranet. It will take more time for internet.</p>
  
  <p>How should I do for improve upload speed?</p>