title: Netty Not Scaling for Higher concurrency
body: <p>We are working on a Netty based HTTP server which receives a Get Request, decodes the request parameters, makes a couple of webservice  calls and then returns the result after some processing</p>
  
  <p>A typical request generally completes in around 320ms out of which apprixmately 300ms are spent waiting for a response from the remote servers.</p>
  
  <p>The problem is that we are not able to generate high throughput on the setup.</p>
  
  <p>This is a 16 core machine with 30 GB RAM. The cpu usage never goes beyond 20% and RAM udage never goes beyond 5GB. The java heap size is set to 25GB</p>
  
  <p>Additionally we have updated the following OS settings</p>
  
  <pre><code>ulimit =65000
  net.core.somaxconn = 2048
  net.ipv4.tcp_tw_recycle = 1
  net.ipv4.tcp_tw_reuse = 1
  net.ipv4.ip_local_port_range = 10240    65535
  </code></pre>
  
  <p>This is an ubuntu machine and we are using the 3.2 branch of Netty</p>
  
  <p>The max throughput that we can achive on the server is around 300 requests per second which also goes down with each subsequent test which finally goes down to 35 req/s.</p>
  
  <p>We have tried using the FixThreadPool as well as the cachedthreadpool while creating the netty server bootstrap.</p>
  
  <p>Whatever we do the performance numbers do not change</p>
  
  <p>Our guess is that Netty is not creating enough threads and hence a lot of time is wasted in the queue.</p>
  
  <p>Is there something basic that we are missing due to which netty is giving us fixed concurrency?</p>
  
  <p>How do we make maximum use of our CPU/RAM to get maximum performance from our hardware?</p>
  
  <p>here is the code used for netty initialization</p>
  
  <pre><code>   ServerBootstrap bootstrap = new ServerBootstrap(
                  new NioServerSocketChannelFactory(
                          Executors.newCachedThreadPool(),
                          Executors.newCachedThreadPool()));
                          //Executors.newFixedThreadPool(5000),5000));
     // Configure the pipeline factory.
  
  
       bootstrap.setPipelineFactory(this.nettyServerPipeline);
         bootstrap.setOption("child.tcpNoDelay", true);
         bootstrap.setOption("backlog",1000);
         bootstrap.setOption("keepAlive", true);
         bootstrap.setOption("connectTimeoutMillis", 10000);
         bootstrap.setOption("client.reuseAddress", true);
         // Bind and start to accept incoming connections.
         bootstrap.bind(new InetSocketAddress(8099));
      /*  ServerBootstrap bootstrap =
          new ServerBootstrap(new         NioServerSocketChannelFactory(Executors.newFixedThreadPool(1),
              Executors.newFixedThreadPool(32)));
      */
  </code></pre>
  
  <p>We have tried multiple combinations of above settings but nothing seems to improve the absolute throughput.</p>
  
  <p>Regards
  Sanket</p>