title: Handling more than 1024 file descriptors, in C on Linux
body: <p>I am working on a threaded network server using epoll (edge triggered) and threads and I'm using httperf to benchmark my server.</p>
  
  <p>So far, it's performing really well or almost exactly at the rate the requests are being sent. Until the 1024 barrier, where everything slows down to around 30 requests/second.</p>
  
  <p>Running on Ubuntu 9.04 64-bit.</p>
  
  <p>I've already tried:</p>
  
  <ul>
  <li>Increasing the ulimit number of file descriptors, successfully. It just doesn't improve the performance above 1024 concurrent connections.
  
  <blockquote>
    <p>andri@filefridge:~/Dropbox/School/Group 452/Code/server$ ulimit -n<br>
    20000</p>
  </blockquote></li>
  </ul>
  
  <p>I am pretty sure that this slow-down is happening in the operating system as it happens before the event is sent to epoll (and yes, I've also increased the limit in epoll).</p>
  
  <p>I need to benchmark how many concurrent connections my program can handle until it starts to slow down (without the operating system interfering).</p>
  
  <p><strong>How do I get my program to run with more than 1024 file descriptors?</strong></p>
  
  <p>This limit is probably there for a reason, but for benchmarking purposes, I need it gone.</p>
  
  <h2>Update</h2>
  
  <p>Thanks for all your answers but I think I've found the culprit. After redefining __FD_SETSIZE in my program everything started to move a lot faster. Of course ulimit also needs to be raised, but without __FD_SETSIZE my program never takes advantage of it.</p>