title: How to set up communication between two processes?
body: <p>I have the following situation:</p>
  
  <ol>
  <li><p>A daemon that does a privileged operation on data that is kept in memory.</p></li>
  <li><p>A multithreaded server currently running on about 30 cores handling user requests.</p></li>
  </ol>
  
  <p>The server (1) would receive queries from (2), process them one by one, and return an answer. Each query to (1) would never block and only take a fraction of a microsecond on (1) to process, so we are guaranteed to get responses back fast unless (1) gets overrun by too much load.</p>
  
  <p>Essentially, I would like to set up a situation where (1) listens to a UNIX domain socket and (2) writes requests and reads responses. However, I would like each thread of (2) to be able to read and write concurrently. My idea is to have one UNIX socket per thread for communication between (1) and (2) have (1) block on epoll_wait on these sockets processing requests one by one. Each thread on (2) would then read and write independently to its socket.</p>
  
  <p>The problem that I see with this approach is that I can't easily dynamically grow the number of threads on (2). Is there a way to accomplish this in a way that is flexible with respect to runtime configuration? I guess one approach would be to have a large number of sockets and a thread on (2) would pick one socket by random, take a mutex on it, write a query and block waiting for a response, then release the mutex once it gets a response back from (1).</p>
  
  <p>Anyone have better ideas?</p>