title: "Multi-process" vs. "single-process multi-threading" for software modules communicating via messaging
body: <p>We need to build a software framework (or middleware) that will enable messaging between different software components (or modules) running on a single machine. This framework will provide such features:</p>
  
  <ul>
  <li>Communication between modules are through 'messaging'.</li>
  <li>Each module will have its own message queue and message handler thread that will synchronously handle each incoming message.</li>
  </ul>
  
  <p>With the above requirements, which of the following approach is the correct one (with its reasoning)?:</p>
  
  <ol>
  <li>Implementing modules as processes, and messaging through shared memory</li>
  <li>Implementing modules as threads in a single process, and messaging by pushing message objects to the destination module's message queue.</li>
  </ol>
  
  <p>Of source, there are some apparent cons &amp; pros:</p>
  
  <ul>
  <li>In Option-2, if one module causes segmentation fault, the process (thus the whole application) will crash. And one module can access/mutate another module's memory directly, which can lead to difficult-to-debug runtime errors.</li>
  <li>But with Option-1, you need to take care of the states where a module you need to communicate has just crashed. If there are N modules in the software, there can be 2^N many alive/crashed states of the system that affects the algorithms running on the modules.</li>
  <li>Again in Option-1, sender cannot assume that the receiver has received the message, because it might have crashed at that moment. (But the system can alert all the modules that a particular module has crashed; that way, sender can conclude that the receiver will not be able to handle the message, even though it has successfully received it)</li>
  </ul>
  
  <p>I am in favor of Option-2, but I am not sure whether my arguments are solid enough or not. What are your opinions?</p>
  
  <p>EDIT: Upon requests for clarification, here are more specification details:</p>
  
  <ul>
  <li>This is an embedded application that is going to run on Linux OS.</li>
  <li>Unfortunately, I cannot tell you about the project itself, but I can say that there are multiple components of the project, each component will be developed by its own team (of 3-4 people), and it is decided that the communication between these components/modules are through some kind of messaging framework.</li>
  <li>C/C++ will be used as programming language.</li>
  <li>What the 'Module Interface API' will automatically provide to the developers of a module are: (1) An message/event handler thread loop, (2) a synchronous message queue, (3) a function pointer member variable where you can set your message handler function.</li>
  </ul>