title: How to use distcc to preprocess and compile everything remotely only?
body: <p><strong>Background:</strong></p>
  
  <ul>
  <li>I have a 128-core server which I would like to use as a build server. </li>
  <li>I have a bunch of client machines which work with a not-so-new and not-so-powerful PC. <em>(Can't upgrade! Not in my control.)</em></li>
  </ul>
  
  <p><strong>What I did:</strong></p>
  
  <ul>
  <li>I followed the <a href="https://distcc.github.io/" rel="nofollow noreferrer">distcc documentation</a>.</li>
  <li>And installed a virtual machine on the server with exactly the same compiler version, the same distcc version -- basically the same distribution, as on the client-machines.</li>
  <li>After configuring the clients and the servers, I can remotely build. I can verify this using the <code>distccmon-text</code> tool. I can see on the server, there are 8 threads started by the distcc daemon and that are awaiting for build-jobs to come. This was good as a first step. You can see the output below to be sure.</li>
  </ul>
  
  <p><a href="https://i.stack.imgur.com/ON81H.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ON81H.png" alt="enter image description here"></a></p>
  
  <ul>
  <li>Second Step: Since the client machines are dual-core machines while the server offers 128-cores, and not all clients will be compiling at the same time, I wanted to offload as much of the build as possible to the build-server.</li>
  </ul>
  
  <p><strong>Problems:</strong></p>
  
  <ol>
  <li><em>First Problem:</em> distcc, no matter how I try to configure it, always tries to distribute the build-jobs equally on the client and the server. Even though my configuration file looks as shown below:</li>
  </ol>
  
  <pre><code># --- /etc/distcc/hosts -----------------------
  # See the "Hosts Specification" section of
  # "man distcc" for the format of this file.
  #
  # By default, just test that it works in loopback mode.
  # 127.0.0.1
  172.24.26.208/8,cpp,lzo
  localhost/0
  </code></pre>
  
  <p>Which as per the <a href="https://distcc.github.io/" rel="nofollow noreferrer">distcc documentation</a> should give higher priority to the build-server and lower-priority to the localhost since it comes later in the list. Also, it should give 8 jobs to the build server and 0 jobs to the localhost. But no, that doesn't happen. Upon typing <code>make -j8</code> what it tries to do is start 4 threads on localhost and 4 on remote. <em>Not good</em>. This you can see from the image below.</p>
  
  <p><a href="https://i.stack.imgur.com/qcUFx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/qcUFx.png" alt="enter image description here"></a></p>
  
  <ol start="2">
  <li><em>Second Problem:</em> What you would also notice is that the pre-processing is being done on the localmachine. For this there is a tool that comes with distcc. It is called the "distcc-pump" or the pump mode and can be used like this.</li>
  </ol>
  
  <pre><code>time pump make CC="distcc gcc" CXX="distcc g++" -j8
  </code></pre>
  
  <p>Unfortunately, pump mode or not, the pre-processing happens to be happening all on the <code>localhost</code>, as you can see from the above image. <em>Sad.</em></p>
  
  <p><strong>Note:</strong> At no point does the <code>distcc</code> program, <em>with the configurations I listed here</em>, throw any errors nor warnings, neither on the server nor on the clients.</p>
  
  <p><strong>Versions:</strong></p>
  
  <ul>
  <li>gcc 4.4.5</li>
  <li>distcc 3.2rc1.2</li>
  </ul>
  
  <p><em>(Before someone suggests - "upgrade software!", newer versions are most likely not possible for me. Anyways, this version of distcc offers the features that I need. Also, I can upgrade the server virtual-machine but then there would be compiler version mistmatch between clients and the server. The clients I cannot upgrade.)</em></p>
  
  <p>Any suggestions, feedback on how to improve this setup/(fix the problems) are most welcome.</p>