title: SLURM controller not being able to connect to workers and state is set as UNKNOWN
body: <p>I am trying to setup a small cluster, managed with SLURM. The controller is also a compute node. The config in <code>/etc/slurm/slurm.conf</code> is:</p>
  <pre class="lang-bash prettyprint-override"><code>NodeName=controller,node[01-02] RealMemory=250000 Sockets=1 CoresPerSocket=32 ThreadsPerCore=2 State=UNKNOWN
  PartitionName=compute Nodes=ALL Default=YES MaxTime=INFINITE State=UP
  </code></pre>
  <p>When running <code>sinfo</code> I get:</p>
  <pre class="lang-bash prettyprint-override"><code>PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
  compute*     up   infinite      2   unk* node[01-02]
  compute*     up   infinite      1   idle controller
  </code></pre>
  <p>However, when running <code>slurmd -C</code> on each node I get:</p>
  <pre class="lang-bash prettyprint-override"><code>NodeName=node01 CPUs=64 Boards=1 SocketsPerBoard=1 CoresPerSocket=32 ThreadsPerCore=2 RealMemory=257655
  UpTime=0-00:30:44
  </code></pre>
  <p>The same on the other node. I have allowed the ports <code>6817</code> and <code>6818</code> (the default slurm ports) on all machines (for TCP - which I assume is the protocol). I have also checked that the <code>/etc/slurm/slurm.conf</code> and <code>/etc/slurm/slurmdbd.conf</code> are the same, along with the munge keys (this works).</p>
  <p>Is there anyway to debug the connection to a given machine?</p>
  <p>Thanks in advance for any help.</p>