title: Docker Node is Down after service restart
body: <p>It seems my server ran out of space and I was having some problems with some of the deployed docker stacks. Took me a while to figure it out, but eventually I did and removed a couple of containers and images to free some space.</p>
  
  <p>I was able to run <code>service docker restart</code> and it worked. However, there are some problems:</p>
  
  <ul>
  <li><code>docker info</code> says the swarm is "Pending"</li>
  <li><code>docker node ls</code> shows the only node I have (Leader), it is <code>available</code> but it is <code>down</code></li>
  <li><code>journalctl -f -u docker</code> says `level=error msg="error removing task " error="incompatible value module=node/agent/worker node.id="</li>
  </ul>
  
  <p>When running <code>docker service ls</code>, all services have <code>0/1</code> replicas. </p>
  
  <p>This is the status when running <code>docker node ls</code></p>
  
  <pre><code>"Status": {
      "State": "down",
      "Message": "heartbeat failure for node in \"unknown\" state",
      "Addr": "&lt;ip and port&gt;"
  },
  "ManagerStatus": {
      "Leader": true,
      "Reachability": "reachable",
      "Addr": "&lt;ip and port&gt;"
  }
  </code></pre>
  
  <p><strong>How can I get my services running again?</strong></p>