title: Akka Singleton Cluster: resolveOnce of a worker by master fails after restart
body: <p>I am using Akka Cluster 2.4.3 and trying to setup a simple cluster in my machine to understand its working better. I have a singleton cluster with remoting enabled with primary and standby master and one worker node. Each of these 3 run in separate JVMs</p>
  
  <p>Things work fine when all the nodes are started the first time. If I kill and restart the worker, I see following issues happening</p>
  
  <p><strong>Restart Worker</strong></p>
  
  <p>When the worker comes back after restart, the master on receiving <code>MemberUp</code> event tries to resolve for the actorRef from the member address the following way</p>
  
  <p><code>context.actorSelection(member.address.toString).resolveOne(15 seconds)</code></p>
  
  <p>This fails with an exception saying <code>ActorNotFound</code>. This works with no problem when all the nodes are coming up for the first time in the cluster.</p>
  
  <p><strong>Restart worker again</strong></p>
  
  <p>This time, the worker comes up with the following message</p>
  
  <p><code>
  [WARN] [04/15/2016 18:24:24.991] [clustersystem-akka.remote.default-remote-dispatcher-5] [akka.remote.Remoting] Tried to associate with unreachable remote address [akka.tcp://clustersystem@host1:2551]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: [The remote system has quarantined this system. No further associations to the remote system are possible until this system is restarted.]
  </code></p>
  
  <p><strong>Restart worker again</strong></p>
  
  <p>This time the <code>resolveOne</code> on a <code>MemberUp</code> event works.</p>
  
  <p>I am having a bit of difficulty in understanding what is happening here, I have looked into the docs but I did not find anything that will help me in there.</p>
  
  <p><strong>application.conf</strong></p>
  
  <pre><code>akka {
    actor {
      provider = "akka.cluster.ClusterActorRefProvider"
    }
  
    remote {
      enabled-transports = ["akka.remote.netty.tcp"]
    }
  
    log-dead-letters = off
    jvm-exit-on-fatal-error = on
  
    loglevel = "DEBUG"
  
    remote {
      log-remote-lifecycle-events = off
      netty.tcp {
        hostname = "host1"
        port = 0
      }
    }
  
    cluster {
      seed-nodes = [
        "akka.tcp://clustersystem@host1:2551",
        "akka.tcp://clustersystem@host1:2552"]
  
      auto-down-unreachable-after = 10s
    }
  
  
    extensions = ["akka.cluster.metrics.ClusterMetricsExtension"]
  }
  </code></pre>
  
  <p>I start master nodes at ports 2551 and 2552 (provide the ports as command line args) and I start the worker on port 3551</p>