title: Connection refused when starting Solr with external Zookeeper
body: <p>I have setup 3 servers with Amazon EC2, and have each server with the following Zookeeper-config.</p>
  
  <pre><code>tickTime=2000
  initLimit=10
  syncLimit=5
  clientPort=2181
  server.1=server1address:2888:3888
  server.2=server3address:2888:3888
  server.3=server3address:2888:3888
  </code></pre>
  
  <p>I start zookeeper on each server, and after I start Solr on the servers, I get errors like this in Solr:</p>
  
  <pre><code>3766 [main] INFO  org.apache.solr.common.cloud.ConnectionManager  – Waiting for client to connect to ZooKeeper
  3790 [main-SendThread(*serverAddress*:2181)] WARN  org.apache.zookeeper.ClientCnxn  – Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
  java.net.ConnectException: Connection refused
      at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
      at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
      at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
      at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
  </code></pre>
  
  <p>This was apparently coming because Zookeeper wasn't running properly. What I then figured out was that zookeeper was producing this error:</p>
  
  <pre><code>2013-06-09 08:00:57,953 [myid:1] - INFO  [ec2amazonaddress.com/ipaddress@amazon:QuorumCnxManager$Listener@493] - Received connection request /ipaddress:60855
  2013-06-09 08:00:57,963 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@368] - Cannot open
   channel to 3 at election address ec2amazonaddress/ipaddress@amazon:
  3888
  java.net.ConnectException: Connection refused
          at java.net.PlainSocketImpl.socketConnect(Native Method)
          at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
          at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
          at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
          at java.net.Socket.connect(Socket.java:579)
          at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:35
  4)
  </code></pre>
  
  <p>So the problem is with ZooKeeper. What I did was to start another server before the server I previously started first, and then it worked. However, after some restarts that didn't work anymore. In other words, it seems like the order of when you start the ZK server matters. I was able to see that some servers who were fired up first went into follower mode instead of leader mode right away, and maybe that's the reason. I have deleted and reinstalled my whole setup, but the problem was still there.</p>
  
  <p>I have checked the ports and have killed all processes using ports 2181 and 2888/3888 before launching Zookeeper. What bothers me is that this has worked with the same setup earlier.</p>
  
  <p>Hope some of you guys have some experience with this problem. Any suggestion that could be related to not being able to connect to ZK-servers is also welcomed</p>