title: Spark worker won't bind to master
body: <p>Launching my spark worker, I got an error which may be related to the possibility from the slave to contact the master machine. But I am unsure.</p>
  
  <pre><code>6/02/12 23:47:13 INFO Utils: Successfully started service 'sparkWorker' on port 38019.
  16/02/12 23:47:13 INFO Worker: Starting Spark worker 192.168.0.38:38019 with 8 cores, 26.5 GB RAM
  16/02/12 23:47:13 INFO Worker: Running Spark version 1.6.0
  16/02/12 23:47:13 INFO Worker: Spark home: /home/romain/spark-1.6.0-bin-hadoop2.6
  16/02/12 23:47:13 INFO Utils: Successfully started service 'WorkerUI' on port 8081.
  16/02/12 23:47:13 INFO WorkerWebUI: Started WorkerWebUI at http://192.168.0.38:8081
  16/02/12 23:47:13 INFO Worker: Connecting to master 192.168.0.39:7078...
  16/02/12 23:47:13 WARN Worker: Failed to connect to master 192.168.0.39:7078
  java.io.IOException: Failed to connect to /192.168.0.39:7078
      at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
      at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
      at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:200)
      at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
      at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
  Caused by: java.net.ConnectException: Connection refused: /192.168.0.39:7078
      at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
      at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
      at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
      at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
      at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      ... 1 more
  </code></pre>
  
  <p>While on the master I see it is up and running : </p>
  
  <pre><code>16/02/12 23:30:30 WARN Utils: Your hostname, pl resolves to a loopback address: 127.0.1.1; using 192.168.0.39 instead (on interface eth0)
  16/02/12 23:30:30 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
  16/02/12 23:30:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  16/02/12 23:30:31 INFO SecurityManager: Changing view acls to: romain
  16/02/12 23:30:31 INFO SecurityManager: Changing modify acls to: romain
  16/02/12 23:30:31 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(romain); users with modify permissions: Set(romain)
  16/02/12 23:30:31 WARN Utils: Service 'sparkMaster' could not bind on port 7077. Attempting port 7078.
  16/02/12 23:30:31 INFO Utils: Successfully started service 'sparkMaster' on port 7078.
  16/02/12 23:30:31 INFO Master: Starting Spark master at spark://pl:7078
  16/02/12 23:30:31 INFO Master: Running Spark version 1.6.0
  16/02/12 23:30:32 INFO Utils: Successfully started service 'MasterUI' on port 3094.
  16/02/12 23:30:32 INFO MasterWebUI: Started MasterWebUI at http://192.168.0.39:3094
  16/02/12 23:30:32 WARN Utils: Service could not bind on port 6066. Attempting port 6067.
  16/02/12 23:30:32 INFO Utils: Successfully started service on port 6067.
  16/02/12 23:30:32 INFO StandaloneRestServer: Started REST server for submitting applications on port 6067
  16/02/12 23:30:32 INFO Master: I have been elected leader! New state: ALIVE
  </code></pre>
  
  <p>Going through blogs and pages it seems it is possible that we would need a secure network (I did install password-less ssh key - but for "romain" user : under which user is spark launch ? the command-line user I guess). </p>
  
  <p>Should I check something on the network ? 
  From this page : 
  <a href="https://stackoverflow.com/questions/33178289/spark-worker-can-not-connect-to-master">Spark worker can not connect to Master</a>
  I tried : </p>
  
  <pre><code>telnet 192.168.0.39
  Trying 192.168.0.39...
  telnet: Unable to connect to remote host: Connection refused
  </code></pre>
  
  <p>But ping works : </p>
  
  <pre><code>romain@wk:~/spark-1.6.0-bin-hadoop2.6$ ping 192.168.0.39
  PING 192.168.0.39 (192.168.0.39) 56(84) bytes of data.
  64 bytes from 192.168.0.39: icmp_seq=1 ttl=64 time=0.233 ms
  64 bytes from 192.168.0.39: icmp_seq=2 ttl=64 time=0.185 ms
  ^C
  --- 192.168.0.39 ping statistics ---
  2 packets transmitted, 2 received, 0% packet loss, time 1000ms
  rtt min/avg/max/mdev = 0.185/0.209/0.233/0.024 ms
  </code></pre>
  
  <p>and I do have passwordless ssh connectivity : </p>
  
  <pre><code>$ ssh 192.168.0.39
  Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.19.0-49-generic x86_64)
  $
  </code></pre>
  
  <p>What should be done to make connectivity possible ? </p>