title: Hadoop connection refused to ResourceManager on localhost:8032
body: <p>I'm new to Hadoop, and have been trying to get it working on my machine since the last several hours. I've mostly followed the installation and setup instructions from these guides - <a href="http://dongfeiwww.com/hadoop/2014/09/05/hadoop-on-mac/" rel="nofollow noreferrer">1</a>, <a href="https://isaacchanghau.github.io/post/install_hadoop_mac/" rel="nofollow noreferrer">2</a>, <a href="https://medium.com/@jeremytarling/apache-spark-and-hadoop-on-a-macbook-air-running-osx-sierra-66bfbdb0b6f7" rel="nofollow noreferrer">3</a>, <a href="https://macmetric.com/how-to-install-hadoop-on-mac/" rel="nofollow noreferrer">4</a>, <a href="https://dtflaneur.wordpress.com/2015/10/02/installing-hadoop-on-mac-osx-el-capitan/" rel="nofollow noreferrer">5</a>, <a href="http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/" rel="nofollow noreferrer">6</a>.</p>
  
  <p>I have also read this SO question on <a href="https://stackoverflow.com/questions/20586920/hadoop-connecting-to-resourcemanager-failed">Hadoop: Connecting to ResourceManager failed</a> several times, but nothing has helped.</p>
  
  <p>Here are my configuration files:</p>
  
  <p><strong>core-site.xml</strong> -</p>
  
  <pre><code>&lt;configuration&gt;
  &lt;property&gt;
     &lt;name&gt;hadoop.tmp.dir&lt;/name&gt;
     &lt;!-- &lt;value&gt;/tmp/hadoop-${user.name}&lt;/value&gt; --&gt;
     &lt;value&gt;/app/hadoop/tmp&lt;/value&gt;
     &lt;description&gt;A base for other temporary directories.&lt;/description&gt;
   &lt;/property&gt; 
  &lt;property&gt;
     &lt;name&gt;fs.default.name&lt;/name&gt;
     &lt;value&gt;hdfs://localhost:9000&lt;/value&gt;
   &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p><strong>hdfs-site.xml</strong> -</p>
  
  <pre><code>&lt;configuration&gt;
  
   &lt;property&gt;
       &lt;name&gt;dfs.replication&lt;/name&gt;
       &lt;value&gt;1&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;dfs.namenode.name.dir&lt;/name&gt;
    &lt;value&gt;${user.home}/hadoop/data/namenode&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
    &lt;name&gt;dfs.datanode.data.dir&lt;/name&gt;
    &lt;value&gt;${user.home}/hadoop/data/datanode&lt;/value&gt;
  &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p><strong>mapred-site.xml</strong> -</p>
  
  <pre><code>&lt;configuration&gt;
      &lt;property&gt;
          &lt;name&gt;mapred.job.tracker&lt;/name&gt;
          &lt;value&gt;localhost:9001&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;mapreduce.framework.name&lt;/name&gt;
          &lt;value&gt;yarn&lt;/value&gt;
      &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p><strong>yarn-site.xml</strong> -</p>
  
  <pre><code>&lt;configuration&gt;
      &lt;!-- Site specific YARN configuration properties --&gt;
      &lt;property&gt;
          &lt;name&gt;yarn.resourcemanager.address&lt;/name&gt;
          &lt;value&gt;localhost:8032&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
          &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;yarn.resourcemanager.scheduler.address&lt;/name&gt;
          &lt;value&gt;localhost:8030&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;yarn.resourcemanager.resource-tracker.address&lt;/name&gt;
          &lt;value&gt;localhost:8031&lt;/value&gt;
      &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>The setup guides also require making changes in <code>hadoop-env.sh</code>, but the required changes are already present in this <code>case</code> for OSX, so I didn't make any further changes -</p>
  
  <pre><code>case ${HADOOP_OS_TYPE} in
    Darwin*)
      export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.realm= "
      export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.kdc= "
      export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.conf= "
    ;;
  esac
  </code></pre>
  
  <p>In my <code>.zshrc</code> file, I've added the <code>hstart</code> and <code>hstop</code> alias like so -</p>
  
  <pre><code># hadoop config
  alias hstart="/usr/local/Cellar/hadoop/3.1.1/sbin/start-dfs.sh;/usr/local/Cellar/hadoop/3.1.1/sbin/start-yarn.sh"
  alias hstop="/usr/local/Cellar/hadoop/3.1.1/sbin/stop-yarn.sh;/usr/local/Cellar/hadoop/3.1.1/sbin/stop-dfs.sh"
  </code></pre>
  
  <p>I think my <code>hstart</code> command is working normally -</p>
  
  <pre><code>➜  ~ hstart
  Starting namenodes on [localhost]
  Starting datanodes
  Starting secondary namenodes [Manish-Macbook-Pro-15.local]
  2019-03-01 02:26:13,502 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  Starting resourcemanager
  Starting nodemanagers
  </code></pre>
  
  <p>Here is the output of <code>jps</code> -</p>
  
  <pre><code>➜  ~ jps
  30548 NameNode
  31172 Jps
  30796 SecondaryNameNode
  30655 DataNode
  </code></pre>
  
  <p>After this, I try to run of the sample jar files (<code>hadoop-mapreduce-examples-3.1.1.jar</code>) with a sample job from one of the guides -</p>
  
  <pre><code>➜  ~ cd /usr/local/Cellar/hadoop/3.1.1/libexec/share/hadoop/mapreduce
  ➜  mapreduce hadoop jar hadoop-mapreduce-examples-3.1.1.jar pi 2 5
  </code></pre>
  
  <p>This is when the <code>ConnectionRefused</code> error shows up -</p>
  
  <pre><code>Number of Maps  = 2
  Samples per Map = 5
  2019-03-01 02:26:39,491 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  Wrote input for Map #0
  Wrote input for Map #1
  Starting Job
  2019-03-01 02:26:42,134 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8032
  2019-03-01 02:26:43,440 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:44,443 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:45,449 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:46,452 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:47,456 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:48,461 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:49,463 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:50,469 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:51,475 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:52,481 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:53,496 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:54,501 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:55,507 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:56,510 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:57,513 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:58,517 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:26:59,520 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:27:00,526 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:27:01,528 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:27:02,530 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  2019-03-01 02:27:02,533 INFO retry.RetryInvocationHandler: java.net.ConnectException: Call From Manish-Macbook-Pro-15.local/192.168.200.79 to localhost:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused, while invoking ApplicationClientProtocolPBClientImpl.getNewApplication over null after 1 failover attempts. Trying to failover after sleeping for 31316ms.
  </code></pre>
  
  <p>After this, it keeps trying to connect repeatedly. I have to open up a new terminal tab to execute <code>hstop</code>.</p>
  
  <p>I have no idea what is causing this error. I've started from scratch with <code>hdfs namenode -format</code> several times, but each time the same error. </p>
  
  <p>A few other observations I have -</p>
  
  <ul>
  <li><p>The guides mention that once hadoop is up and running, I can view the status of things (like ResourceManager) in a browser using the corresponding default ports, but I've tried several ports - <code>localhost:50070</code>, <code>localhost:8088</code> and many more, but nothing works.</p></li>
  <li><p>In the <code>yarn-site.xml</code> file, I've used <code>localhost:8032</code> as the value for <code>&lt;name&gt;yarn.resourcemanager.address&lt;/name&gt;</code>, but in my terminal logs, it shows <code>2019-03-01 02:26:48,461 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032.</code>. I don't know why it's <code>localhost/127.0.0.1:8032</code> instead of <code>localhost:8032</code>.</p></li>
  </ul>
  
  <p>Here are the contents of <code>/etc/hosts</code> -</p>
  
  <pre><code>##
  # Host Database
  #
  # localhost is used to configure the loopback interface
  # when the system is booting.  Do not change this entry.
  ##
  127.0.0.1       localhost
  255.255.255.255 broadcasthost
  ::1             localhost
  </code></pre>
  
  <p>Would appreciate any help with this.</p>