title: unable to connect to jmx port remotely
body: <p>I'm trying to connect to a jmx port remotely but I can't seem to connect to it even though the port is open. Its a java process running in a container on a server thats a Nomad worker. Its running on <code>29406</code>. </p>
  
  <p>Here is what <code>netstat</code> shows:</p>
  
  <pre><code>netstat -tulpn | grep 29406
  (Not all processes could be identified, non-owned process info
   will not be shown, you would have to be root to see it all.)
  tcp        0      0 10.137.2.166:29406      0.0.0.0:*               LISTEN      -               
  udp        0      0 10.137.2.166:29406      0.0.0.0:*                           -   
  </code></pre>
  
  <p>And this is whats in <code>/etc/hosts</code></p>
  
  <pre><code>cat /etc/hosts
  127.0.0.1 localhost
  
  # The following lines are desirable for IPv6 capable hosts
  ::1 ip6-localhost ip6-loopback
  fe00::0 ip6-localnet
  ff00::0 ip6-mcastprefix
  ff02::1 ip6-allnodes
  ff02::2 ip6-allrouters
  ff02::3 ip6-allhosts
  </code></pre>
  
  <p>I've downloaded <code>jmxterm</code> on the server to try and connect to it, and noticed an interesting behavior. When I try using <code>localhost</code> to connect to the port, I get this:</p>
  
  <pre><code>#RuntimeIOException: Runtime IO exception: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: 
      java.net.ConnectException: Connection refused (Connection refused)]
  </code></pre>
  
  <p>When I use its own IP address, it then seems to work:</p>
  
  <pre><code>$&gt;open 10.137.2.166:29406
  #Connection to 10.137.2.166:29406 is opened
  $&gt;
  </code></pre>
  
  <p>Curious to understand why localhost doesn't work when I'm running this on the server itself...</p>
  
  <p>The only way I've gotten <code>jconsole</code> (running on my laptop) to connect to it is by using an ssh tunnel like this:</p>
  
  <pre><code>ssh -Nf -D 7777 10.137.2.166
  jconsole -J-DsocksProxyHost=localhost -J-DsocksProxyPort=7777 service:jmx:rmi:///jndi/rmi://10.137.2.166:29406/jmxrmi -J-DsocksNonProxyHosts=
  </code></pre>
  
  <p>I feel like I should be able to connect to it without creating a tunnel but unsure why I can't. If I run telnet locally from my laptop to the host, the connection does seem to open...</p>
  
  <pre><code>telnet 10.137.2.166 29406
  Trying 10.137.2.166...
  Connected to 10.137.2.166.
  Escape character is '^]'.
  </code></pre>