title: Why is HttpUriRequest.getURI() returning null when the uri variable is not null?
body: <h1>SUMMARY</h1>
  <p>I am encountering a unique scenario where a call to an instance of httpcomponents <code>HttpUriRequest</code> method <code>getURI()</code> is returning <code>null</code>, when the variable in the instance is not <code>null</code>.  It is impossible in this scenario for <code>HttpUriRequest.getURI()</code> to return a <code>null</code>, yet that is what is happening.</p>
  <h1>CODE WALKTHROUGH</h1>
  <p>This is the code walkthrough.  It demonstrates how the <code>HttpUriRequest.getURI()</code> method cannot possibly return a <code>null</code>, yet it is returning <code>null</code>.</p>
  <p>The relevant libraries used are</p>
  <ul>
  <li>com.sun.jersey.contribs:jersey-apache-client4:jar:1.19.4</li>
  <li>org.apache.httpcomponents:httpclient:jar:4.5.12</li>
  </ul>
  <p>Our in-house application calls third-party code that uses <code>jersey-apache-client4</code> to make RESTful calls to an external system via <code>apache httpcomponents httpclient</code>.</p>
  <p>For the purposes of this question, requests begin at the call to <code>ApacheHttpClient4Handler.handle(ClientRequest)</code>, source below.</p>
  <ul>
  <li><strong>Numbered</strong> in-line comments will guide the walk-through.</li>
  </ul>
  <p>Relevant source for class <code>ApacheHttpClient4Handler</code> v1.19.4</p>
  <pre class="lang-java prettyprint-override"><code>    package com.sun.jersey.client.apache4;
      
      // ... irrelevant code removed
  
      public final class ApacheHttpClient4Handler extends TerminatingClientHandler {
  
          private final HttpClient client;
          private final boolean preemptiveBasicAuth;
  
          // ... irrelevant code removed
  
          public ClientResponse handle(final ClientRequest cr) throws ClientHandlerException {
  
              // {1} this is where the HttpUriRequest instance is created.  
              // See source walkthrough of getUriHttpRequest(ClientRequest) below.
              final HttpUriRequest request = getUriHttpRequest(cr);
  
              // {8} - calling request.getURI() here returns not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
              writeOutBoundHeaders(cr.getHeaders(), request);
  
              // {11} - calling request.getURI() here returns not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
              try {
                  HttpResponse response;
  
                  if(preemptiveBasicAuth) {
  
                      // this code block is never entered as preemptiveBasicAuth is always false for the scenario
                      // ... irrelevant code removed
  
                  } else {
  
                      // {12} - calling request.getURI() here returns not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
                      // the following line throws the unexpected NullPointerException originating from the call to getHost(request).
                      // See source walkthrough of getHost(HttpUriRequest) below.
                      response = getHttpClient().execute(getHost(request), request);
                  }
  
                  // from here to the catch is never reached due to the NPE two lines above.
                  ClientResponse r = new ClientResponse(response.getStatusLine().getStatusCode(),
                          getInBoundHeaders(response),
                          new HttpClientResponseInputStream(response),
                          getMessageBodyWorkers());
                          
                  if (!r.hasEntity()) {
  
                      r.bufferEntity();
                      r.close();
                  }
  
                  return r;
  
              } catch (Exception e) {
  
                  // {15} - NPE is caught here and rethrown.
                  throw new ClientHandlerException(e);
              }
  
          }
  
          // ... irrelevant code removed
  
          // this is the method where the unexpected NPE is thrown.
          private HttpHost getHost(final HttpUriRequest request) {
  
              // {13} - calling request.getURI() here returns null.  
              // It is not possible for this to be happening.
  
              // {14} - since getURI() returns null, a NPE will be thrown because .getHost() can't be called on a null return.
              // WHY IS getURI() returning null ???!!!???
              return new HttpHost(request.getURI().getHost(), request.getURI().getPort(), request.getURI().getScheme());
          }
  
          // {2} - this is called by handle(ClientRequest)
          private HttpUriRequest getUriHttpRequest(final ClientRequest cr) {
  
              final String strMethod = cr.getMethod();
              final URI uri = cr.getURI();
  
              // {3} - uri is not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
              final Boolean bufferingEnabled = isBufferingEnabled(cr);
              final HttpEntity entity = getHttpEntity(cr, bufferingEnabled);
              final HttpUriRequest request;
  
              // {4} - uri is not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
              // even odds for a HttpGet, HttpPost, HttpPut, HttpDelete; HttpHead and HttpOptions do not get referenced by this scenario.
              // it does not matter which class the request is, the uri will end up being null.
              if (strMethod.equals(&quot;GET&quot;)) {
                  request = new HttpGet(uri);
              } else if (strMethod.equals(&quot;POST&quot;)) {
                  request = new HttpPost(uri);
              } else if (strMethod.equals(&quot;PUT&quot;)) {
                  request = new HttpPut(uri);
              } else if (strMethod.equals(&quot;DELETE&quot;)) {
                  request = new HttpDelete(uri);
              } else if (strMethod.equals(&quot;HEAD&quot;)) {
                  request = new HttpHead(uri);
              } else if (strMethod.equals(&quot;OPTIONS&quot;)) {
                  request = new HttpOptions(uri);
              } else {
  
                  // this block of code is never hit by the requests for this scenario.
                  request = new HttpEntityEnclosingRequestBase() {
                      @Override
                      public String getMethod() {
                          return strMethod;
                      }
  
                      @Override
                      public URI getURI() {
                          return uri;
                      }
                  };
              }
  
              // {5} - checking uri shows it is not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
              // calling request.getURI() here returns not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
              // {6} - sometimes this is called, sometimes it is not; regardless, the error scenario still happens
              if(entity != null &amp;&amp; request instanceof HttpEntityEnclosingRequestBase) {
                  ((HttpEntityEnclosingRequestBase) request).setEntity(entity);
              } else if (entity != null) {
                  throw new ClientHandlerException(&quot;Adding entity to http method &quot; + cr.getMethod() + &quot; is not supported.&quot;);
              }
  
              // ... irrelevant code removed
  
              // {7} - calling request.getURI() here returns not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
  
              return request;
          }
  
          // ... irrelevant code removed
  
          private void writeOutBoundHeaders(final MultivaluedMap&lt;String, Object&gt; headers, final HttpUriRequest request) {
  
              // {9} - none of this code is relevant; it does not change the value of the uri
              for (Map.Entry&lt;String, List&lt;Object&gt;&gt; e : headers.entrySet()) {
                  List&lt;Object&gt; vs = e.getValue();
                  if (vs.size() == 1) {
                      request.addHeader(e.getKey(), ClientRequest.getHeaderValue(vs.get(0)));
                  } else {
                      StringBuilder b = new StringBuilder();
                      for (Object v : e.getValue()) {
                          if (b.length() &gt; 0) {
                              b.append(',');
                          }
                          b.append(ClientRequest.getHeaderValue(v));
                      }
                      request.addHeader(e.getKey(), b.toString());
                  }
              }
  
              // {10} - calling request.getURI() here returns not null, and logging produces the expected &quot;https://{serverpath}&quot; on toString()
          }
  
          // ... irrelevant code removed
      }
  </code></pre>
  <p>Relevant source for interface <code>HttpUriRequest</code> v4.5.12</p>
  <pre class="lang-java prettyprint-override"><code>    package org.apache.http.client.methods;
  
      import java.net.URI;
  
      import org.apache.http.HttpRequest;
  
      public interface HttpUriRequest extends HttpRequest {
  
          // ... irrelevant code removed
  
          URI getURI();
  
          // ... irrelevant code removed
      }
  </code></pre>
  <p>Relevant source for abstract class <code>HttpRequestBase</code> v4.5.12 (<code>HttpGet</code>, <code>HttpPost</code>, etc. extend this)</p>
  <pre class="lang-java prettyprint-override"><code>    package org.apache.http.client.methods;
  
      import java.net.URI;
  
          // ... irrelevant code removed
  
      public abstract class HttpRequestBase extends AbstractExecutionAwareRequest
          implements HttpUriRequest, Configurable {
  
          // ... irrelevant code removed
  
          private URI uri;
  
          // ... irrelevant code removed
  
          @Override
          public URI getURI() {
              return this.uri;
          }
  
          // ... irrelevant code removed
  
          public void setURI(final URI uri) {
              this.uri = uri;
          }
      }
  </code></pre>
  <p>Relevant source for class <code>HttpGet</code> v4.5.12 (<code>HttpPost</code>, etc. are implemented similarly)</p>
  <pre class="lang-java prettyprint-override"><code>    package org.apache.http.client.methods;
  
      import java.net.URI;
  
      public class HttpGet extends HttpRequestBase {
  
          public final static String METHOD_NAME = &quot;GET&quot;;
  
          public HttpGet() {
              super();
          }
  
          public HttpGet(final URI uri) {
              super();
              setURI(uri);
          }
  
          /**
           * @throws IllegalArgumentException if the uri is invalid.
           */
          public HttpGet(final String uri) {
              super();
              setURI(URI.create(uri));
          }
  
          @Override
          public String getMethod() {
              return METHOD_NAME;
          }
      }
  </code></pre>
  <p>Given the source walkthrough and the supporting comments proving the viability of the state of the Object instance, <strong>how is it possible the call to <code>request.getURI()</code> is returning <code>null</code>?</strong></p>
  <h1>ENVIRONMENT</h1>
  <ul>
  <li>This code runs under IBM Liberty on Linux servers (JVM 8.0-5.40).</li>
  <li>This code also runs under Wildfly on Windows servers (JVM 1.7).</li>
  <li>The code is identical on both system types.</li>
  <li>The code never has the above stated problem on the Windows servers.</li>
  <li>The code does not have the above stated problem on Liberty servers where the Liberty management administration console is installed.</li>
  </ul>
  <h1>OBSERVATIONS</h1>
  <ul>
  <li>The first 2-3 pass-throughs of <code>ApacheHttpClient4Hander.handle(ClientRequest)</code> succeed without <code>request.getURI()</code> returning <code>null</code>; it is only on the 4th and subsequent requests that it returns <code>null</code>.</li>
  </ul>
  <h1>INVESTIGATION</h1>
  <p>Here are some investigation steps I have taken so far:</p>
  <p>1 - I created this case on SO for group-mind insight.</p>
  <p>2 - I opened a case with IBM.</p>
  <p>3 - Customize <code>ApacheHttpClient4Handler</code></p>
  <ul>
  <li>I add debug logging to all methods via SL4J.</li>
  </ul>
  <p><strong>RESULT</strong></p>
  <ul>
  <li>the log output proves the call to <code>HttpUriRequest.getURI()</code> only starts returning <code>null</code> after entering <code>ApacheHttpClient4Handler.getHost(final HttpUriRequest request)</code>.</li>
  <li>any call immediately before <code>getHost(HttpUriRequest)</code> shows <code>getURI()</code> to return not-null.</li>
  <li>after adding the debug logging, instead of after the 2-3 and subsequent requests (per OBSERVATIONS), <code>HttpUriRequest.getURI()</code> now returns null after the 1-2 (and subsequent) requests.  I suspect this is due to debug logging proactively calling <code>HttpUriRequest.getURI()</code>, but I cannot confirm this and do not know why the scenario is now shorter.</li>
  </ul>
  <p>4 - Customize <code>ApacheHttpClient4Handler</code> further</p>
  <ul>
  <li>change all references of <code>HttpUriRequest</code> to <code>HttpRequestBase</code></li>
  </ul>
  <p><strong>RESULT</strong></p>
  <ul>
  <li>doing this did not solve the problem; <code>getURI()</code> still returns <code>null</code>.</li>
  </ul>
  <p>5 - Customize <code>HttpRequestBase</code></p>
  <ul>
  <li>add debug logging to <code>getURI()</code> / <code>setURI(...)</code> methods</li>
  </ul>
  <p><strong>RESULT</strong></p>
  <ul>
  <li><strong>Application started working as designed, except with lots of debug logging!!!</strong></li>
  <li>see section <strong>MAJOR UPDATE</strong> below.</li>
  </ul>
  <p>6 - Customize <code>HttpGet</code>, <code>HttpPut</code>, etc.</p>
  <ul>
  <li>add debug logging to constructors.</li>
  </ul>
  <p><strong>RESULT</strong></p>
  <ul>
  <li>see section <strong>MAJOR UPDATE</strong> below.</li>
  </ul>
  <p>7 - Update IBM JVM from 8.0-5.40 to 8.0-6.11</p>
  <ul>
  <li>List of NPE-related bugfixes from the IBM release notes</li>
  <li>IJ17563 | NPE WHEN CALLING JAVA.LANG.STRING.INDEXOF API WITH A LARGE OFFSET</li>
  <li>IJ19669 | NPE OCCURS WHEN READING AN EMPTY PKCS12 KEYSTORE USING JAVA 8</li>
  <li>IJ20528 | NPE OR SEGFAULT FOR METHOD CALL TO OBJECT OF A PRIMITIVE WRAPPER TYPE</li>
  <li>IJ23079 | UNEXPECTED NPE CAUSED BY AN INCORRECT JIT COMPILER OPTIMIZATION</li>
  <li>TODO: results to be posted in update</li>
  </ul>
  <h1>CONCLUSION</h1>
  <p><strong>What is going on here?</strong></p>
  <p><strong>Is the IBM implementation of the JVM used by Liberty mistreating this code, or is there something else?</strong></p>
  <p>Based on the code, and the walkthrough, there is no way <code>HttpUriRequest.getURI()</code> should be returning <code>null</code>.</p>
  <h1>MAJOR UPDATE</h1>
  <p>Per Step 5 in INVESTIGATION, I customized <code>httpclient</code> with debug logging using source from the global distribution.  After adding the JAR to my project, it started working.  I rolled back my changes, rebuilt the JAR from unmodified source, added it to my project, and the application started working.</p>
  <p>I redeployed again using the globally-distributed <code>httpclient-4.5.12.jar</code> <strong>My application failed.</strong></p>
  <p>Again, I did not make any changes to the code; I took the <code>httpclient-4.5.12-sources.jar</code>, compiled it in Eclipse and redeployed with my newly-named JAR, <code>httpclient-4.5.12.joshdm.jar</code>.  And it works.</p>
  <p>I decompiled both JARs using JAD to examine the code.  There were minimal differences, only in labels, which means the compilers were different.  I compared MANIFEST.MF for both JARs; the Build-Jdk entries were different.  I compiled using Jdk 1.8.0_191.  The global one was compiled using Jdk 1.8.0_1<strong>8</strong>1.</p>
  <p>I have reported this strange discrepancy to IBM, provided both compiled JARs and the source JAR, and am waiting to hear back as to why mine works with their JVM and why the global one does not.  I am now 99% certain this is an IBM JVM or server configuration issue, and has nothing to do with the deployed application.</p>
  <h1>UPDATE #2</h1>
  <p>Adding the following configuration and rolling back to the global JAR works (also works with the recompiled JAR).</p>
  <p><code>-Xshareclasses:none</code> - disables shared class caching and AOT compilation.</p>
  <p><code>-Xnojit</code> - disables JIT compilation</p>
  <p>Disabling shared class caching, disabling AOT compilation and disabling JIT compilation on Liberty seems to have resolved the issue.  I will write-up an actual answer once I hear back from IBM why the above settings were needed.</p>