title: WSO2: API cannot authenticate in independent gateway
body: <p>We are having similar problem as reported in <a href="https://stackoverflow.com/questions/48644486/wso2-authentication-failure-on-gateway-node-running-wso2-am-2-1-0">WSO2 Authentication failure on gateway node running WSO2 AM 2.1.0</a>, but our configuration is slightly different, and we have done some additional test</p>
  
  <p>Environment:
  Ubuntu LTS 16.04.4
  Java "1.8.0_171" (known reported issues, see below)
  WSO2 2.1.0
  We have deactivated the compression in the 3 nodes, editing the catalina-server.xml
  Additional: We have repeated the tests after downgrading Java to 1.8.0_144. Same results.</p>
  
  <p>We have deployed, in the same machine, 3 nodes
  - 1 manager wso2server.sh start
  - 2 gateways wso2server.sh -Dprofile=gateway-manager start
  - Unique H2 databases set, shared for all nodes
  (we are able to rebuild the environment in any moment, from scratch)</p>
  
  <h1>Initial test</h1>
  
  <ul>
  <li>We have configured API Key Validator using thrift. There is only one ThriftServer, started in the manager node</li>
  <li><p>Manager node key API Key Validator (changed real host name by example.com)</p>
  
  <pre><code>    &lt;APIKeyValidator&gt;
      &lt;ServerURL&gt;https://apimanager.example.com:9443/services/&lt;/ServerURL&gt;
      &lt;Username&gt;${admin.username}&lt;/Username&gt;
      &lt;Password&gt;${admin.password}&lt;/Password&gt;
      &lt;KeyValidatorClientType&gt;ThriftClient&lt;/KeyValidatorClientType&gt;
      &lt;ThriftClientConnectionTimeOut&gt;10000&lt;/ThriftClientConnectionTimeOut&gt;
      &lt;EnableThriftServer&gt;true&lt;/EnableThriftServer&gt;
      &lt;ThriftServerHost&gt;apimanager.example.com&lt;/ThriftServerHost&gt;
      &lt;ThriftClientPort&gt;10397&lt;/ThriftClientPort&gt;
      &lt;KeyValidationHandlerClassName&gt;org.wso2.carbon.apimgt.keymgt.handlers.DefaultKeyValidationHandler&lt;/KeyValidationHandlerClassName&gt;
      &lt;/APIKeyValidator&gt;
  </code></pre></li>
  <li><p>Sandbox and gateway nodes key API Key Validator are the same configuration, except the parameter <code>&lt;EnableThriftServer&gt;</code> is <code>false</code></p></li>
  </ul>
  
  <p><em>Test result</em>
  In API Console, we get (we are using the PizzaShack API sample in GET /menu)</p>
  
  <pre><code>{ "fault": { "code": 900900, "message": "Unclassified Authentication Failure", "description": "Error while accessing backend services for API key validation" } }
  </code></pre>
  
  <p><em>Gateway trace</em></p>
  
  <pre><code>TID: [-1234] [] [2018-05-21 13:15:17,351]  WARN {org.wso2.carbon.apimgt.gateway.handlers.security.thrift.ThriftKeyValidatorClient} -  Login failed.. Authenticating again.. {org.wso2.carbon.apimgt.gateway.handlers.security.thrift.ThriftKeyValidatorClient}
  TID: [-1234] [] [2018-05-21 13:15:17,392]  WARN {org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler} -  API authentication failure due to Unclassified Authentication Failure {org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler}
  </code></pre>
  
  <p><em>Manager trace (API console)</em></p>
  
  <pre><code>TID: [-1234] [] [2018-05-21 13:15:17,372]  INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -  'admin@carbon.super [-1234]' logged in at [2018-05-21 13:15:17,372+0200] from IP address  {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil}
  </code></pre>
  
  <h1>Test with debug enabled</h1>
  
  <p>Testing with debug is not possible due a previous bug (see comment of <a href="https://stackoverflow.com/users/805563/bee">Bee</a> below)</p>
  
  <h1>Additional tests</h1>
  
  <ul>
  <li>Following the suggestions of the original post, we have changed the APIKeyValidator to WSClient. After some initial configuration problem, this work fine</li>
  <li>We have tried also calling with CURL, no change.</li>
  </ul>