title: Does Spark allow to use Amazon Assumed Role and STS temporary credentials for Glue cross account access on EMR
body: <p>We are trying to connect to the cross-account AWS Glue catalog with the EMR spark job.
  I did a study that AWS supports cross-account access for the Glue catalog in two ways.</p>
  <ol>
  <li><strong>IAM</strong> role-based.  (This is not working for me)</li>
  <li><strong>Resource-based</strong> policy. (This worked for me)</li>
  </ol>
  <blockquote>
  <p>So the problem scenario is, Account <strong>A</strong> creates <strong>EMR</strong> with its
  role <strong>role_Account_A</strong>. And role <strong>role_Account_A</strong> wants to access
  the <strong>glue catalog of Account B</strong>.</p>
  <ul>
  <li>Account A creates EMR cluster with role <strong>role_Account_A</strong></li>
  <li>Account B has role <strong>role_Account_B</strong> which has access to glue and s3     with <strong>role_Account_A</strong> in trusted entities.</li>
  <li><strong>role_Account_A</strong> has sts:AssumeRole policy for resource     <strong>role_Account_B</strong></li>
  <li>using sdk we are able to assume role <strong>role_Account_B</strong> from     <strong>role_Account_A</strong> and getting temporary credentials.</li>
  <li>EMR has configurations[{&quot;classification&quot;:&quot;spark-hive-site&quot;,&quot;properties&quot;:{&quot;hive.metastore.glue.catalogid&quot;:&quot;Account_B&quot;,  &quot;hive.metastore.client.factory.class&quot;: &quot;com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory&quot;}}]</li>
  </ul>
  </blockquote>
  <pre><code>    SparkSession sparkSession=SparkSession.builder().appName(&quot;testing glue&quot;)
                  .enableHiveSupport()
                  .getOrCreate();
  sparkSession.sparkContext().hadoopConfiguration().set(&quot;fs.s3a.aws.credentials.provider&quot;, &quot;org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider&quot;);
  sparkSession.sparkContext().hadoopConfiguration().set(&quot;fs.s3a.access.key&quot;, assumedcreds.getAccessKeyId());
  sparkSession.sparkContext().hadoopConfiguration().set(&quot;fs.s3a.secret.key&quot;, assumedcreds.getSecretAccessKey());
  sparkSession.sparkContext().hadoopConfiguration().set(&quot;fs.s3a.session.token&quot;, assumedcreds.getSessionToken());
  sparkSession.sparkContext().conf().set(&quot;fs.s3a.access.key&quot;,  assumedcreds.getAccessKeyId());
  sparkSession.sparkContext().conf().set(&quot;fs.s3a.secret.key&quot;,  assumedcreds.getSecretAccessKey());
  sparkSession.sparkContext().conf().set(&quot;fs.s3a.session.token&quot;, assumedcreds.getSessionToken());
  sparkSession.sql(&quot;show databases&quot;).show(10, false);
  </code></pre>
  <p><strong>The error that we are getting is</strong></p>
  <pre><code>    Caused by: MetaException(message:User: arn:aws:sts::Account_A:assumed-role/role_Account_A/i-xxxxxxxxxxxx is not authorized to perform: glue:GetDatabase on resource: arn:aws:glue:XX-XXXX-X:Account_B:catalog 
  because no resource-based policy allows the glue:GetDatabase action (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: X93Xbc64-0153-XXXX-XXX-XXXXXXX))
  </code></pre>
  <blockquote>
  <p><strong>Questions:-</strong></p>
  <ul>
  <li>Does spark supports glue-based authentication properties for example    aws.glue.access.key?</li>
  <li>As per error spark is not using assumed role    role_Account_B. It uses role_Account_A with which EMR was created.    Can we make it use
  assumed role role_Account_B?</li>
  </ul>
  </blockquote>
  <p>I will update the question details if I am missing something.</p>