title: Accessing Google BigQuery from AWS SageMaker
body: <p>When running locally, my Jupyter notebook is able to reference Google BigQuery like so:</p>
  
  <pre><code>%%bigquery some_bq_table
  
  SELECT *
  FROM
    `some_bq_dataset.some_bq_table` 
  </code></pre>
  
  <p>So that later in my notebook I can reference some_bq_table as a pandas dataframe, as exemplified here: <a href="https://cloud.google.com/bigquery/docs/visualize-jupyter" rel="nofollow noreferrer">https://cloud.google.com/bigquery/docs/visualize-jupyter</a></p>
  
  <p>I want to run my notebook on AWS SageMaker to test a few things. To authenticate with BigQuery it seems that the only two ways are using a service account on GCP (or locally) or pointing the the SDK to a credentials JSON using an env var (as explained here: <a href="https://cloud.google.com/docs/authentication/getting-started" rel="nofollow noreferrer">https://cloud.google.com/docs/authentication/getting-started</a>).</p>
  
  <p>For example</p>
  
  <pre><code>export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"
  </code></pre>
  
  <p>Is there an easy way to connect to bigquery from SageMaker? My best idea right now is to download the JSON from somewhere to the SageMaker instnace and then set the env var from the python code.</p>
  
  <p>For example, I would do this:</p>
  
  <pre><code>os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/home/user/Downloads/[FILE_NAME].json"
  </code></pre>
  
  <p>However, this isn't very secure - I don't like the idea of downloading my credentials JSON to a SageMaker instance (this means I would have to upload the credentials to some private s3 bucket and then store them on the SageMaker instance). Not the end of the world but I rather avoid this. </p>
  
  <p>Any ideas?</p>