title: How to Pass Variables into Azure Databricks Cluster Init Script
body: <p>I'm trying to use workspace environment variables to pass access tokens into my custom cluster init scripts. </p>
  
  <p>It appears that there are only a few supported environment variables that we can access in our custom cluster init scripts as described at <a href="https://docs.databricks.com/clusters/init-scripts.html#environment-variables" rel="nofollow noreferrer">https://docs.databricks.com/clusters/init-scripts.html#environment-variables</a></p>
  
  <p>I've attempted to write to the base cluster configuration using</p>
  
  <pre class="lang-cs prettyprint-override"><code>Microsoft.Azure.Databricks.Client.SparkEnvironmentVariables.Add("WORKSPACE_ID", workspaceId)
  </code></pre>
  
  <p>My init scripts are still failing to uptake this variable in the following line:</p>
  
  <pre class="lang-sh prettyprint-override"><code>[[ -z "${WORKSPACE_ID}" ]] &amp;&amp; LOG_ANALYTICS_WORKSPACE_ID='default' || LOG_ANALYTICS_WORKSPACE_ID="${WORKSPACE_ID}"
  </code></pre>
  
  <p>With the above lines of code, my init script causes the cluster to fail with the following error:</p>
  
  <pre><code>Spark Error: Spark encountered an error on startup. This issue can be caused by
  invalid Spark configurations or malfunctioning init scripts. Please refer to the Spark
  driver logs to troubleshoot this issue, and contact Databricks if the problem persists.
  Internal error message: Spark error: Driver down
  </code></pre>
  
  <p>The logs don't say that any part of my bash script is failing, so I'm assuming that it's just failing to pick up the variable from the environment variables. </p>
  
  <p>Has anyone else dealt with a problem with this? I realize that I could write this information to dbfs, and then read it into the init script, but I'd like to avoid doing that since I'll be passing in access tokens. What other approaches can I try?</p>
  
  <p>Thanks for any help!</p>