title: Pass parameters/arguments to HDInsight/Spark Activity in Azure Data Factory
body: <p>I have an on-demand HDInsight cluster that is launched from a Spark Activity within Azure Data Factory and runs PySpark 3.1. To test out my code, I normally launch Jupyter Notebook from the created HDInsight Cluster page.</p>
  <p>Now, I would like to pass some parameters to that Spark activity and retrieve these parameters from within Jupyter notebook code. I've tried doing so in two ways, but none of them worked for me:</p>
  <p><strong>Method A.</strong> as Arguments and then tried to retrieve them using <code>sys.argv[]</code>.</p>
  <p><strong>Method B.</strong> as Spark configuration and then tried to retrieve them using <code>sc.getConf().getAll()</code>.</p>
  <p>I suspect that either:</p>
  <ul>
  <li>I am not specifying parameters correctly</li>
  <li>or using a wrong way to retrieve them in Jupyter Notebook code</li>
  <li>or parameters are only valid for the Python <code>*.py</code> scripts specified in the &quot;File path&quot; field, but not for the Jupyter notebooks.</li>
  </ul>
  <p>Any pointers on how to pass parameters into HDInsight Spark activity within Azure Data Factory would be much appreciated.</p>
  <p><a href="https://i.stack.imgur.com/NWhp7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NWhp7.png" alt="enter image description here" /></a></p>