title: Is it must to pass SparkConf parameter from ADF pipeline to reflect in Spark job
body: <p>We are using custom activity in MS Azure ADF and configuring Spark job.
  We have set all <code>SparkConf</code> parameters in code like below:</p>
  
  <pre><code>.set("spark.app.name","SparkAppForStudent")
  .set("spark.streaming.driver.writeAheadLog.allowBatching", "true")
  .set("spark.streaming.driver.writeAheadLog.batchingTimeout", "60000")
  .set("spark.streaming.receiver.writeAheadLog.enable", "true")
  </code></pre>
  
  <p>These parameters are not reflecting in Spark job.
  Do we need to pass <code>SparkConf</code> parameter explicitly from ADF pipeline like below
  to reflect in Spark job?</p>
  
  <pre><code> "sparkConfig": {
       "spark.app.name": "SparkAppForStudent"
   }
  </code></pre>
  
  <p>Thanks</p>