title: Spark with yarn cluster in docker
body: <p>I worked with my spark project as standalone mode (in eclipse IDE). Here is some code that I used (working well).</p>
  
  <pre><code>val conf = new SparkConf()
  .setAppName("My Application")
  .setMaster("local[*]") 
  .set("spark.executor.memory", "4g")
  .set("spark.driver.memory", "4g")
  
  SparkSession.builder
  .config(conf)
  .appName("spark app")
  .config("spark.sql.warehouse.dir", "file:///.")
  .getOrCreate()
  }
  </code></pre>
  
  <p>Until now, I wrapped this application as a jar and used in another java project. </p>
  
  <p>Now I'm going to change it to yarn clustering mode for multi user. So, I installed hadoop clustering in docker with this image(<a href="https://github.com/kiwenlau/hadoop-cluster-docker" rel="nofollow noreferrer">link</a>). </p>
  
  <ol>
  <li>I think It's already set yarn configuration. Do i need to do something more to do spark yarn cluster?</li>
  <li>How should I set sparkConf? How pass hadoop ip, my application jar and other infomation needed?</li>
  </ol>