title: How to set up dynamic allocation on Cloudera 5 in YARN?
body: <p>I'm trying to run a Spark dynamic resource allocation on Cloudera 5 using Yarn.</p>
  
  <p>In <a href="https://spark.apache.org/docs/latest/job-scheduling.html#configuration-and-setup" rel="nofollow">Spark documentation</a> the properties required to set on <code>yarn-site.xml</code> are missing from the Cloudera configuration interface. Even though I set manually those properties in <code>yarn-site.xml</code> and I add <code>yarn-shuffle.jar</code> to the classpath, the shuffling service is not working.</p>
  
  <p>I got the following exception</p>
  
  <blockquote>
    <p>org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The
    auxService:spark_shuffle does not exist</p>
  </blockquote>