title: Cannot run PySpark (not using interactive shell) on Cloudera VM
body: <p>When I follow <a href="http://www.cloudera.com/documentation/enterprise/5-5-x/topics/spark_develop_run.html" rel="nofollow">this example</a> and try to use the command <code>spark-submit</code> within the cloudera vm envirionrment, I constantly get the following error:</p>
  
  <pre><code>ERROR spark.SparkContext: Error initializing SparkContext.
  org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=WRITE, inode="/user/spark/applicationHistory":spark:supergroup:drwxr-xr-x
  ....
  Traceback (most recent call last):
  File "/home/cloudera/wordcount.py", line 9, in &lt;module&gt;
  sc = SparkContext(conf=conf)
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/context.py", line 115, in __init__
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/context.py", line 172, in _do_init
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/context.py", line 235, in _initialize_context
  File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 1064, in __call__
  File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value
  py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
  : org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=WRITE, inode="/user/spark/applicationHistory":spark:supergroup:drwxr-xr-x
  </code></pre>
  
  <p>I have tried these two commands:</p>
  
  <p>1, <code>$ spark-submit --master yarn --deploy-mode client --executor-memory 1g \ --name wordcount --conf "spark.app.id=wordcount" wordcount.py hdfs://namenode_host:8020/path/to/inputfile.txt</code></p>
  
  <p>2, <code>$ spark-submit --master yarn --deploy-mode client --executor-memory 1g \ --name wordcount --conf "spark.app.id=wordcount" wordcount.py inputfile.txt</code> </p>
  
  <p>Can somebody help?</p>