title: Python hadoop mapreduce job using mrjob subprocess.CalledProcessError
body: <p>I am trying to run basic example from the mrjob's website on my custom data. I have run Hadoop map reduce successfully using streaming, I have also successfully tried the script without Hadoop, but now I am trying to run it on Hadoop via mrjob by following command.</p>
  
  <pre><code>./mapred.py -r hadoop --hadoop-bin /usr/bin/hadoop -o hdfs:///user/cloudera/wc_result_mrjob hdfs:///user/cloudera/books
  </code></pre>
  
  <p>Source code of mapred.py is following:</p>
  
  <pre><code>#! /usr/bin/env python
  
  from mrjob.job import MRJob
  
  
  class MRWordFrequencyCount(MRJob):
  
      def mapper(self, _, line):
          yield "chars", len(line)
          yield "words", len(line.split())
          yield "lines", 1
  
      def reducer(self, key, values):
          yield key, sum(values)
  
  if __name__ == '__main__':
      MRWordFrequencyCount.run()
  </code></pre>
  
  <p>Unfortunately I am getting following error:</p>
  
  <pre><code>no configs found; falling back on auto-configuration
  no configs found; falling back on auto-configuration
  creating tmp directory /tmp/mapred.cloudera.20140824.195414.420162
  writing wrapper script to /tmp/mapred.cloudera.20140824.195414.420162/setup-wrapper.sh
  STDERR: mkdir: `hdfs:///user/cloudera/tmp/mrjob/mapred.cloudera.20140824.195414.420162/files/': No such file or directory
  Traceback (most recent call last):
  File "./mapred.py", line 18, in &lt;module&gt;
  MRWordFrequencyCount.run()
  File "/usr/lib/python2.6/site-packages/mrjob/job.py", line 494, in run
  mr_job.execute()
  File "/usr/lib/python2.6/site-packages/mrjob/job.py", line 512, in execute
  super(MRJob, self).execute()
  File "/usr/lib/python2.6/site-packages/mrjob/launch.py", line 147, in execute
  self.run_job()
  File "/usr/lib/python2.6/site-packages/mrjob/launch.py", line 208, in run_job
  runner.run()
  File "/usr/lib/python2.6/site-packages/mrjob/runner.py", line 458, in run
  self._run()
  File "/usr/lib/python2.6/site-packages/mrjob/hadoop.py", line 238, in _run
  self._upload_local_files_to_hdfs()
  File "/usr/lib/python2.6/site-packages/mrjob/hadoop.py", line 265, in _upload_local_files_to_hdfs
  self._mkdir_on_hdfs(self._upload_mgr.prefix)
  File "/usr/lib/python2.6/site-packages/mrjob/hadoop.py", line 273, in _mkdir_on_hdfs
  self.invoke_hadoop(['fs', '-mkdir', path])
  File "/usr/lib/python2.6/site-packages/mrjob/fs/hadoop.py", line 109, in invoke_hadoop
  raise CalledProcessError(proc.returncode, args)
  subprocess.CalledProcessError: Command '['/usr/bin/hadoop', 'fs', '-mkdir', 'hdfs:///user/cloudera/tmp/mrjob/mapred.cloudera.20140824.195414.420162/files/']' returned non-zero exit status 1
  </code></pre>
  
  <p>It seems to me that mrjob is not able to create some directory in HDFS, but unfortunately I don't know how to fix this problem.</p>
  
  <p>My Hadoop is the cloudera CDH5.1. quickstart.</p>
  
  <p>Thank you in advance for any suggestions and advices.</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>I have tried to run the same code on cloudera CDH4.7. quickstart and it worked well. So my modified question is: Is cloudera CDH5.1. supported by mrjob framework? If so, then how to run it?</p>