Path function ( JobConf arg0 ) { String loc0 = arg0 . get ( JobContext . TASK_OUTPUT_DIR ) ; return loc0 == null ? null : new Path ( loc0 ) ; }
get the path to the task 's temporary output directory for the map-reduce job tasks ' side-effect files note : the following is valid only if the outputcommitter is fileoutputcommitter . if outputcommitter is not a fileoutputcommitter , the task 's temporary output directory is same as #getoutputpath jobconf i.e. $ mapreduce.output.fileoutputformat.outputdir $ some applications need to create/write-to side-files , which differ from the actual job-outputs . in such cases there could be issues with 2 instances of the same tip running simultaneously e.g. speculative tasks trying to open/write-to the same file path on hdfs . hence the application-writer will have to pick unique names per task-attempt e.g. using the attemptid , say attempt_200709221812_0001_m_000000_0 , not just per tip . to get around this the map-reduce framework helps the application-writer out by maintaining a special $ mapreduce.output.fileoutputformat.outputdir / _ temporary / _ $ taskid sub-directory for each task-attempt on hdfs where the output of the task-attempt goes . on successful completion of the task-attempt the files in the $ mapreduce.output.fileoutputformat.outputdir / _ temporary / _ $ taskid only are promoted to $ mapreduce.output.fileoutputformat.outputdir . of course , the framework discards the sub-directory of unsuccessful task-attempts . this is completely transparent to the application . the application-writer can take advantage of this by creating any side-files required in $ mapreduce.task.output.dir during execution of his reduce-task i.e. via #getworkoutputpath jobconf , and the framework will move them out similarly - thus she does n't have to pick unique paths per task-attempt . note : the value of $ mapreduce.task.output.dir during execution of a particular task-attempt is actually $ mapreduce.output.fileoutputformat.outputdir / _ temporary / _ $ taskid , and this value is set by the map-reduce framework . so , just create any side-files in the path returned by #getworkoutputpath jobconf from map/reduce task to take advantage of this feature . the entire discussion holds true for maps of jobs with reducer = none i.e. 0 reduces since output of the map , in that case , goes directly to hdfs . concode_field_sep PlaceHolder placeHolder concode_field_sep Path getTaskOutputPath concode_elem_sep void setWorkOutputPath concode_elem_sep RecordWriter<K,V> getRecordWriter concode_elem_sep boolean getCompressOutput concode_elem_sep void setCompressOutput concode_elem_sep void setOutputCompressorClass concode_elem_sep Class<? extends CompressionCodec> getOutputCompressorClass concode_elem_sep void checkOutputSpecs concode_elem_sep Path getPathForCustomFile concode_elem_sep Path getOutputPath concode_elem_sep void setOutputPath concode_elem_sep String getUniqueName