How to read file names and word count in respective files in Hadoop?
Question: I am trying to fetch file names from sequence file from hadoop with the help
of dumbo package of python. But it provides me some kind of identifier. How
can i map this to file name?

Below is my steps on hadoop system for getting filenames :

**_Steps 1)_ Generating Sequence file**

Command :

    
    
    hadoop jar /mnt/Clustering/Checking/AllJars/binarypig-1.0-SNAPSHOT-jar-with-dependencies.jar com.endgame.binarypig.util.BuildSequenceFileFromDir /mnt/Clustering/Checking/text_files text_files_seq
    

**_Step 2)_ Running python script on sequence file through hadoop**

Command:

    
    
    dumbo start dumbo_map_red.py -input text_files_seq -output out_res -hadoop /usr/local/hadoop
    

**_Step 3)_ Getting output in local directory**

Command:

    
    
    dumbo cat out_res/part-* -hadoop /usr/local/hadoop > out_res.txt
    

where dumbo_map_red.py is

    
    
    #!/usr/bin/env python
    
    def mapper(key, value):
        yield key, 1
    
    def reducer(key, values):
        yield key, sum(values)
    
    if __name__ == "__main__":
        import dumbo
        dumbo.run(mapper, reducer)
    

Please help me know how to fetch filenames. If their is other package in
python which will allow me to work that way, please let me know..


Answer: Finally got the hint of mapping identifier in sequence file to actual file.

**The identifier is MD5 of file in the directory.**

