Why my hadoop output is many parts of file?
Question: I tried to count the frequency of word, and write the file:  
`mapper.py`:

    
    
    #!/usr/bin/env python
    
    import sys
    
    # input comes from STDIN (standard input)
    for line in sys.stdin:
        # remove leading and trailing whitespace
        line = line.strip()
        # split the line into words
        words = line.split()
        # increase counters
        for word in words:
            # write the results to STDOUT (standard output);
            # what we output here will be the input for the
            # Reduce step, i.e. the input for reducer.py
            #
            # tab-delimited; the trivial word count is 1
            print '%s\t%s' % (word, 1)
    

use the hadoop statement: `hadoop streaming \ -input
"/app/hadoop_learn_test/book.txt" \ -mapper "python mapper.py" \ -reducer
"cat" \ -output "/app/hadoop_learn_test/book_out" \ -file "mapper.py" \ ` the
`book.txt` is:

    
    
    foo foo quux labs foo bar quux
    

but I got 400 files named as `part-00000.gz` and when I use `hadoop dfs -cat
path` to get the contents, I got nothing.

Why can't I got the result?

I use `cat book.txt | python mapper.py | sort` in the local terminal and got the following: 
    
    
    bar     1
    foo     1
    foo     1
    foo     1
    labs    1
    quux    1
    quux    1
    


Answer: i think You need to use Counter

    
    
    #!/usr/bin/env python
    
    import sys
    from collection import Counter
    
    # input comes from STDIN (standard input)
    for line in sys.stdin:
        # remove leading and trailing whitespace
        line = line.strip()
        # split the line into words
        words = line.split()
        # increase counters
        wordcount=Counter(words)
    
        for word,count in wordcount.items():
    
            print '%s\t%s' % (word, count)
    

