first steps in big data it' possible to use the popular tool impala to query hive data up to times faster we won' go into impala in this bookbut more information can be found at impala iowe already had short intro to mapreduce in but let' elaborate bit here because it' such vital part of hadoop mapreducehow hadoop achieves parallelism hadoop uses programming method called mapreduce to achieve parallelism mapreduce algorithm splits up the dataprocesses it in paralleland then sortscombinesand aggregates the results back together howeverthe mapreduce algorithm isn' well suited for interactive analysis or iterative programs because it writes the data to disk in between each computational step this is expensive when working with large data sets let' see how mapreduce would work on small fictitious example you're the director of toy company every toy has two colorsand when client orders toy from the web pagethe web page puts an order file on hadoop with the colors of the toy your task is to find out how many color units you need to prepare you'll use mapreduce-style algorithm to count the colors first let' look at simplified version in figure map file reduce green greenblueblue blueorange blue orange file green greenredred blueorange blue green blue orange red orange figure simplified example of mapreduce flow for counting the colors in input texts as the name suggeststhe process roughly boils down to two big phasesmapping phase--the documents are split up into key-value pairs until we reducewe can have many duplicates reduce phase--it' not unlike sql "group by the different unique occurrences are grouped togetherand depending on the reducing functiona different result can be created here we wanted count per colorso that' what the reduce function returns in reality it' bit more complicated than this though