spark python product top 5 numbers from a file
You distribute the data you read among nodes.
Every node finds it's 5 local maximums.
You combine all the local maximums and you keep the 5 max of them,
which is the answer.
total noob question. I have a file that contains a number on each line, there are approximately 5 millions rows, each row has a different number, how do i find the top 5 values in the file using spark and python.