Get the count of each key in each Mapper or globally in Spark MapReduce model
You can use pairRDD.countByKey() function for counting the rows according their keys.
We need to get the count of each key (the keys are not known before executing), and do some computation dynamically in each Mapper. The key count could be global or only in each Mapper. What is the best way to implement that? In Hadoop this is similar to an aggregator function.
The accumulator in Spark needs to be defined before the Mapper jobs run. But we do not know what and how many keys are there.