Bigdata cluster compatible distributed predictive model
Spark does have a pretty good Python API, check out this tutorial.

For traditional Hadoop stack, take a look at mrjob, it lets you write MapReduce jobs in Python and run them on several platforms.