Spark Scala alternative Machine Learning Library?
The way you can use scikit-learn is basically broadcasting your data to the workers and then do different folds of crossvalidation or different parameter settings in your grid_search on different workers. That is all that the scikit-learn package in pySpark does as far as I know. This is similar to a normal mapping. Implementing this should be relatively easy for any Machine Learning library in Scala (I do not use Scala, so I cannot help you with suggestions in that regard)