Scikit-learn and pyspark integration
The fact that you are using spark shouldn't hold you from using external python libraries.
You can import sklearn library in your spark-python code, and use sklearn logistic regression model with the saved pkl file.
I have trained a logistic regression model in sklearn and saved the model to .pkl files. Is there a method of using this pkl file from within spark?