Saving Large Spark ML Pipeline to HDFS
You can do the following:
A Pipeline can be made of other pipelines. Isn't that great?
A Pipeline inherit from the Estimator class and by definition, a PipelineStage can be either an Estimator or a Transformer.
This way, you can build smaller pipelines, save them separately and on the other software/class, join them again as a single one and call transform on the DataFrame.