title: Can only zip RDDs with same number of elements in each partition : PySpark Mlib
body: <p>I get <code>Can only zip RDDs with same number of elements in each partition</code> on running bisecting k-means on large datasets  . 
  I am using spark 2.0.0</p>
  
  <p>Code :   </p>
  
  <pre><code>data = spark_context.textFile(input_path)
  parsed_data = data.map(lambda line: [float(x) for x in line.split(delimiter)])
  model = BisectingKMeans.train(parsed_data.map(lambda x: x[1:]), no_clusters, maxIterations=iterations)
  pred = model.predict(parsed_data.map(lambda x: x[1:])).collect()
  </code></pre>
  
  <p>The input file is of size 17 GB. 
  Note: First field in file is ID , so I am skipping that when i do clustering. 
  Spark does zip opearation internally. I think the issue is fixed for ordinary kmeans. <a href="https://issues.apache.org/jira/browse/SPARK-13178?focusedCommentId=15131436&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15131436" rel="nofollow noreferrer">https://issues.apache.org/jira/browse/SPARK-13178?focusedCommentId=15131436&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15131436</a> </p>