title: Difference between QuantileDiscretizer and Bucketizer in Spark
body: <p>This may be a novice question, however I'm unable to comprehend if there is any specific advantage of using <code>QuantileDiscretizer</code> over <code>Bucketizer</code>in spark 2.1 ?</p>
  
  <p>I understand that <code>QuantileDiscretizer</code> is an estimator and handles NAN values whereas <code>Bucketizer</code> is a transformer and raises error if data has NAN values.</p>
  
  <p>from the spark <a href="http://spark.apache.org/docs/latest/ml-features.html" rel="noreferrer">documentation</a> , below code produces similar outputs</p>
  
  <pre><code>from pyspark.ml.feature import QuantileDiscretizer
  from pyspark.ml.feature import Bucketizer
  
  data = [(0, 18.0), (1, 19.0), (2, 8.0), (3, 5.0), (4, 2.2)]
  df = spark.createDataFrame(data, ["id", "hour"])
  
  result_discretizer = QuantileDiscretizer(numBuckets=3, inputCol="hour",outputCol="result").fit(df).transform(df)
  result_discretizer.show()
  
  splits = [-float("inf"),3, 10,float("inf")]
  result_bucketizer = Bucketizer(splits=splits, inputCol="hour",outputCol="result").transform(df)
  result_bucketizer.show()
  </code></pre>
  
  <p>Output :</p>
  
  <pre><code>+---+----+------+
  | id|hour|result|
  +---+----+------+
  |  0|18.0|   2.0|
  |  1|19.0|   2.0|
  |  2| 8.0|   1.0|
  |  3| 5.0|   1.0|
  |  4| 2.2|   0.0|
  +---+----+------+
  
  +---+----+------+
  | id|hour|result|
  +---+----+------+
  |  0|18.0|   2.0|
  |  1|19.0|   2.0|
  |  2| 8.0|   1.0|
  |  3| 5.0|   1.0|
  |  4| 2.2|   0.0|
  +---+----+------+
  </code></pre>
  
  <p>Please let me know if there is any significant advantage of one over other?</p>