title: Spark dataframe to pandas profiling
body: <p>I am trying to achieve a data profiling with pandas-profiling library. i am fetching data directly from hive. this is the error i am receiving</p>
  <pre><code>Py4JJavaError: An error occurred while calling o114.collectToPython.
  : org.apache.spark.SparkException: Job aborted due to stage failure: Task 2 in stage 14.0 failed 4 times, most recent failure: Lost task 2.3 in stage 14.0 (TID 65, bdgtr026x30h4.nam.nsroot.net, executor 11): org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 0, required: 15823824. To avoid this, increase spark.kryoserializer.buffer.max value.
  </code></pre>
  <p>i tried to set my spark on jupyter notebook in python but i am receiving the same error</p>
  <pre><code>spark.conf.set(&quot;spark.kryoserializer.buffer.max&quot;, &quot;512&quot;)
  spark.conf.set('spark.kryoserializer.buffer.max.mb', 'val')
  </code></pre>
  <p>based on my code, am imissing any steps?</p>
  <pre><code>df = spark.sql('SELECT id,acct from tablename').cache()
  report = ProfileReport(df.toPandas())
  </code></pre>