title: Mysterious 'pyarrow.lib.ArrowInvalid: Floating point value truncated' ERROR when use toPandas() on a DataFrame in pyspark
body: <p>I use toPandas() on a DataFrame which is not very large, but I get the following exception:</p>
  
  <pre><code>18/10/31 19:13:19 ERROR Executor: Exception in task 127.2 in stage 13.0 (TID 2264)
  org.apache.spark.api.python.PythonException: Traceback (most recent call last):
      File "/home/hadoop/spark2.3.1/python/lib/pyspark.zip/pyspark/worker.py", line 230, in main
        process()
      File "/home/hadoop/spark2.3.1/python/lib/pyspark.zip/pyspark/worker.py", line 225, in process
        serializer.dump_stream(func(split_index, iterator), outfile)
      File "/home/hadoop/spark2.3.1/python/lib/pyspark.zip/pyspark/serializers.py", line 261, in dump_stream
        batch = _create_batch(series, self._timezone)
      File "/home/hadoop/spark2.3.1/python/lib/pyspark.zip/pyspark/serializers.py", line 239, in _create_batch
        arrs = [create_array(s, t) for s, t in series]
      File "/home/hadoop/spark2.3.1/python/lib/pyspark.zip/pyspark/serializers.py", line 239, in &lt;listcomp&gt;
        arrs = [create_array(s, t) for s, t in series]
      File "/home/hadoop/spark2.3.1/python/lib/pyspark.zip/pyspark/serializers.py", line 237, in create_array
        return pa.Array.from_pandas(s, mask=mask, type=t)
      File "pyarrow/array.pxi", line 474, in pyarrow.lib.Array.from_pandas
      File "pyarrow/array.pxi", line 169, in pyarrow.lib.array
      File "pyarrow/array.pxi", line 69, in pyarrow.lib._ndarray_to_array
      File "pyarrow/error.pxi", line 81, in pyarrow.lib.check_status
  pyarrow.lib.ArrowInvalid: Floating point value truncated
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:298)
          at org.apache.spark.sql.execution.python.ArrowPythonRunner$$anon$1.read(ArrowPythonRunner.scala:171)
          at org.apache.spark.sql.execution.python.ArrowPythonRunner$$anon$1.read(ArrowPythonRunner.scala:121)
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:252)
          at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
          at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:439)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
          at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage19.agg_doAggregateWithKeys_0$(Unknown Source)
          at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage19.processNext(Unknown Source)
          at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
          at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$10$$anon$1.hasNext(WholeStageCodegenExec.scala:614)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
          at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:125)
          at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
          at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
          at org.apache.spark.scheduler.Task.run(Task.scala:109)
          at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
  </code></pre>
  
  <p>Sometimes, this exception can be ignored and I can get the right result, but more often, the program exited. Does anyone know about this mysterious error?</p>