title: When to use Kryo serialization in Spark?
body: <p>I am already compressing RDDs using <code>conf.set("spark.rdd.compress","true")</code> and <code>persist(MEMORY_AND_DISK_SER)</code>. Will using Kryo serialization make the program even more efficient, or is it not useful in this case? I know that Kryo is for sending the data between the nodes in a more efficient way. But if the communicated data is already compressed, is it even needed?</p>