title: How does spark driver handle data bigger than its memory? Think of TBs of data on a cluster and we do a reduce by key or a collect
body: <p>How does spark driver handle data bigger than its memory? Think of TBs of data on a cluster and we do a reduce by key or a collect. Does it spill data to disk? Even if it does, what happens when we have a GB of Disk space on the driver and We are handling TBs.</p>