title: Stack overflow error occurred when same data frame is repeated inside pyspark
body: <p>When same dataframe is repeated inside loop then stack overflow error occurred.
  Data volume is just 40k records. Cluster size is tried with single node 14Gb/28gb.</p>
  <p>Sample data.</p>
  <pre><code>FT/RT,Country,Charge_Type,Tariff_Loc,Charge_No,Status,Validity_from,Validity_to,Range_Basis,Limited_Parties,Charge_Detail,Freetime_Unit,Freetime,Count_Holidays,Majeure,Start_Event,Same/Next_Day,Next_Day_if_AFTER,Availability_Date,Route_Group,Route_Code,Origin,LoadZone,FDischZone,PODZone,FDestZone,Equipment_Group,Equipment_Type,Range_From,Range_To,Cargo_Type,commodity,SC_Group,SC_Number,IMO,Shipper_Group,Cnee_Group,Direction,Service,haulage,Transport_Type,Option1,Option2,1st_of_Route_Group,1st_of_LoadZone,1st_of_FDischZone,1st_of_PODZone,1st_of_FDestZone,1st_of_Equipment_Group,1st_of_SC_Group,1st_of_Shipper_Group,1st_of_Cnee_Group,operationalFacilityGroup,operationalFacility,operator,commodityGroup,equipmentType,consignee,consigneeGroup,shipper,shipperGroup,serviceContract,serviceContractGroup,transportMode,agreementType
  FT,IN,DET,INCCU,34298,EXPIRED,02-07-2020,30-11-2020,C/B,Y,2,DAY,14,Y,N,DISCHARG,S,null,N,MSL,null,null,null,null,null,null,ADRY,null,null,null,null,2313,null,ONLINE1,null,null,null,IMP,null,null,null,null,null,A1,null,null,null,null,20BULK,null,null,null,INCCU,,MSL,MSL,null,,null,,null,ONLINE1,null,null,SPOT
  </code></pre>
  <p>Expected output as below</p>
  <p><a href="https://i.stack.imgur.com/S07jZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/S07jZ.png" alt="enter image description here" /></a></p>
  <p>Works for few records,if dataframe has mroe records stackoverflow error occured.</p>
  <p>Please find the attached screenshot.</p>
  <p><a href="https://i.stack.imgur.com/LKfuF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/LKfuF.png" alt="enter image description here" /></a></p>