title: Spark / Python : cannot pickle '_thread.RLock' object when passing multiple columns to UDF
body: <p>Using jupyter notebook, I have 2 dataframes :</p>
  <p>ApplePrices :</p>
  <pre><code>+--------------------+-------+---------+--------+
  |          Model_name|  Price|  Country|Currency|
  +--------------------+-------+---------+--------+
  |        24-inch iMac|1919.01|Australia|     AUD|
  |         AirPods Max| 908.47|Australia|     AUD|
  |         AirPods Pro|  403.2|Australia|     AUD|
  |AirPods(2nd gener...| 221.31|Australia|     AUD|
  |AirPods(3rd gener...| 281.94|Australia|     AUD|
  |Apple Pencil (2nd...|  201.1|Australia|     AUD|
  |         Apple TV 4K| 251.62|Australia|     AUD|
  |      Apple Watch SE| 433.52|Australia|     AUD|
  |Apple Watch Series 3| 302.15|Australia|     AUD|
  |         MacBook Air| 1514.8|Australia|     AUD|
  |         Magic Mouse| 110.15|Australia|     AUD|
  |          Sport Band|  69.73|Australia|     AUD|
  |                iPad| 504.26|Australia|     AUD|
  |            iPad Pro|1211.64|Australia|     AUD|
  |           iPhone 12|1009.53|Australia|     AUD|
  |           iPhone 13|1211.64|Australia|     AUD|
  |           iPhone SE| 686.16|Australia|     AUD|
  |        24-inch iMac|1597.97|   Canada|     CAD|
  |         AirPods Max|  778.5|   Canada|     CAD|
  |         AirPods Pro| 328.79|   Canada|     CAD|
  +--------------------+-------+---------+--------+
  </code></pre>
  <p>currencyConversion : (ratio 1$ = xx Currency)</p>
  <pre><code>+--------+--------------------+
  |ISO_4217|Dollar_To_Curr_Ratio|
  +--------+--------------------+
  |     EUR|                0.89|
  |     CAD|                1.27|
  |     CZK|                21.7|
  |     DKK|                6.59|
  |     HUF|              319.05|
  |     INR|               74.42|
  |     MXN|                20.5|
  |     NOK|                 8.9|
  |     PHP|               51.22|
  |     PLN|                4.03|
  |     RUB|               76.28|
  |     SEK|                9.15|
  |     THB|               33.57|
  |     USD|                 1.0|
  |     AUD|                 1.4|
  +--------+--------------------+
  </code></pre>
  <p>My goal is to convert all prices contained in ApplePrices to US dollars.
  So I used UDF :</p>
  <pre><code>def convertPriceToDollar(price, currency):
      ratio = currencyConversion.select(&quot;Dollar_To_Curr_Ratio&quot;).where(col(&quot;ISO_4217&quot;) == currency)
      return price * ratio
  toDollarConverter = udf(convertPriceToDollar)
  </code></pre>
  <p>And select by calling the function by passing the 2 columns :
  <code>applePrices.select(toDollarConverter('Price', 'Currency').alias(&quot;Price&quot;, &quot;truc&quot;)).show()</code></p>
  <p>But I got this error : <code>PicklingError: Could not serialize object: TypeError: cannot pickle '_thread.RLock' object</code></p>
  <p>It works fine if I just pass one column to the function like this :</p>
  <pre><code>def convertPriceToDollar(price):
      return price * 1.27
  toDollarConverter = udf(convertPriceToDollar)
  
  applePrices.select(toDollarConverter('Price').alias(&quot;Price&quot;)).show()
  </code></pre>
  <p>Results :</p>
  <pre><code>+------------------+
  |             Price|
  +------------------+
  |         2437.1427|
  |         1153.7569|
  |           512.064|
  |          281.0637|
  |          358.0638|
  |           255.397|
  |319.55740000000003|
  |          550.5704|
  |383.73049999999995|
  |          1923.796|
  |          139.8905|
  |           88.5571|
  |          640.4102|
  |1538.7828000000002|
  |         1282.1031|
  |1538.7828000000002|
  |          871.4232|
  |         2029.4219|
  |           988.695|
  |          417.5633|
  +------------------+
  </code></pre>