title: Call function in pyspark with values from dataframe as strings
body: <p>I have to call a function <code>func_test(spark,a,b)</code> which accepts two string values and create a df out of it. <code>spark</code> is a <code>SparkSession</code> variable
  These two string values are two columns of another dataframe and would be different for different rows of that dataframe.</p>
  
  <p>I am unable to achieve this.</p>
  
  <p>Things tried so far:<br>
  1.</p>
  
  <pre><code>ctry_df = func_test(spark, df.select("CTRY").first()["CTRY"],df.select("CITY").first()["CITY"])
  </code></pre>
  
  <p>Gives CTRY and CITY of only the first record of the df.</p>
  
  <p>2.</p>
  
  <pre><code>ctry_df = func_test(spark, df['CTRY'],df['CITY'])
  </code></pre>
  
  <p>Gives <code>Column&lt;b'CTRY'&gt;</code> and <code>Column&lt;b'CITY'&gt;</code> as values.</p>
  
  <p>Example:
  df is:</p>
  
  <pre><code>+----------+----------+-----------+
  |     CTRY |     CITY |    XYZ    |
  +----------+----------+-----------+
  |      US  |     LA   |      HELLO|                                    
  |      UK  |     LN   |      WORLD|
  |      SN  |     SN   |      SPARK|
  +----------+----------+-----------+
  </code></pre>
  
  <p>So, I want first call to fetch <code>func_test(spark,US,LA)</code>; second call to go <code>func_test(spark,UK,LN)</code>; third call to be <code>func_test(spark,SN,SN)</code> and so on.</p>
  
  <pre><code>Pyspark - 3.7
  Spark - 2.2
  </code></pre>
  
  <p>Edit 1:</p>
  
  <p>Issue in detail:</p>
  
  <p><code>func_test(spark,string1,string2)</code> is a function which accepts two string values. Inside this function is a set of various dataframe operations done. For example:- First spark sql in the func_test is a normal select and these two variables string1 and string2 are used in the where clause. The result of this spark sql which generates a df is a temp table of next spark sql and so on. Finally, it creates a df which this function <code>func_test(spark,string1,string2)</code> returns.</p>
  
  <p>Now, In the main class, I have to call this func_test and the two parameters string1 and string2 will be fetched from records of dataframe. So that, first func_test call generates query as <code>select * from dummy where CTRY='US' and CITY='LA'</code>. And the subsequent operations happen which results in df. Second call to func_test becomes <code>select * from dummy where CTRY='UK' and CITY='LN'</code>. Third call becomes <code>select * from dummy where CTRY='SN' and CITY='SN'</code> and so on.</p>