title: Use Requests Module and return response to Pyspark Dataframe
body: <p>I'm hitting an API that sends a JSON response with two key:value pairs. I'm currently saving the response to my dataframe by hitting the API 2 different times and using withColumn to save each key:value pair to a column separately, instead of hitting the API once and saving both key:value pairs at once. Does anyone have guidance on how to do this properly?</p>
  
  <p>My dataframe is one column that contains the values I ship to the API:</p>
  
  <pre><code>   businessId
      dksldfaw2
      kkldsdok3
      djdfkdfk3
      23lksdlk8
  </code></pre>
  
  <p>The JSON response:</p>
  
  <pre><code>{response:
  FirstDataPoint: "SomethingSomething"
  SecondDataPoint: "SomethingSomething"}
  </code></pre>
  
  <p>My code (which calls the API twice to parse one response):</p>
  
  <pre><code>def FirstVariableCode(businessId):
    response = api.query(id=businessId)
    x = response['FirstVariable'].encode('utf-8').strip()
    return x
  
  
  def SecondVariableCode(businessId):
    response = api.query(id=businessId)
    y = response['SecondVariable'].encode('utf-8').strip()
    return y
  
  FirstVariableCode = udf(FirstVariableCode, StringType())
  SecondVariableCode = udf(SecondVariableCode, StringType())
  
  df.withColumn('FirstVariable', FirstVariableCode(df.businessId))
  df.withColumn('SecondVariable', SecondVariableCode(df.businessId))
  </code></pre>
  
  <p>How can I hit the API once and save both values to their respective columns correctly? I feel like I'm using 'withColumn' as a crutch and not doing this properly.</p>
  
  <p>Thanks</p>