title: Getting values from a pandas data frame that match values in a Pyspark data frame columns to create a new pysaprk column
body: <p>I have the following pandas df:</p>
  <pre><code>cbg_fips    latitude    longitude
  </code></pre>
  <p>0   360050001000    40.798418   -73.888787</p>
  <br>
  and a Pyspark dataframe that looks like this:
  <br>
  <p>|  safegraph_place_id| census_block_group | num_visits|</p>
  <p>|sg:69249d17cb754b...|      361190148051    |            4|</p>
  <p>I want to cross the census_block_group column in pyspark with the cbg_fips in pandas so that I can get the latitude and longitude values of each row (stored in the pandas df) in census_block_group.
  I tried this so far and is not working:</p>
  <pre><code>def lat_lon(index, cbg):
    if index == 0: next(cbg)
    if cbg!= None:
    i = locations.index[str(locations['cbg_fips']) == (cbg)]
  
  lat = locations[latitude][i]
  lon = locations['longitude'][i]
  return lat, lon
  
  home = visitor_home_cbgs.select(&quot;census_block_group&quot;)
  home_latLon = home.rdd.mapPartitionsWithIndex(lat_lon)
  home_latLon.take(2)
  </code></pre>
  <p>home is a column in my data frame in pyspark visitor_home_cbgs.</p>