title: efficiently calculating connected components in pyspark
body: <p>I'm trying to find the connected components for friends in a city. My data is a list of edges with an attribute of city.</p>
  
  <p>City | SRC | DEST</p>
  
  <p>Houston Kyle -> Benny</p>
  
  <p>Houston Benny -> Charles</p>
  
  <p>Houston Charles -> Denny</p>
  
  <p>Omaha Carol -> Brian</p>
  
  <p>etc. </p>
  
  <p>I know the connectedComponents function of pyspark's GraphX library will iterate over all the edges of a graph to find the connected components and I'd like to avoid that. How would I do so?</p>
  
  <p>edit:
  I thought I could do something like</p>
  
  <p>select connected_components(*) from dataframe
  groupby city</p>
  
  <p>where connected_components generates a list of items.</p>