What is the best way to create a graph from a compact pandas dataframe?
You've identified your problem, although you may not realize it.  You have a graph with 40M edges, but you appropriately avoid the table explosion.  You do have to code that explosion in some form, because your graph needs all 40M edges.
For what little trouble it might save you, I suggest that you write a simple generator expression for the edges: take one node from PostalCode, iterating through the nearPC list for the other node.  Let Python and NetworkX worry about the in-line expansion.
You switch the nx build method you call, depending on the format you generate.  You do slow down the processing somewhat, but the explosion details get hidden in the language syntax.  Also, if there is any built-in parallelization between that generator and the nx method, you'll get that advantage implicitly.
I have a dataset of postal codes for each store and nearby postal codes for each. It looks like the following:




PostalCode
nearPC
Travel Time




L2L 3J9
[N1K 0A1', 'N1K 0A2', 'N1K 0A3', 'N1K 0A4', '...
[nan,nan,9,5,nan...]




I know I can explode the data but that would result in tons more rows ~40M. Another preprocessing step I can perform is to remove the values in each list where the travel time is not available. However, then I would need to remove it from the nearPC list.
Is there a way to incorporate networkx to create this graph? I've tried using
G = nx.from_pandas_edgelist(df,'near_PC','PostalCode',['TravelTime'])
but I don't think it allows lists as the source or targets.
TypeError: unhashable type: 'list'
Is there a way around this? If not how can I remove the same indices of a list per row based on a conditional in an efficient way?