Is there a way to drop duplicated rows based on an unhashable column?
It is true that a set is not hashable (it cannot be used as a key in a hashmap a.k.a a dictionary). So what you can do is to just convert the column to a type that is hashable - I would go for a tuple.

I made a new column that is just the "z" column you had, converted to tuples. Then you can use the same method you tried to, on the new column:

In [1] : import pandas as pd 
    ...:  
    ...: lnks = [ ( 'a' , 'b' , { 'a' , 'b' } ) , ( 'b' , 'c' , { 'b' , 'c' } ) 
    ...: , ( 'b' , 'a' , { 'a' , 'b' } ) ] 
    ...: lbls = [ 'x' , 'y' , 'z' ] 
    ...: df = pd.DataFrame.from_records( lnks , columns = lbls)                 

In [2]: df["z_tuple"] = df.z.apply(lambda x: tuple(x))                         

In [3]: df.drop_duplicates(subset="z_tuple", keep="first")                     
Out[3]: 
   x  y       z z_tuple
0  a  b  {b, a}  (b, a)
1  b  c  {c, b}  (c, b)


The apply method lets you apply a function to each item in a column, and then returns the values as a new column (a Pandas Series object). This lets you assign it back to the original DataFrame as a new column, as I did.

You can also remove the "z_tuple" column then if you no longer want it:

In [4] : df.drop("z_tuple", axis=1, inplace=True)                               

In [5] : df                                                                     
Out[5] : 
   x  y       z
0  a  b  {b, a}
1  b  c  {c, b}
2  b  a  {b, a}