Make a dataframe containing rows that were not matched after merging df1 and df2
new_df = df2[~df2.index.isin(matched.index)]
Explanation: You are saying "keep only the rows in df2 that are not in the matched data frame, and save this as a new dataframe"
I have df1 with around 3,67,000 rows.
df2 has 30k rows.
Their common columns are first_name, middle_name and last_name, where first name and last name are exact matches, and middle_name has some constraints.
The matched df has 20k rows.
I want to make a dataframe containing df2-matched (30k-20k= 10k rows).
Essentially, I want to find the rows in df2 that were not a match to any rows in df1, but I cannot concat or merge because the columns are different.