Is there a straightforward way to run pandas.DataFrame.isin in parallel?
Unfortunately, parallelization is not yet implemented in pandas. You can join this github issue if you want to participate in the development of this feature.

I don't know any "magic unicorn package" for this purposes, so the best thing will be write your own solution. But if you still don't want to spend time on that and want to learn something new – you can try the two methods built into MongoDB (map reduce and agg framework).  See mongodb_agg_framework.