Is there an asymmetric version of nominal correlation?
I found what I was looking for - it's called Theil's U, or the Uncertainty Coefficient.

I've used it in this Kaggle kernel, you can check it out for an example and code implementation in Python

EDIT: I also have a blogpost about it.