Test For MultiIndex not contained item
- [x] closes #21094
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

I just made the test that was requested in this issue but in fact it isn't actually passing like mentioned there. Maybe my master is not updated for this. Should tests be added to whatsnew entry too?
