BUG: Fix duplicates in intersection of multiindexes
- [x] closes #36915
- [x] xref #31326 (closes the intersection part)
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Seems like this was not introduced on purpose. Probably introduced in #31312