REF: smarter NaN handling in remove_unused_levels()
- [x] tests passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Sorry for the bad timing @jreback , only after you merged #18426 this simpler way to proceed came to my mind.
