BUG: Crosstab bug in #18321
- [x] closes #18321
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry
Fixing bug in #18321
The bug is because the function `df.pivot_table` tries to remove only the top level of the column in crosstab data frame. When the series names are tupples, the added column `__dummy__` in function `crosstab` has several levels, meaning removing only the top layer in the resulting crosstab as done in the current `pivot_table` function won't be enough. The fix keeps track of the exact added column, then remove the extra levels in resulting crosstab data frame it in the same function rather than delegate the removal to `df.pivot_table`. Based on my investigation, `df.pivot_table` removes the dummy layers after all the calculation, so moving the removal of the extra layers outside won't affect the current behavior. Also, by add the columns and removing the resulting extra layers in the crosstab in the same function, the logic is easier to read.