CI: test_drop_duplicates_categorical_non_bool is flaky
I've noticed recently that the xfailed part of the following test is occasionally xpassing, which is causing the test suite to fail due to being strict with xpass:

https://github.com/pandas-dev/pandas/blob/90a85e0b154833577a4023f643d8aace0c0ae7f5/pandas/tests/series/test_analytics.py#L1405-L1412

The quick fix is to update the `xfail` with `strict=False` but additional investigation into why this is occasionally passing would also be appreciated.

Examples of the xpass:
- Select Attempt # 1 here: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=10441&view=logs
- https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=10459&view=logs&jobId=521b7dfd-2989-5ff8-bc8c-7481906480fa&taskId=07b8d9d4-6363-5e2d-bc2b-146a30521256&lineStart=64&lineEnd=70&colStart=1&colEnd=24
