BUG: fix Categorical.astype for dtype=np.int32 argument
- [x] closes #39402
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Fixing a bug I introduced in #37355