CLN: tslibs imports and unused variables
- [x] tests added / passed


- Remove unused imports, mostly `Py_ssize_t`, and variables
- Remove a block that is not hit after the `DatetimeIndex.normalize` performance upgrades https://github.com/pandas-dev/pandas/pull/23634 
