BUG: Cleanup timedelta offset
- [x] closes #21762
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Fixed to use same unit parsing logic in `Timedelta` and `to_timedelta`. Needs to update after #23264 and #23259.



