Pd.series.map performance
There are other places also to refactor to improve the performance, but this current change has greater impact aswell.

- [ ] closes #34717
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry `improved the performance of pd.series.map`
