Performance regression in replace.ReplaceDict.time_replace_series
Setup

```python
import pandas as pd
import numpy as np

inplace = False

N = 10 ** 5
start_value = 10 ** 5
to_rep = dict(enumerate(np.arange(N) + start_value))
s = pd.Series(np.random.randint(N, size=10 ** 3))

%timeit s.replace(to_rep, inplace=inplace)
```


```
# 1.0.2
2.48 s ± 53.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# master
5.77 s ± 117 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```


https://pandas.pydata.org/speed/pandas/index.html#replace.ReplaceDict.time_replace_series?p-inplace=False&commits=acb525a79fd3496a57b93fcfdb86be3de28a1815-aa8e869d76878f07dff065f947c99b5663342087 points to 

```
aa8e869d76 BLD: recursive inclusion of DLLs in package data (#33246)
610568c146 REF: remove replace_list kludge (#33445)
df68cceeb3 REF: .dot tests (#33214)
8d299e4ce1 REF: call pandas_dtype up-front in Index.__new__ (#33407)
e7cbe6dc1a BUG: df.iloc[:, :1] with EA column (#32959)
31ea45f8a3 BUG: Add test to ensure, that bug will not occur again. #33058 (#33072)
f334fcc681 Properly handle missing attributes in query/eval strings (#32408)
12ffb23a72 DOC: Change doc template to fix SA04 errors in docstrings #28792 (#32972)
7f276c8ba1 DOC: Fixed examples in `pandas/core/groupby/` (#33230)
982b4aadef CI: fix lint issue (#33461)
6bc8a49f25 DOC: Fix heading capitalization in doc/source/whatsnew - part2 (#32550) (#33403)
47de449681 API/TST: Call __finalize__ in more places (#33379)
916d1f3786 DOC: Fix EX01 in DataFrame.drop_duplicates (#33283)
4fc8c2515e DOC: Fix heading capitalization in doc/source/whatsnew - part3 (#32550) (#33429)
ef9b9387c8 BUG: Fix bug when concatenating Index of strings (#33436)
40fd73ab8f Update citation webpage (#33311)
3cca07c8a5 BUG: Fix replacing in `string` series with NA (pandas-dev#32621) (#32890)
a2cdd50427 INT: provide helpers for accessing the values of DataFrame columns (#33252)
efa85af76d PERF: improve IntegerArray fast constructor (#33359)
716689a9bf REGR: Fix construction of PeriodIndex from strings (#33304)
4a74463d02 Updated headers for files in doc/source/whatsnew (#33376)
542ef40bb4 Updated headers for files in doc/source/whatsnew (#33378)
12b0d4523a TYP: F used in decorators to _typing (#33456)
d72116b0e7 BUG/PLT: Multiple plots with different cmap, colorbars legends use first cmap (#33392)
2a68c12509 BUG: `weights` is not working for multiple columns in df.plot.hist (#33440)
b5f6e59cb4 BUG: Fix ValueError when grouping by read-only category (#33410) (#33446)
0c69615ad5 DEP: Bump min version of dateutil to 2.7.3 (#33363)
be86b6583f CLN: remove unnecessary Series._convert calls (#32949)
4334482c34 BUG/API: prohibit dtype-changing IntervalArray.__setitem__ (#32782)
6e3537dbab CLN: Static types in `pandas/_lib/lib.pyx` (#33329)
54f9b03cf6 BUG/REF: unstack with EA dtypes (#33356)
496c982b4e REF: reshape.concat operate on arrays, not SingleBlockManagers (#33125)
d72dc24e62 CLN: avoid accessing private functions (#33427)
a142ad7bb2 BUG: DataFrame.diff(axis=1) with mixed (or EA) dtypes (#32995)
185a654e3d BUG: scalar indexing on 2D DTA/TDA/PA (#33342)
5f2cdf8e1d REF: call _block_shape from EABlock.make_block (#33308)
5d0faa8ca5 BUG: Series.__getitem__ with MultiIndex and leading integer level (#33404)
fe42954e08 BUG: Timedelta == ndarray[td64] (#33441)
991f784a72 STY: Using __all__; removed "noqa" comment (#33143)
fbae09e11a CLN/TYP: redundant casts and unused ignores (#33453)
78d8af1fe3 Bump cython for asv environment (#33454)
```

perhaps one of based on the commit message. Not sure.

* 610568c146 REF: remove replace_list kludge (#33445)
* 3cca07c8a5 BUG: Fix replacing in `string` series with NA (pandas-dev#32621) (#32890)
