PERF: Slow import-time expression slowing down test collection
### 

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this issue exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [X] I have confirmed this issue exists on the master branch of pandas.


### Reproducible Example

I'm a pytest dev that's been looking to speed up collection time, using pandas as a benchmark. I thought you'd be interested to know that there is a case which is slow (~1s) just because it does something heavy at the module scope:

https://github.com/pandas-dev/pandas/blob/65998341034861527517756e6dd134dafea2b766/pandas/tests/test_algos.py#L1800-L1805

These arrays are heavy to create, better to make it lazy by wrapping in a lambda or just split to two separate tests.

There's also something slow in the pandas/tests/reshape/merge/test_merge.py module but I'm not sure what.

### Installed Versions

```
INSTALLED VERSIONS
------------------
commit           : 65998341034861527517756e6dd134dafea2b766
python           : 3.9.7.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.14.6-arch1-1
Version          : #1 SMP PREEMPT Sat, 18 Sep 2021 16:19:35 +0000
machine          : x86_64
processor        : 
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.4.0.dev0+833.g6599834103
numpy            : 1.21.2
pytz             : 2021.3
dateutil         : 2.8.2
pip              : 21.2.4
setuptools       : 58.0.4
Cython           : 0.29.24
pytest           : 6.3.0.dev745+g483f239d0
hypothesis       : 6.23.1
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : 3.0.1
lxml.etree       : 4.6.3
html5lib         : 1.1
pymysql          : None
psycopg2         : None
jinja2           : 3.0.1
IPython          : None
pandas_datareader: None
bs4              : 4.10.0
bottleneck       : None
fsspec           : None
fastparquet      : None
gcsfs            : None
matplotlib       : 3.4.3
numexpr          : 2.7.3
odfpy            : None
openpyxl         : 3.0.9
pandas_gbq       : None
pyarrow          : None
pyxlsb           : None
s3fs             : None
scipy            : 1.7.1
sqlalchemy       : None
tables           : 3.6.1
tabulate         : 0.8.9
xarray           : None
xlrd             : 2.0.1
xlwt             : 1.3.0
numba            : None
```

### Prior Performance

_No response_