sluggish `read_pickle` on pandas 1.0 ?
#### Code Sample, a copy-pastable example if possible

```python
auths = pd.read_pickle('auths.pkl')
```
#### Problem description

Suddenly it takes 14-18 minutes to read a mid-size (2.1 GB) pickle file after upgrading to pandas 1.0. 
Pandas 0.25 can read the very same file in 90 seconds (the file was created using pd.to_pickle a couple of months ago using pandas 0.25).

To try to make sure it has to do with Pandas 1.0, I created a couple of minimal virtual environments, one with latest pandas and one with 0.25. (I am using conda)

```python
conda create -n min_env_101 -y
conda activate min_env_101
conda install -y pandas
```

```python
conda create -n min_env_025 -y
conda activate min_env_025
conda install -y pandas=0.25.1
```

You can see the output of `pd.show_versions()` below. I noticed the latest pandas was installed along python 3.8.1, and pandas 0.25 with python 3.7.6, so I created another virtual environment with latest pandas, but making sure it uses python 3.7.6.

```python
conda create -n min_env_101_py376 python=3.7.6 -y
conda activate min_env_101_py376
conda install -y pandas
```

Then, reading the pickle file takes more than 14 minutes in the virtual environments with pandas 1.0, and just 93 seconds with pandas 0.25.

I checked the issues already reported and did not find anything similar. At some point I thought it was related to [this one](https://github.com/pandas-dev/pandas/issues/29542), but it cannot be since that one was using pandas 0.25.

So I am not sure what is going on (I did not see in pandas 1.0 release notes major changes related to pickle, but maybe I missed something?) and what to do about it.

EDIT: I cannot share the file and I guess you would not accept it either, 'cause reading pickle files may not be safe.

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]

**virtual environment with pandas 0.25.1**

--- 93.72341346740723 seconds ---
>>>
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.6.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : None.None

pandas           : 0.25.1
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 45.2.0.post20200210
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None

</details>

<details>

[paste the output of ``pd.show_versions()`` here below this line]

**virtual environment with pandas 1.0.1**

--- 854.9136216640472 seconds ---
>>>
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.8.1.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : English_United States.1252

pandas           : 1.0.1
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 45.2.0.post20200210
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : None
pyxlsb           : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
numba            : None

</details>

<details>

[paste the output of ``pd.show_versions()`` here below this line]

**virtual environment with pandas 1.0.1, making sure it uses python 3.7.6**

--- 1087.5447969436646 seconds ---
>>>
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.6.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : None.None

pandas           : 1.0.1
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 45.2.0.post20200210
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : None
pyxlsb           : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
numba            : None


</details>
