PERF: DataFrame.mean() is slow with datetime columns
#### Code Sample, a copy-pastable example if possible

```python
import numpy as np
import pandas as pd

# A DataFrame containing a datetime64 column
n = 1000000
df = pd.DataFrame({
    "x": np.random.normal(0.0, 1.0, n),
    "d": pd.date_range(pd.datetime.today(), periods=n, freq="1H").tolist()
})

# Calculating the mean of individual columns is pretty much instantaneous
df["x"].mean()
## 1000 loops, best of 3: 1.35 ms per loop
df["d"].mean()
## 100 loops, best of 3: 2.91 ms per loop

# Using the DataFrame's .mean() method, it takes a really long time
%timeit df.mean()
## 1 loop, best of 3: 9.23 s per loop
```
#### Problem description

When DataFrame contains a `datetime64` column, the time taken to run the `.mean()` method for the whole DataFrame is thousands of times longer than than time taken to run the `.mean()` method on each column individually.

#### Expected Output

Answer is correct; just too slow.

#### Output of ``pd.show_versions()``

<details>

pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.6.1.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 18.7.0
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : en_US.UTF-8
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.3
numpy            : 1.13.3
pytz             : 2017.2
dateutil         : 2.7.3
pip              : 19.2.3
setuptools       : 44.0.0.post20200106
Cython           : 0.25.2
pytest           : 3.0.7
hypothesis       : None
sphinx           : 1.5.6
blosc            : None
feather          : None
xlsxwriter       : 0.9.6
lxml.etree       : 3.7.3
html5lib         : 0.999
pymysql          : 0.9.3
psycopg2         : 2.8.3 (dt dec pq3 ext lo64)
jinja2           : 2.10.1
IPython          : 5.3.0
pandas_datareader: None
bs4              : 4.6.0
bottleneck       : 1.2.1
fastparquet      : None
gcsfs            : None
lxml.etree       : 3.7.3
matplotlib       : 2.0.2
numexpr          : 2.6.2
odfpy            : None
openpyxl         : 2.4.7
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : 0.19.1
sqlalchemy       : 1.3.8
tables           : 3.3.0
xarray           : None
xlrd             : 1.0.0
xlwt             : 1.2.0
xlsxwriter       : 0.9.6

</details>

I asked about this on Stack Overflow (https://stackoverflow.com/questions/59759107/how-to-avoid-poor-performance-of-pandas-mean-with-datetime-columns); one respondent hazarded a guess the the issue may lie with these lines of code (but no rigorous debugging was done).

https://github.com/pandas-dev/pandas/blob/v0.25.3/pandas/core/arrays/datetimes.py#L601-L603