Error when max with a datetime.date Series


```python
import datetime as dt
s = pd.Series([dt.date(2018,1,1), None, dt.date(2018,1,1)])
print(s.max()) # TypeError: '>=' not supported between instances of 'datetime.date' and 'float'

s1 = pd.to_datetime(s).dt.date
print(s1.max()) # TypeError: '>=' not supported between instances of 'datetime.date' and 'float'
print(max(s1)) # OK

s2 = pd.Series([dt.datetime(2018,1,1), None, dt.datetime(2018,1,1)])
print(s2.max()) # OK
```
#### Problem description

It's a little tricky for this bug. 
I received an error when I tried to maximize a series type of **date** with None value, but it failed throwing an error. 
Then using to_datetime().dt.date, it converts to date. Still not work for s1.max(), but this time, works fine for max(s1), Too noisy.
While for the **datetime**, it works fine for everything.

#### Expected Output

#### Output of ``pd.show_versions()``

<details>

[INSTALLED VERSIONS
------------------
commit: None
python: 3.6.5.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.23.4
pytest: None
pip: 18.1
setuptools: 39.0.1
Cython: None
numpy: 1.15.1
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.3
openpyxl: 2.5.9
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
None]

</details>
