Rolling max and min on datetime column incorrect when NaN in window
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
import numpy as np

df = pd.DataFrame({'B': [0, 1, np.nan, 3, 4], 'C': [4, 3, np.nan, 1, 0],
	'Time': [pd.Timestamp('20130101 09:00:00'), 
	pd.Timestamp('20130101 09:00:01'), 
	pd.Timestamp('20130101 09:00:02'), 
	pd.Timestamp('20130101 09:00:03'), 
	pd.Timestamp('20130101 09:00:04')]})

df.rolling('4s', on='Time').max()
df.rolling('4s', on='Time').min()

```
#### Problem description

When running a rolling max or min window on a datetime column, a NaN value seems to prevent the max or min function from considering values that follow it, even if those values are within the window. 
##### dataframe
```
     B                Time    C
0  0.0 2013-01-01 09:00:00  4.0
1  1.0 2013-01-01 09:00:01  3.0
2  NaN 2013-01-01 09:00:02  NaN
3  3.0 2013-01-01 09:00:03  1.0
4  4.0 2013-01-01 09:00:04  0.0

```
#### max() Output (column B)
```
In[1]: df.rolling('4s', on='Time').max()
Out[1]:
     B                Time    C
0  0.0 2013-01-01 09:00:00  4.0
1  1.0 2013-01-01 09:00:01  4.0
2  1.0 2013-01-01 09:00:02  4.0
3  1.0 2013-01-01 09:00:03  4.0
4  1.0 2013-01-01 09:00:04  3.0

```
#### Expected max() Output (column B)
```
In[1]: df.rolling('4s', on='Time').max()
Out[1]:
     B                Time    C
0  0.0 2013-01-01 09:00:00  4.0
1  1.0 2013-01-01 09:00:01  4.0
2  1.0 2013-01-01 09:00:02  4.0
3  3.0 2013-01-01 09:00:03  4.0
4  4.0 2013-01-01 09:00:04  3.0
```

#### min() Output (column C)
```
In[2]: df.rolling('4s', on='Time').min()
Out[2]:
     B                Time    C
0  0.0 2013-01-01 09:00:00  4.0
1  0.0 2013-01-01 09:00:01  3.0
2  0.0 2013-01-01 09:00:02  3.0
3  0.0 2013-01-01 09:00:03  3.0
4  1.0 2013-01-01 09:00:04  3.0

```
#### Expected min() Output (column C)
```
In[2]: df.rolling('4s', on='Time').min()
Out[2]:
     B                Time    C
0  0.0 2013-01-01 09:00:00  4.0
1  0.0 2013-01-01 09:00:01  3.0
2  0.0 2013-01-01 09:00:02  3.0
3  0.0 2013-01-01 09:00:03  1.0
4  1.0 2013-01-01 09:00:04  0.0

```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.13.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.4
pytest: None
pip: 18.0
setuptools: 40.4.3
Cython: None
numpy: 1.15.2
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>
