Changes to major/minor locator on a DateTime index leads to messed up axis
#### Code Sample

```python
import matplotlib
matplotlib.use('Agg')
import matplotlib.ticker as ticker
import matplotlib.dates as mdates
from matplotlib import pyplot as plt

from datetime import datetime
import pandas

# Creates problems
hours = range(24)

# Works fine
# hours = list(range(12)) + list(range(13, 24))

times = []
for hour in hours:
     times.append(datetime(year=2019, month=5, day=6, hour=hour))
df_example = pandas.DataFrame({'column':range(len(times))}, index=times)

f, ax = plt.subplots()

# Works fine
# ax.plot(df_example.index, df_example['column'])

# Creates problems
df_example.plot(ax=ax)

hour_locator = mdates.HourLocator(byhour=range(0, 24, 6))
ax.xaxis.set_minor_formatter(mdates.DateFormatter('%H:%M'))
ax.xaxis.set_minor_locator(hour_locator)
ax.xaxis.set_major_locator(ticker.NullLocator())
ax.xaxis.set_major_formatter(ticker.NullFormatter())
f.savefig('example.png')
```
#### Problem description

I was plotting a DataFrame with a DateTime index and tried to change the major and minor ticks afterwards. I expected an x-axis with ticks every 6-hours. Instead the x-axis became filled with ticks at what I assume to be hourly intervals. 

This behavior does not occur, when either matplotlib is used to plot the columns `ax.plot(...)` or when the DateTime index contains gaps. Both cases are shown in the comments above. 

The issue appeared both with pandas versions 0.24.1 and 0.24.2.
#### Expected Output

![Expected](https://i.ibb.co/tDDJjQx/example.png)

#### Output

![Output](https://i.ibb.co/2ybKyWB/example.png)
