test_dateindex_conversion fails on Python 3.4 / NumPy 1.8.1 / MPL 1.4 master / Ubuntu 12.04
With a recent checkout from master (bfd5348d824a721dd0d896bb06e63e4ad801ba51), running this

```
python3.4 `which nosetests` pandas
```

gives me this:

```
======================================================================
ERROR: test_dateindex_conversion (pandas.tseries.tests.test_converter.TestDateTimeConverter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/pandas/tseries/tests/test_converter.py", line 77, in test_dateindex_conversion
    xp = converter.dates.date2num(dateindex)
  File "/usr/local/lib/python3.4/dist-packages/matplotlib/dates.py", line 310, in date2num
    return _to_ordinalf_np_vectorized(d)
  File "/usr/local/lib/python3.4/dist-packages/numpy/lib/function_base.py", line 1573, in __call__
    return self._vectorize_call(func=func, args=vargs)
  File "/usr/local/lib/python3.4/dist-packages/numpy/lib/function_base.py", line 1633, in _vectorize_call
    ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)
  File "/usr/local/lib/python3.4/dist-packages/numpy/lib/function_base.py", line 1597, in _get_ufunc_and_otypes
    outputs = func(*inputs)
  File "/usr/local/lib/python3.4/dist-packages/matplotlib/dates.py", line 204, in _to_ordinalf
    base = float(dt.toordinal())
AttributeError: 'numpy.datetime64' object has no attribute 'toordinal'

----------------------------------------------------------------------
Ran 7152 tests in 4107.297s

FAILED (SKIP=381, errors=1)
```

Looks like it's actually matplotlib's fault -- I notice there's a relevant ticket here:

https://github.com/matplotlib/matplotlib/issues/2259

But is this test expected to fail, as a result of that? And if so, should it be marked as skipped?

I'm using a very recent checkout of matplotlib (https://github.com/matplotlib/matplotlib/commit/4b1bd6301d69f856deca9c614af563f5fb4d1e90).
