PERF: pandas' import time
I wouldn't normally be concerned about this, as of it course it only happens once, but our import time has gotten quite long, to the point I notice it hanging my `ipython` startup.  

I don't have a good sense of what would be required to improve this, probably deferring more imports to be just in time?

on `0.20.2` - each import in a separate process
```python
>>> import timeit
>>> timeit.timeit('import pandas', number=1)
1.0524120664765442
>>> quit()

>>> timeit.timeit('import numpy', number=1)
0.1550516492424085
>>> quit()

>>> timeit.timeit('import matplotlib', number=1)
0.24022248792225612
```

Below is a single process, importing deps first
```python
>>> import timeit
>>> timeit.timeit('import matplotlib', number=1)
0.2508611853454641
>>> timeit.timeit('import numpy', number=1)
1.2033075643458346e-05
>>> timeit.timeit('import pandas', number=1)
0.840005673777485
```
