CI: very slow tests
from a recent build on master, these builds are *very* slow. Yes we have 140k tests, but 30m is a lot of time (esp considering the azure builds are 2x faster).

needs some investigation on what is happening.
- https://travis-ci.org/github/pandas-dev/pandas/jobs/746072524
- https://travis-ci.org/github/pandas-dev/pandas/jobs/746072525

cc @pandas-dev/pandas-core 