PERF: Some asv tests take a long time to setup
I hacked asv to log the total execution time, including setup, for each benchmark. Some of these are parametrized over several cases, so they may not actually be slow. `time` is in seconds. 

<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>time</th>
      <th>file</th>
      <th>klass</th>
      <th>method</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>68</th>
      <td>00:03:11.379267</td>
      <td>frame_ctor</td>
      <td>FrameConstructorDTIndexFromOffsets</td>
      <td>time_frame_ctor</td>
    </tr>
    <tr>
      <th>394</th>
      <td>00:01:01.671905</td>
      <td>inference</td>
      <td>to_numeric_downcast</td>
      <td>time_downcast</td>
    </tr>
    <tr>
      <th>199</th>
      <td>00:00:46.330012</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_describe</td>
    </tr>
    <tr>
      <th>559</th>
      <td>00:00:24.698904</td>
      <td>replace</td>
      <td>replace_large_dict</td>
      <td>time_replace_large_dict</td>
    </tr>
    <tr>
      <th>204</th>
      <td>00:00:24.212386</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_mad</td>
    </tr>
    <tr>
      <th>210</th>
      <td>00:00:22.481497</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_pct_change</td>
    </tr>
    <tr>
      <th>215</th>
      <td>00:00:18.909368</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_skew</td>
    </tr>
    <tr>
      <th>200</th>
      <td>00:00:18.732072</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_diff</td>
    </tr>
    <tr>
      <th>212</th>
      <td>00:00:18.317290</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_rank</td>
    </tr>
    <tr>
      <th>219</th>
      <td>00:00:16.845357</td>
      <td>groupby</td>
      <td>GroupBySuite</td>
      <td>time_unique</td>
    </tr>
  </tbody>
</table>

Ideally we could optimize the setup time on these as well. We could maybe modify the benchmark to do less work / run faster, but I'd like to avoid that if possible.

Link to the full CSV: https://gist.github.com/9d80aa45750224d7453863f2f754160d
