Perf regressions in 0.13.1
Ive started work on the release [checklist](https://github.com/pydata/pandas/wiki/Release-Checklist), here's the vbench run against v0.13.0 (>30%):

```
indexing_dataframe_boolean_rows              |   0.5086 |   0.3866 |   1.3155 |
groupby_transform                            | 265.5996 | 194.6437 |   1.3645 |
query_datetime_series                        |  52.5440 |  33.2900 |   1.5784 |
reshape_stack_simple                         |   4.7570 |   2.9666 |   1.6035 |
indexing_panel_subset                        |   1.3873 |   0.7103 |   1.9530 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Target [cef41bf] : potentialy 0.13.1
Base   [a5410ed] : RLS: 0.13.0 final
```

A more detailed look, with summary stats over 10 runs for each vb to
supress noise, the quantiles suggest at least some of these are real.

```
 ./test_perf.sh -H -u 3 -S -n 5 -N 10 -c 3  -r 'indexing_panel_subset|reshape_stack_simple|query_datetime_series|groupby_transform|indexing_dataframe_boolean_rows'  | tee log 

Target [a5410ed] : RLS: 0.13.0 final

                                        count        mean       std         min         25%         50%         75%         max
groupby_transform                          10  202.459224  1.248072  201.371670  201.713940  201.888839  202.670515  205.056349
indexing_dataframe_boolean_rows            10    0.385046  0.001513    0.382344    0.384192    0.384967    0.385841    0.388066
indexing_dataframe_boolean_rows_object     10    0.633168  0.001114    0.631332    0.632445    0.633637    0.633697    0.634670
indexing_panel_subset                      10    0.709184  0.001551    0.706275    0.708421    0.709136    0.710249    0.711600
reshape_stack_simple                       10    2.804049  0.010563    2.785047    2.800802    2.803524    2.813021    2.818028
query_datetime_series                      10   35.682742  0.926633   35.169999   35.275022   35.349131   35.618087   38.246711

Target [cef41bf] : potentialy 0.13.1

                                        count        mean       std         min         25%         50%         75%         max
groupby_transform                          10  271.738656  0.326605  271.227598  271.522979  271.702965  271.987200  272.217353
indexing_dataframe_boolean_rows            10    0.488933  0.002478    0.485341    0.488122    0.488679    0.489871    0.494321
indexing_dataframe_boolean_rows_object     10    0.739876  0.003478    0.735362    0.737846    0.739853    0.740846    0.747681
indexing_panel_subset                      10    1.404691  0.005823    1.394351    1.401464    1.405795    1.408557    1.412312
reshape_stack_simple                       10    4.802346  0.015722    4.780690    4.792313    4.796505    4.812539    4.831711
query_datetime_series                      10   51.154764  0.140856   51.050027   51.059425   51.092466   51.194866   51.424026
```

https://github.com/pydata/pandas/issues/6103
cc @jreback 
