First pass at a vbsuite results review
So, now that vbsuite is more stable, and collecting that data is much faster
due to the new build cache and perf tweaks to test_perf, I've set up a job to 
gradually sample the commit history perf profile. 
Even with the faster iterations, It'll still take a few days to collect the data
(I'm going back to 0.9.0), but here are some results from 300 uniform sample 
points. (This is all ad-hoc and might contain errors, I'm working on some new
tools to make this easier).

Jeff, this one should be familiar :)  : 

``` python
In [304]: pd.options.display.mpl_style='default'
```

![perf](https://f.cloud.github.com/assets/1820866/346747/a061c45a-9e5c-11e2-8ce0-1b09c98fa4a1.png)

There are also several regression,again, it'll take a few days to collect
the full dta and then bisect to the changepoint commit.

Here are are all vbenches at least 20% slower today, compared to
0.9.0:

```
reindex_fillna_backfill                 1.208748
write_store_table                       1.209075
frame_drop_dup_na_inplace               1.209770
replace_fillna                          1.221003
replace_replacena                       1.244413
stat_ops_level_frame_sum_multiple       1.248258
misc_cache_readonly                     1.250000
stat_ops_level_series_sum_multiple      1.269106
write_store_table_panel                 1.298524
groupby_apply_dict_return               1.304015
read_store_table_mixed                  1.348121
series_constructor_ndarray              1.359551
append_frame_single_mixed               1.370398
frame_multi_and_no_ne                   1.372070
write_store_table_wide                  1.461807
reindex_frame_level_reindex             1.532964
reindex_frame_level_align               1.537642
stat_ops_series_std                     1.566043
series_getitem_scalar                   1.800000
ctor_index_array_string                 1.873016
write_store_table_dc                    8.468982
```

some graphs:

![ctor_index_array_string](https://f.cloud.github.com/assets/1820866/346758/7d47b338-9e5e-11e2-9021-b4edef55ff01.png)

![write_store_table_dc](https://f.cloud.github.com/assets/1820866/346759/81232a00-9e5e-11e2-82b0-f35f68f9996f.png)

![frame_multi_no_ne](https://f.cloud.github.com/assets/1820866/346760/88be7bfc-9e5e-11e2-83f4-8ea206dfe6fd.png)

The spikes are probably due to git log having strange commit ordering when merges are involved, 
working on it.

collect.sh

```
#!/bin/bash
function gho
{
    git show-ref --heads --abbrev=8 $1 | head -n 1
}

function gch {
    gho $(git rev-parse --abbrev-ref HEAD) | cut -d ' ' -f 1 # show curerent branch head
}
git fetch upstream
git reset --hard upstream/master

gch > .head;
./hash_stab.py | while read N; do
   git reset --hard $N
   FNAME="vbench_results/$N.pickle"

   if [  -e  $FNAME ]; then
        echo skipping $N
   else
   ./sane.sh # copy over the setup.py and vb_suite/* from upstream/HEAD
   python ./setup.py develop;
        time ./test_perf.sh -H -c 1 -d $FNAME;
   fi;
done

```

hash_stab.py

```
#!/usr/bin/env python
import sh
r=sh.git('log --format="%h" v0.9.0.. --'.split(" "),_tty_out=False).replace('"',"").split("\n")[:-1]
print("%d commits" % len(r))
l = [r[0],r[-1]]
segsize = len(r)

while len(l) < len(r):
    cands = range(0+segsize/2,len(r),segsize)
    cands = [r[i] for i in cands if r[i] and r[i] not in l]
    segsize/=2
    if len(cands) == 0:
        l.extend([x for x in r if x not in l])
        break
    l.extend(cands)

for h in l:
    print h

```

analyze results:

``` python
h=sh.git('log --date-order --format="%h###%ci" --no-merges'.split(" ")).replace('"',"").split("\n")[:-1]
h=map(lambda x:x.split("###"),h)
from dateutil import parser
h=dict(map(lambda x: (x[0],parser.parse(x[1])),h))
fs = [x for x in os.listdir('.') if x.split('.')[0] in h]
fs.sort(key=lambda x: h[x.split('.')[0]])
results = list()
for o in fs:
    s = Series(pd.load(o).icol(0),name=o.split(".")[0])
    results.append(s)
df=pd.DataFrame(results).T
f=df.icol(-1)/df.icol(0)
f=f[f> 1.2]
def show(n):
    if isinstance(n,basestring):
        s=df.ix[n]
    else:
        s=df.ix[f.index[n]]
    name = s.name
    s=mom.rolling_mean(s,5,center=True).dropna()
    s.name=name
    plt.title(s.name)
    plt.ylim(0,s.max())
    s.plot()
    s=s.dropna()
    return name

show(3)
# plot here
show('dti_reset_index_tz')
# plot here
```
