groupby.agg should refuse non-aggregation operations
Since `groupby.agg()` is, ["by definition a reducer."](https://github.com/pandas-dev/pandas/issues/14741#issuecomment-262953242) (in other words, produces one value per series in a group), it seems odd that it accepts non-aggregating operations such as 'rank':

Related to https://github.com/pandas-dev/pandas/issues/22509#issuecomment-500103115 where `groupby.transform('rank')` applies the wrong code path to 'rank'
because it is treated as an aggregator (array->scalar) instead of a shape-preserving operation. 

```python
In [82]: df=pd.DataFrame(dict(price=[1,2,2,3,3,3],cost=(1,2,1,3,2,1)))
    ...: df
Out[82]: 
   price  cost
0      1     1
1      2     2
2      2     1
3      3     3
4      3     2
5      3     1

In [83]: g=df.groupby('price')
    ...: g.agg('rank').astype(int)
Out[83]: 
   cost
0     1
1     2
2     1
3     3
4     2
5     1```
