Unable to do calculations with categorical dtypes
```
df = pd.DataFrame({'a':[1,2,3,2,1,3,4,5,6,7],
                   'b':[1,2,3,2,1,3,4,5,6,7]})
df['a'] = df['a'].astype('category')
df['a'] * df['b']
```
#### Problem description

On pandas v0.20.3, this is possible:

#### Expected Output
```
Out[23]: 
0     1
1     4
2     9
3     4
4     1
5     9
6    16
7    25
8    36
9    49
dtype: int64
```
#### On v0.23.0, the output is:
```
TypeError: Series cannot perform the operation *
```

</details>
