title: pivot_table requires more memory if dtype is category (MemoryError)
body: <p>I have the following strange error with pandas(pandas==0.23.1) :</p>
  
  <pre><code>import pandas as pd
  df = pd.DataFrame({'t1': ["a","b","c"]*10000, 't2': ["x","y","z"]*10000, 'i1': list(range(5000))*6, 'i2': list(range(5000))*6, 'dummy':0})
  # works fast with less memory
  piv = df.pivot_table(values='dummy', index=['i1','i2'], columns=['t1','t2'])
  
  d2 = df.copy()
  d2.t1 = d2.t1.astype('category')
  d2.t2 = d2.t2.astype('category')
  
  # needs &gt; 20GB of memory and takes for ever
  piv2 = d2.pivot_table(values='dummy', index=['i1','i2'], columns=['t1','t2'])
  </code></pre>
  
  <p>I am wondering if this is expected and I am doing something wrong, or if this is a bug in pandas. Should dtype <code>category</code> for <code>str</code> not be very transparent (for this use case)?</p>