title: get_dummies python memory error
body: <p>I'm having a problem with a data set that has 400,000 rows and 300 variables. I have to get dummy variables for a categorical variable with 3,000+ different items. At the end I want to end up with a data set with 3,300 variables or features so that I can train a RandomForest model.</p>
  
  <p>Here is what I've tried to do:</p>
  
  <pre><code> df = pd.concat([df, pd.get_dummies(df['itemID'],prefix = 'itemID_')], axis=1)
  </code></pre>
  
  <p>When I do that I'll always get an memory error. Is there a limit to the number of variables i can have?</p>
  
  <p>If I do that with only the first 1,000 rows (which got 374 different categories) it just works fine.</p>
  
  <p>Does anyone have a solution for my problem?  The computer I'm using has 8 GB of memory.</p>