title: data preparation for random forest and predictive modeling in python
body: <p>I am working on a predictive modeling exercise using a categorical output (pass/fail: binary 1 or 0) and about 200 features. I have about 350K training examples for this, but I can increase the size of my dataset if needed. Here are a few issues that I running into:</p>
  
  <p>1-  I am dealing with <strong>severely imbalanced classes</strong>. Out of those 350K examples, only 2K are labelled as “fail” (i.e. categorical output = 1). How do I account for this? I know there are several techniques, such as up-sampling with bootstrap;  </p>
  
  <p>2- <strong>Most of my features (~ 95%) are categorical</strong> (e.g. city, language, etc.) with less than 5-6 levels each. Do I need to transform them into binary data for each level of a feature? For instance if the feature “city” has 3 levels with New York, Paris, and Barcelona, then I can transform it into 3 binary features: city_New_york, city_Paris, and city_Barcelona; </p>
  
  <p>3 - <strong>Picking the model</strong> itself: I am thinking about a few such as SVM, K-neighbors, Decision tree, Random Forest, Logistic Regression, but my guess is that Random Forest will be appropriate for this because of the large number of categorical features. Any suggestions there? </p>
  
  <p>4 - <strong>If I use Random Forest</strong>, do I need to (a) do feature scaling for the continuous variables (I am guessing not), (b) change my continuous variables to binary, as explained in question 2 above (I am guessing not), (c) account for my severe imbalanced classes, (d) remove missing values.</p>
  
  <p>Thanks in advance for your answers!</p>