title: Multi-Class Classification: SMOTE oversampling for multiple columns in a row
body: <p>I have an imbalanced dataset contained in a dataframe called <em>city_country</em> that is made up of 5 columns:</p>
  
  <ol>
  <li>Content of a tweet = <strong>preprocessed</strong></li>
  <li>An event type (e.g. tweet relates to earthquake = 'earthquake', typhoon = 'typhoon', etc.) = <strong>event_type</strong></li>
  <li>Latitude of where the tweet was sent = <strong>lat</strong></li>
  <li>Longitude of where the tweet was sent = <strong>long</strong></li>
  <li>An event label (e.g. tweet relates to earthquake = 1, typhoon = 2, etc.) = <strong>event_id</strong></li>
  </ol>
  
  <p>In the dataframe called <em>city_country</em>, the class (<strong>event_id</strong>) is imbalanced. Before testing the predictive power of different text classifiers, to predict the <strong>event_id</strong> from the content of a tweet (<strong>preprocessed)</strong>, I want to oversample the minority classes.</p>
  
  <p>It is important that when I duplicate the entries that belong to minority classes that I duplicate all 5 columns.</p>
  
  <p>What I have done so far (incorrectly) is only oversample the tweet content (<strong>preprocessed</strong> and the <strong>event_id</strong>. In the below code, I convert the tweets into vectors (which I do not want to do but to my knowledge, I have to) and then overrepresent minority classes. This only oversamples the vectorised tweet (<em>x_words</em>) and the event_id (<em>y</em>). </p>
  
  <pre><code>tfidf_words = TfidfVectorizer(sublinear_tf=True, min_df=0, norm='l2', encoding='latin-1', ngram_range=(1,1), stop_words='english')
  
  x_words = tfidf_words.fit_transform(city_country.preprocessed).toarray()
  
  # new dataframe 'label' that contains the event_id for each preprocessed tweet
  y = city_country.event_id
  
  x_train_words, x_test_words, y_train, y_test = train_test_split(x_words, y, test_size = 0.25, random_state = 0)
  
  # Use SMOTE to oversample the minority classes
  from imblearn.over_sampling import SMOTE
  sm = SMOTE(random_state=12)
  x_train_words_sm, y_train_words_sm = sm.fit_sample(x_train_words, y_train)
  
  # Count the number of occurences in the y_train sets to ensure that the oversampling worked 
  from collections import Counter
  class_check_woSMOTE = Counter(y_train)
  class_check_words = Counter(y_train_words_sm)
  </code></pre>
  
  <p>To my knowledge, the use of <em>SMOTE</em> in <em>imblearn.over_sampling</em> requires you to feed real values (not strings) and only 2 values - an 'x' and a 'y'. In this case, 'x' is my vectorised training set of tweets and 'y' is my event label.</p>
  
  <p>Is there a way for me to simply split my dataframe into training and test sets, then oversample all 5 columns for the minority classes, so that the output is a larger dataframe that contains all 5 columns? I can then use this to predict the event_id and hopefully perform the equivalent of a vlookup, so that I can join the tweet with its respective <em>lat</em> and <em>long</em> values. </p>