How to Work with Imbalanced Data I am building a binary classifier from a set of feature vectors some of which are categorical like Yes or No (two options). I am replacing them with 1 and 0 and since there is strong imbalance between 1 and 0 my model learns basically on those features. I am afraid that this particular feature might be an artifact - samples that have 0 can also sometimes be classified as 1. What to do in this situation? Should I drop the feature completely?

Update:

Let me explain in detail the problem since the answers mostly focus on imbalanced classes. I have a dataset that comprises of around 30 features and binary classes {0,1}. Features are mostly numerical (continuous), but there are also binary categorical features like YES/NO, MALE/FEMALE etc. 

One aspect of this dataset is imbalanced classes (there are more ones than zeros) and the other aspect is that one categorical feature, lets say $x$ (YES/NO) is not very balanced also. In fact if you make predictions solely based on $x$ like: $x = 1 \rightarrow 1, x=0 \rightarrow 0$ you will perform better than a naive model, which predicts only $0$ (remember imbalanced classes).

Now, my dilemma is what to do in that case? Should I remove that variable completely from modelling or maybe use some techniques of bias removal?
Generally, there are three main solutions for imbalance data classification.


Over-sampling the thin class by generating artificial data or some other ways. This solution can be helpful in situations in which generating accurate artificial data is possible.
Under-sampling the populous class. This solution also can be suitable for situations in which the ratio of classes population is not very low (for example 30% to 70% rather than 1% to 99%)
Using a proper loss function which can handle imbalanced data. There are many metrics that can evaluate the performance of the classifiers at the presence of imbalanced data.


Disclaimer:
If you use python, PyCM module can help you to find out these metrics.

Here is a simple code to get the recommended parameters from this module:

&gt;&gt;&gt; from pycm import *

&gt;&gt;&gt; cm = ConfusionMatrix(matrix={"Class1": {"Class1": 1, "Class2":2}, "Class2": {"Class1": 0, "Class2": 5}})  

&gt;&gt;&gt; print(cm.recommended_list)
["Kappa", "SOA1(Landis &amp; Koch)", "SOA2(Fleiss)", "SOA3(Altman)", "SOA4(Cicchetti)", "CEN", "MCEN", "MCC", "J", "Overall J", "Overall MCC", "Overall CEN", "Overall MCEN", "AUC", "AUCI", "G", "DP", "DPI", "GI"]


After that, each of these parameters you want to use as the loss function can be used as follows:

&gt;&gt;&gt; y_pred = model.predict      #the prediction of the implemented model

&gt;&gt;&gt; y_actu = data.target        #data labels

&gt;&gt;&gt; cm = ConfusionMatrix(y_actu, y_pred)

&gt;&gt;&gt; loss = cm.Kappa             #or any other parameter (Example: cm.SOA1)