Is my model over fitting or not? I have 50000 observations with 70% positive and 30% negative target variable. I'm getting accuracy of around 96-99% which seems unreal of course and I'm worried that my model is over-fitting which I don't understand why. I replaced all outliers with 5th and 95th quantile. Standardized the data yet it is showing this unreal accuracy. 
A bit of online search and people suggested to check for the difference between training and test data accuracy, which i did and for Random Forest it came

Training Accuracy: 0.997975
Test Accuracy: 0.9715


For logistic regression it shows 

Training Accuracy: 0.967225
Test Accuracy: 0.9647


This is the code I used for running the model:

clf = LogisticRegression()
trained_model = clf.fit(X_train, y_train)
trained_model.fit(X_train, y_train)
predictions = trained_model.predict(X_test)      

accuracy_score(y_train, trained_model.predict(X_train))
accuracy_score(y_test, predictions)


I also tried kfold cross validation which gave similar results 

skfold = StratifiedKFold(n_splits=10, random_state=100)
model_skfold = LogisticRegression()
results_skfold = model_selection.cross_val_score(model_skfold, X, Y, cv=skfold)
print("Accuracy: %.2f%%" % (results_skfold.mean()*100.0))


Lastly I applied regularization technique to check for results and this is the result  I got

for c in C:
    clf = LogisticRegression(penalty='l1', C=c, solver='liblinear')
    clf.fit(X_train, y_train)
    y_pred_log_reg = clf.predict(X_test)
    acc_log_reg = round( clf.score(X_train, y_train) * 100, 2)
    print (str(acc_log_reg) + ' percent')
    print('C:', c)
    print('Coefficient of each feature:', clf.coef_)
    print('Training accuracy:', clf.score(X_train_std, y_train))
    print('Test accuracy:', clf.score(X_test_std, y_test))
    print('')


The results

96.72 percent
C: 10
Coefficient of each feature: [[-2.50e+00 -1.40e-03  2.65e+00  4.09e-02 -2.03e-03  2.75e-04  1.79e-02
  -2.13e-03 -2.18e-03  2.90e-03  2.69e-03 -4.93e+00 -4.89e+00 -4.88e+00
  -3.27e+00 -3.30e+00]]
Training accuracy: 0.5062
Test accuracy: 0.5027

96.72 percent
C: 1
Coefficient of each feature: [[-2.50e+00 -1.41e-03  2.66e+00  4.10e-02 -2.04e-03  2.39e-04  1.68e-02
  -3.29e-03 -3.80e-03  2.52e-03  2.62e-03 -4.22e-02 -9.55e-03  0.00e+00
  -1.73e+00 -1.77e+00]]
Training accuracy: 0.482525
Test accuracy: 0.4738

96.74 percent
C: 0.1
Coefficient of each feature: [[-2.46e+00 -1.38e-03  2.58e+00  4.03e-02 -1.99e-03  2.22e-04  1.44e-02
  -4.49e-03 -5.13e-03  2.03e-03  2.20e-03  0.00e+00  0.00e+00  0.00e+00
   0.00e+00 -6.54e-03]]
Training accuracy: 0.616675
Test accuracy: 0.6171

95.92 percent
C: 0.001
Coefficient of each feature: [[-1.43e+00 -6.82e-04  1.19e+00  2.73e-02 -1.10e-03  1.22e-04  0.00e+00
  -2.74e-03 -2.55e-03  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00
   0.00e+00  0.00e+00]]
Training accuracy: 0.655075
Test accuracy: 0.6565


The codes I used for Standardization and replace outliers

std_scale = preprocessing.StandardScaler().fit(X_train)
X_train_std = std_scale.transform(X_train)
X_test_std  = std_scale.transform(X_test)

X.clip(lower=X.quantile(0.05), upper=X.quantile(0.95), axis = 1, inplace = True)


Do let me know if any other information is required and any guidance will be appreciated
I have less than enough karma to leave a comment, but I'd like to support Derek O's assessment but also add 1 more point: if your 50K observations are repeated measures (multiple rows coming from the same individual or unit) - then you'll want to make sure that in your cross-fold setup that you are ensuring that 100% of each individual's observations are falling into the same fold.
An analogy would be that if you were modeling teeth for a probability of developing a cavity - a human head has 32 teeth, and if a particular human's teeth end up in both your testing and training folds, this would be considered a form of leakage. This is because the 32 teeth are not totally independent, but correlated with other teeth within the same head. This particular form of leakage often slips people's minds.