Is my model over fitting or not?
I have less than enough karma to leave a comment, but I'd like to support Derek O's assessment but also add 1 more point: if your 50K observations are repeated measures (multiple rows coming from the same individual or unit) - then you'll want to make sure that in your cross-fold setup that you are ensuring that 100% of each individual's observations are falling into the same fold.
An analogy would be that if you were modeling teeth for a probability of developing a cavity - a human head has 32 teeth, and if a particular human's teeth end up in both your testing and training folds, this would be considered a form of leakage. This is because the 32 teeth are not totally independent, but correlated with other teeth within the same head. This particular form of leakage often slips people's minds.