Class Imbalence Problem even after Balancing Data So I am training a neural network on a binary classification problem and my Case (1) and Controls (0) were imbalanced so I oversampled my cases so that that the training set was 0.5053 made up of controls. I did not balance my test set which was 0.562 controls. In the beginning my train and test accuracy rises (it is not very accurate still but I expect this to be the case) but then the training accuracy steeply drops while the test accuracy plateaus.

They end up with accuracies of 0.5053 and 0.562 respectively so the network is just classifying everything the same. I do not understand how this behavior arises as I thought that balancing my training set would avoid the problem of classifying everything the same? Also, the training set begins to learn initially upwards from 50/50 but I cannot understand its reversion. Is there anything I can do to prevent this? Or should I just employ early stopping when the training accuracy begins to decrease?

Any insight would be appreciated!

opt = tf.keras.optimizers.SGD(lr=0.000001, momentum=0.9, decay=0, nesterov=True) 

model = keras.Sequential([keras.layers.Dense(100,kernel_initializer='he_uniform',bias_initializer=keras.initializers.Constant(value=0.01),activation=tf.nn.relu,kernel_regularizer=regularizers.l2(0.1)), keras.layers.Dense(100,kernel_initializer='he_uniform',bias_initializer=keras.initializers.Constant(value=0.01),activation=tf.nn.relu, kernel_regularizer=regularizers.l2(0.1)), keras.layers.Dense(1, activation=tf.nn.sigmoid)]) 

model.compile(optimizer=opt, loss='binary_crossentropy', metrics=['accuracy'])
The fact tha balancing the dataset Will prevent the overfitting and thus, good results in the test set, Works under the assumption that the model does not underfit. 

What it may be happening is that the model is too bad that it can't perform the classification task in either the un-balanced and the balanced dataset. 

It's difficult to analyse the problema without more information, you may have facing convergence issues (too large learning rate, vanishing gradients), you may also be computing the accuracy in a bad way...