How to avoid overfitting in deep learning when features are binary in nature
I am constructing a deep learning model using 2048 bits of binary fingerprints (0 and 1's) for some 2000 samples to predict their outputs (positive (1) OR negative(0)). The feature data is quite sparse i.e. lots of zeros and rare 1's.
I have used 'binary cross entropy' but my validation accuracy doesn't increase more than 70%. I have balanced data. The model seems to be overfitting. I can't normalize my data since fetures are binary. How can I avoid overfitting?
    earlystop = EarlyStopping(monitor='val_acc', patience=20, mode='max')
    final_model = Sequential()
    final_model.add(Dense(1012, input_dim=2048, activation = 'relu'))
    final_model.add(BatchNormalization())
    final_model.add(Dropout(0.9))
    final_model.add(Dense(512, activation='relu'))
    final_model.add(BatchNormalization())
    final_model.add(Dropout(0.9))
    final_model.add(Dense(128, activation='relu'))
    final_model.add(BatchNormalization())
    final_model.add(Dropout(0.5))
    final_model.add(Dense(32, activation='relu'))
    final_model.add(Dense(1, activation='sigmoid'))
    adam_opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
    final_model.compile(loss='binary_crossentropy', optimizer=adam_opt, metrics=['accuracy'])


If you want to do a binary classification, binary crossentropy is the loss function you are looking for.
Achieving a well generalizing model includes more than just the right loss function choice (Preprocessing Data, Finding a proper Network Architecture, Finding the right hyper parameter choice, ...).
You can find a discussion about generalization of Deep Learning Models here:
https://stats.stackexchange.com/questions/365778/what-should-i-do-when-my-neural-network-doesnt-generalize-well
