How to make a neural network generalizes better? I designed a neural network model with large number of output predicted by softmax function. However, I want categorize all the outputs into 5 outputs without modifying the architecture of other layers. The model performs well in the first case but when I decrease the number of output it loses accuracy and get a bad generalization. My question is: Is there a method to make my model performs well even if there is just 5 outputs? for example: adding dropout layer before output layer, using other activation function, etc.
First make sure your dataset is labelled properly in 5 distinct classes from 0 to 4 and no more.
Then add dropout layer in between each layer with prob of 0.1 and gradually increase till 0.5 until you find  a good val score.
In your optimizer add weight decay term with value around 0.1.
Find some appropriate regularizer for your data type online.