Test accuracy of neural net is going up and down
Based on the image you are sharing, the training accuracy continues to increase, the validation accuracy is changing around the 50%. I think either you do not have enough data to use neural network or the network is small to capture all the information, in both cases I feel there either under fitting or over fitting problem. Can you give us some information about the database so we can help more