Neural Networks - Strategies for problems with high Bayes error rate
You have a high Bayes error rate and it means that you almost can not learn anything. You have to add extra features and investigate whether your data has a small Bayes error or not. Currently, it is worse than a disaster. This large Bayes error illustrates that you have patterns, input vectors, that have completely same components as each feature but different labels. Take a look at here.