Deep network not able to learn imbalanced data beyond the dominant class
1) A five-layer neural network is one heck of a complex model for a data set with less than 1 million points. (I’m trying to find a good link for this, but the intuition is that your choice of model should be driven by the complexity of the available data, and not by what you think the real target function is like.) If this is for a real-world project, a tool like XGBoost might work better on this data set — out of the box, you’ll spend less time dealing with problems related to imbalanced classes, poorly scaled data, or outliers. Of course if this is specifically for learning about neural networks, that advice isn’t much help!

2) For a class distribution that’s as skewed as your data, you might get more mileage from re-sampling the training data rather than re-weighting the classes during training. Down-sample the majority class first (just throw away majority samples at random); if that’s not satisfactory then try something more complicated like using SMOTE to up-sample the minority classes. Try taking this to the extreme; build a (collection of) new training sets by randomly sampling only 1,000 points from each class.

The intuition here is that, for neural networks, as far as I know, re-weighting classes basically means re-scaling the gradient steps during training based on the class weights. If the classes are skewed 10:1, that makes sense: we take a step that’s 10 times as far for a minority sample. If the classes are skewed 1000:1, as in your case, it makes less sense — we’ll take 1,000 small steps as we optimize on the majority class, and then a single gigantic step in an essentially random direction when we happen to see a minority sample, followed by 1,000 tiny steps trying to un-do this work, etc. We don’t see enough minority samples to allow information about their class to average out.