Tensorflow classification - maximize the accuracy of certain classes I'm doing some experimentation and trying to train a forex trading model to classify based on three classes:


Buy
Sell
No action


Input rows are labeled as buy when the price at some later time is X% greater than the current price and labeled as sell when the future price is X% less than the current price.  All other rows are labeled as no action, which is the large majority of rows.

The problem I'm running into is that the no action items represent about 84% of the total dataset, however the model is predicting no action almost 99% of the time.  I suppose the reason for this is that the model is aiming for maximum accuracy and it doesn't realize that no action items are of no use.

So what I would like, if possible, is for the model to only care about buy and sell items in trying to maximize accuracy to hopefully increase the number of buy and sell predictions (to the extent that this helps the profitability of the model).

One way that I think I could do this is to increase the number of classes and have each class represent a roughly equal percentage of the total items, however, that would be pretty messy, so I'm curious if there's a way that directly affects the way accuracy is calculated.



EDIT:  I've added additional details here: Variability in CNN test results

But the focus is still in maximizing the accuracy of the buy/sell categories.  Looking through the Tensorflow metrics (https://www.tensorflow.org/api_docs/python/tf/metrics), I think I could use two separate binary classifiers and use the TruePositives metric, however, I'd prefer to keep things simple and get it all in one model.
The best approach for this problem would be to change the value of X. For example, if you are using -%2 and %2 as the classification limit such as  sell(&lt;-%2), buy(>%2) and no action otherwise; you can reduce this to %1, which will in turn reduce the number of samples that fall into this class while increasing number of samples in buy and sell classes.

If you want to keep that value, the way to go is undersampling: when training you take equal number of samples from each class, meaning you ignore some portion of the data from no action class. So you training and validation sets will be balanced and you can use accuracy as a proper metric.

If this also is not a good option for you, another way would be to try changing the classification threshold for each output so that their possible outcomes are roughly equal. But this might be a little complicated to implement as you have 3 classes (for 2 classes this can be done in a relatively simple way by changing classification threshold from 0.5).

If none of these suit you, you can find a more general approach to imbalance problems here. In case you wish to use a different metric, the most popular ones are listed and discussed in detail here. You probably don't need to define a custom metric because there is a one for pretty much every choice.