how to consider some miss classifications “half correct” using as base categorical_crossentropy - for a trading system I have a trading system where the model receives 9 time-series and predict :

A - strong down
B - week down
C - neutral 
D - week up
E - strong up


(these classes are generated from an histogram to have a balanced training dataset ... the histogram is separated in 20% parts of examples centered in 0... accuracy of 20% is the baseline) 

For each class I activate a different parametrized trading mechanism...  

My model is giving acceptable results. Here is the resulting confusion matrix for val data (28.99% acc):

[[32 20  3  6  8]
 [35 19  9  7 16]
 [30  9  6 14 24]
 [21 14  9 18 29]
 [ 9 14  3 14 45]]


My question starts here : 

I.e. If the model predicts B-"week down" but in reality is A-"strong down" is a miss, but in reality, it will make money...

So in this confusion matrix we can see that it happens 20 times (cell[0,1]) ... also if it is B but the model says A it will make money in 35 trades (cell[1,0])...

And also the same for the UP cases.. 

All together (from the confusion matrix) :
32+20+35+19  +  18+29+14+45 = 212 winning trades 
21+14+9+14  + 6+8+7+16 = 95 losing trades 

Assuming negative trades cancel in equal (in reality will not be equal..) positive trades, the total is = 117 winning trades. 

What I want is to create a loss function based on categorical_crossentropy 
 but somehow consider:


pred A real B - half miss
pred B real A - half miss
pred D real E - half miss
pred E real D - half miss


Do not penalize too much this cases. I think this will increase a bit the total number of positive trades. It will guide the learning a litle bit better (maybe not for accuracy but for a better loss that generates a better confusion matrix for profit )...

I have created a custom loss function that reduces 3% the loss for these cases: 

def my_loss(y_pred, y_true):

    y_pre_indexes = K.argmax(y_pred, axis=1) 
    y_test_indexes= K.argmax(y_true, axis=1)

    TN = K.tf.logical_or( K.tf.logical_or (K.tf.logical_and(K.equal(y_pre_indexes,0),K.equal(y_test_indexes,1)),
                          K.tf.logical_and(K.equal(y_pre_indexes,1),K.equal(y_test_indexes,0))) 
                        ,
                          K.tf.logical_or (K.tf.logical_and(K.equal(y_pre_indexes,3),K.equal(y_test_indexes,4)),
                          K.tf.logical_and(K.equal(y_pre_indexes,4),K.equal(y_test_indexes,3))))

    pos_neg = K.cast(TN, K.floatx()) *(-0.03) + 1 

    return K.categorical_crossentropy(y_pred, y_true)*pos_neg 


(in the code the classes are :  0-A 1-B 3-D 4-E.   2-C is predicting neutral - ignore..)

but fixing to a fixed number of 3% to reduce loss for these cases seems a little bit hard coded.... Something better inside the categorical_crossentropy math philosophy should be better.

Any suggestions? 

Thanks in advance!
I worked on a similar problem where classes were "ordered", so mis-classifications of class B as C was better than mis-classification as D.

In that project, "weighted quadratic kappa" loss function worked well. 

Details : 

https://en.wikipedia.org/wiki/Cohen%27s_kappa#Weighted_kappa

Example of Keras implementation : 

https://github.com/benhamner/Metrics/blob/master/Python/ml_metrics/quadratic_weighted_kappa.py