Classification - Divide the interval (0 - 1] to lets say 100 classes and use each class to make a calculation class-1 represents 0.01, class-i represents 0.01*i, class-100 represents 1.00.

Thus, when the classifier predicts the class-y and it should have predicted class-(y+1) there is a small error so we can accept class-y. 

Is there a way to express this behaviour in a neural network? Maybe with a distribution or something?

PS: Not interested in regression.
Correct me if I am wrong but If I understand your question correctly, what you want is a classifier such that classes close to each other (say class 2 and class 3) are preferable to those far away (class 2 and class 99). If this is the case, this problem is called "Ordinal Categorical Classification". 

I was working on a similar problem a while ago, and I found this loss function during my research. I ended up not using it so I don't really know how good it works but anyways, hope that helps.