Neural Networks: Predicting probabilities of the possible values of y, instead of just predicting y I have a true value y that I'd like to predict with a regression, but I'm interested in the probabilities that y will be different values. Y is theoretically continuous but in the dataset it is rounded to integers. Let's say y could be 0-9. I want 10 probabilities, one for each possible value. I tried doing this categorically, with a neural network having 10 output nodes, this hurts the predictions since we lose the relationships between categories, 1 is closer to 2 than it is to 10.

Example toy problem: 

y is the weight of an object in pounds. The dataset has Y values rounded s.t. y can be from 0 to 9 pounds. Predict the probabilities that y will be 0-9 pounds based on the features X.

Example output: [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1,] (ten # summing to 1)

I'd like to be able to accomplish this with Keras.
Here are two approaches that use the idea of havng your prediction be a distribution over your continuous variables rather than a single value.

"Easy" but computationally expensive

Independently train K regressors.  You now have K samples from a distribution, and you can

Big downside - you have to train lots of networks, and run lots of networks at inference time.

For some details "Confidence and prediction intervals for neural network ensembles"

Harder

Have one network explicitly predict a parametrized (e.g. Gaussian/ mixture of gaussians) distribution.  For details see "Mixture density network", for example

So in the case of a single Gaussian, the network would output a mean and standard deviation, and so you have an estimate of the uncertainty.

A possible danger here is that you might need to be clever about how to train the variance parameter.  You could imagine a case where a network does a good job on the means but is either over- or under- confident in its predictions.