Applying bayesian methods to a simple neural network This is a really simple neural network with backprop. If one had to apply bayesian "inferences" to update the weights and biases, what would change in the code.

#Forward Propogation
hidden_layer_input1=np.dot(X,wh)
hidden_layer_input=hidden_layer_input1 + bh               # linear transformation
hiddenlayer_activations = sigmoid(hidden_layer_input)     # non-linear transformation
output_layer_input1=np.dot(hiddenlayer_activations,wout)
output_layer_input= output_layer_input1+ bout             # linear transformation
output = sigmoid(output_layer_input)

#Backpropagation
E = y-output
slope_output_layer = derivatives_sigmoid(output)
slope_hidden_layer = derivatives_sigmoid(hiddenlayer_activations)
derivative_output = E * slope_output_layer
Error_at_hidden_layer = derivative_output.dot(wout.T)
derivative_hiddenlayer = Error_at_hidden_layer * slope_hidden_layer
wout += hiddenlayer_activations.T.dot(d_output) *lr
bout += np.sum(derivative_output, axis=0,keepdims=True) *lr
wh += X.T.dot(d_hiddenlayer) *lr                          # update weight
bh += np.sum(d_hiddenlayer, axis=0,keepdims=True) *lr     # update bias


If my understanding is correct of using bayes method to derive the weights and biases, it would help in getting to those values faster, leaving behind unnecessary loops for weights which would not converge. 

Apart from that, coding and observing it is the best bet to get a real perception.

Would like an understanding.
Bayesian models specify priors to inform and constrain the models and get uncertainty estimation in form of a posterior distribution. Non-Bayesian Deep Learning computes a scalar value for weights and biases at each layer. Bayesian Deep Learning calculates a posterior distribution of weights and biases at each layer which better estimates uncertainty but increases computational cost. 

Edward is a Python package for Bayesian inference, including Deep Learning.