I have the following code, where x in the input data, w is the weight matrix, bv and bh are the biases for the visible and hidden units.import theano.tensor as Tx_states = x > numpy.random.rand(training_examples, feats)hid =  T.nnet.sigmoid(T.dot(x_states, w) + bh)    # Activation function of hidden layerhid_states = hid > numpy.random.rand(training_examples, nhidden)# Construct Theano expression graphvis = T.nnet.sigmoid(T.dot(hid_states, w.T) + by) vis_states = vis > numpy.random.rand(training_examples, feats)hid2 = T.nnet.sigmoid(T.dot(vis_states, w) + bh)hid2_states = hid2 > numpy.random.rand(training_examples, nhidden)xent = T.sum((x - vis)**2)parameters = [w, bh, by]  # this line defines all parameters for each layercost = xent.mean()pos_associations = T.dot(x.T, hid)neg_associations = T.dot(vis.T, hid2)  wparameters = [w]byparameters = [by]bhparameters = [bh]update = []for wparam, byparam, bhparam in zip(wparameters, byparameters, bhparameters):    update.append((wparam, wparam + tr_rate * (pos_associations - neg_associations)))    update.append((byparam, byparam + tr_rate*(T.sum(x.T[:]) - T.sum(vis.T[:]))))          update.append((bhparam, bhparam + tr_rate*(T.sum(hid.T[:]) - T.sum(hid2.T[:]))))train = theano.function(          inputs=[x],          outputs=[cost],          updates=update)What is the correct implementation for the update? In my example this is clearly not working. I tested this on the classic MNIST dataset.This is the result I get for the cost. As you can see it does not decreases. What is wrong?Epoch: 0cost =  [ 969572.73014003]Epoch: 1cost =  [ 258872.77507019]Epoch: 2cost =  [ 258872.77507019]Epoch: 3cost =  [ 258872.77507019]Epoch: 4cost =  [ 258872.77507019]Epoch: 5cost =  [ 2003326.79850769]Epoch: 6cost =  [ 258872.77507019]Epoch: 7cost =  [ 258872.77507019]Epoch: 8cost =  [ 258872.77507019]Epoch: 9cost =  [ 258872.77507019]Epoch: 10cost =  [ 258872.77507019]Epoch: 11cost =  [ 258872.77507019]Epoch: 12cost =  [ 2003326.79850769]Epoch: 13cost =  [ 258872.77507019]Epoch: 14cost =  [ 258872.77507019]Epoch: 15cost =  [ 258872.77507019]Epoch: 16cost =  [ 258872.77507019]Epoch: 17cost =  [ 258872.77507019]Epoch: 18cost =  [ 258872.77507019]Epoch: 19cost =  [ 258872.77507019]Epoch: 20cost =  [ 2003326.79850769]Epoch: 21cost =  [ 258872.77507019]Epoch: 22cost =  [ 258872.77507019]Epoch: 23cost =  [ 258872.77507019]Epoch: 24cost =  [ 258872.77507019]Epoch: 25cost =  [ 258872.77507019]Epoch: 26cost =  [ 258872.77507019]Epoch: 27cost =  [ 258872.77507019]Epoch: 28cost =  [ 2003326.79850769]Epoch: 29cost =  [ 258872.77507019]Epoch: 30cost =  [ 258872.77507019]Epoch: 31cost =  [ 258872.77507019]Epoch: 32cost =  [ 258872.77507019]Epoch: 33cost =  [ 258872.77507019]Epoch: 34cost =  [ 258872.77507019]Epoch: 35cost =  [ 258872.77507019]Epoch: 36cost =  [ 2003326.79850769]Epoch: 37cost =  [ 258872.77507019]Epoch: 38cost =  [ 258872.77507019]Epoch: 39cost =  [ 258872.77507019]Epoch: 40cost =  [ 258872.77507019]Epoch: 41cost =  [ 258872.77507019]Epoch: 42cost =  [ 258872.77507019]Epoch: 43cost =  [ 2003326.79850769]Epoch: 44cost =  [ 258872.77507019]Epoch: 45cost =  [ 258872.77507019]Epoch: 46cost =  [ 258872.77507019]Epoch: 47cost =  [ 258872.77507019]Epoch: 48cost =  [ 258872.77507019]Epoch: 49cost =  [ 258872.77507019]Epoch: 50cost =  [ 258872.77507019]Epoch: 51cost =  [ 2003326.79850769]Epoch: 52cost =  [ 258872.77507019]Epoch: 53cost =  [ 258872.77507019]Epoch: 54cost =  [ 258872.77507019]Epoch: 55cost =  [ 258872.77507019]Epoch: 56cost =  [ 258872.77507019]Epoch: 57cost =  [ 258872.77507019]Epoch: 58cost =  [ 258872.77507019]Epoch: 59cost =  [ 2003326.79850769]Epoch: 60cost =  [ 258872.77507019]Epoch: 61cost =  [ 258872.77507019]Epoch: 62cost =  [ 258872.77507019]Epoch: 63cost =  [ 258872.77507019]Epoch: 64cost =  [ 258872.77507019]Epoch: 65cost =  [ 258872.77507019]Epoch: 66cost =  [ 258872.77507019]Epoch: 67cost =  [ 2003326.79850769]Epoch: 68cost =  [ 258872.77507019]Epoch: 69cost =  [ 258872.77507019]Epoch: 70cost =  [ 258872.77507019]Epoch: 71cost =  [ 258872.77507019]Epoch: 72cost =  [ 258872.77507019]Epoch: 73cost =  [ 258872.77507019]Epoch: 74cost =  [ 2003326.79850769]Epoch: 75cost =  [ 258872.77507019]Epoch: 76cost =  [ 258872.77507019]Epoch: 77cost =  [ 258872.77507019]Epoch: 78cost =  [ 258872.77507019]Epoch: 79cost =  [ 258872.77507019]Epoch: 80cost =  [ 258872.77507019]Epoch: 81cost =  [ 258872.77507019]Epoch: 82cost =  [ 2003326.79850769]Epoch: 83cost =  [ 258872.77507019]Epoch: 84cost =  [ 258872.77507019]Epoch: 85cost =  [ 258872.77507019]Epoch: 86cost =  [ 258872.77507019]Epoch: 87cost =  [ 258872.77507019]Epoch: 88cost =  [ 258872.77507019]Epoch: 89cost =  [ 258872.77507019]Epoch: 90cost =  [ 2003326.79850769]Epoch: 91cost =  [ 258872.77507019]Epoch: 92cost =  [ 258872.77507019]Epoch: 93cost =  [ 258872.77507019]Epoch: 94cost =  [ 258872.77507019]Epoch: 95cost =  [ 258872.77507019]Epoch: 96cost =  [ 258872.77507019]Epoch: 97cost =  [ 258872.77507019]Epoch: 98cost =  [ 2003326.79850769]Epoch: 99cost =  [ 258872.77507019]
Contrastive divergence in RBM
neural network;deep learning;theano;rbm