Training Restricted Boltzmann Machines (RBMs) using gradient descent Hey I am a little new to the whole RBM entropy/energy training thing, having some trouble understanding and trying to figure whether it is worth the effort needed to understand.

Can't RBMs quite easily be trained with a standard gradient descent algorithm? Is the current way just much faster? 

Here is some sample Theano code of what I mean:

def rbm(inputs, shape, cost=0, last_cost=1):
    X = T.vector()                     # input
    W = theano.shared(np.zeros(shape)) # weights

    b = theano.shared(np.zeros( shape[1] )) # bias to hidden
    B = theano.shared(np.zeros( shape[0] )) # bias to reconstruction

    H = sigmoid(T.dot(X, W) + b) # hidden activation
    Y = T.dot(H, W.T) + B        # reconstruction

    L = ((X - Y) ** 2).mean()                    # loss function
    G = T.grad(L, W), T.grad(L, b), T.grad(L, B) # gradients

    a = .1                                                  # learning rate
    U = [(W, W - a*G[0]), (b, b - a*G[1]), (B, B - a*G[2])] # shared updates

    train = theano.function([X], L, updates=U) # training function

    while abs(last_cost-cost) &gt; 0.01:
        last_cost, cost = cost, 0
        for input in inputs: cost += train(input)

    return W.get_value(), b.get_value()


And yes this code is very crude.
What you're trying to do is to minimize the difference between input and its reconstruction. They may be completely different values and RBM may be working fine! It's due to the fact that Restricted Boltzmann Machine models some probabilistic distribution. Learning rule is to find values of weights to model the distribution of the data. You may clamp a data (visible units) and see a reconstruction, but since hidden units are stochastic units, they would have many values. Even when you set your network free and let the units have some values according to stochastic units rule, after some time you can get a sample from RBM, which should be from your data distribution.