 The neural net architecture might be feed-forward or recurrent, and while the former is simpler the latter is more common.Typically, neural net language models are constructed and trained as probabilistic classifiers that learn to predict a probability distribution

I.e., the network is trained to predict a probability distribution over the vocabulary, given some linguistic context.This is done using standard neural net training algorithms such as stochastic gradient descent with backpropagation.