What loss function should I use for auto encoders for text
The cause of your huge loss is probably a lack of normalization. As an example, in the Keras-blog post you linked, they conduct such normalization by dividing the input by 255, which is the maximum value an element in the input can be of. 

As for which loss function to use, if you have a look at this keras-blog post, they construct a LSTM-based auto encoder for text which uses categorical crossentropy as a loss function.