why we sample when predicting with Recurent Neural Network
It seems to me you have two questions:


Why use sampling to generate text from a trained RNN language model?
How does this particular sampling function from Keras work?


Why use sampling to generate text from a trained RNN language model

After training a language model, very often you would like to use the model to generate new text. For a word-level RNN language model, text is generated one word at a time. In each step, the model outputs a probability distribution over the entire vocabulary. With a toy vocabulary such as:

$$
vocabulary=\hspace{0.2cm} \begin{bmatrix}
           cat \\
           dog \\
           frog
         \end{bmatrix}
$$

the distribution output by the model at each generation step could look like

$$
predictions=\begin{bmatrix}
           0.7 \\
           0.1 \\
           0.2
         \end{bmatrix}
$$

each entry in the vector corresponds to one entry in our vocabulary.

After that, it is up to the user how to select a word from this distribution as the next word in the generated text. Two very different methods to pick a word from the distribution are:


Greedy search: take the word with the highest probability (will pick "cat")
Sampling: sample from the distribution, take into account probabilities (will pick a random word, but "cat" has the highest chance of being picked)


So, searching will return a very probable string (and the same one every time we try), while sampling will return somewhat likely, but more varied text. Often, this means that sampling leads to more "interesting" text.

How does this particular sampling function work?

As far as I can tell, the input for this sampling function is a vector of normalized probabilities after softmax, such as

np.array([0.7, 0.1, 0.2])


Here are some explanations about each line:

preds = np.asarray(preds).astype('float64')


This just makes sure that other array-like objects, such as Python lists, work as input, but are converted to the correct numpy array type.

preds = np.log(preds) / temperature


preds is a probability distribution that was created by applying the softmax function to the final output of the model network (those final values are called logits). Logits are assumed to be unnormalized log probabilities, applying softmax normalizes them.

This is a problem for softmax with temperature, because it is the unnormalized logits that are supposed to be divided by the temperature:

$$
q_i = \frac{exp(z_i/T)}{\sum_j{exp(z_j/T)}}
$$

So, np.log(preds) just moves the values back into logspace. Without this, we would not be scaling the actual network outputs, but exponentiated values.

exp_preds = np.exp(preds)
preds = exp_preds / np.sum(exp_preds)


Those two lines just apply the softmax function, regardless of what happens before or after.

probas = np.random.multinomial(1, preds, 1)


This function with those particular arguments is sampling a one-hot vector such as 

$$
predictions=\begin{bmatrix}
       1 \\
       0 \\
       0
     \end{bmatrix}
$$

where the probability of each index being the 1 is dictated by preds.

return np.argmax(probas)


Finally, this returns the index of the largest element in the vector. This index will correspond to one word in our vocabulary.



I hope this helps you understand the matter, or perhaps express more clearly what you would need additional explanations for.