Gumbel-Softmax trick vs Softmax with temperature
Let's say you have two states, $X_1$ and $X_2$, and you have a model, $M$, that produces a score $M(X_i)$ for each state (i.e, the logits). Next you can use the logits to compute some distribution 

$$P = softmax(\{M(X_1), M(X_2)\})$$ 

and take the state with the highest probability 

$$X=argmax_{X_i}(P)$$ 

But what if you actually want to sample from $P$ instead of just taking the argmax - and you want the sample operation to be differentiable! This is where the Gumbel Trick comes in - instead of softmax, you compute

\begin{equation} 
\ X = argmax_{X_i}(\{M(X_i)+Z_i\})
\end{equation} 

Where $Zi$ are i.i.d Gumbel(0,1). It turns out that $X$ will be equal to $X_1$ about $P(X_1)$ of the times and to $X_2$ about $P(X_2)$ of the times. In other words, the equation above samples from $P$. 

But it still not differentiable because of the argmax operation. So instead of doing that we'll compute the Gumbel-Softmax distribution. Now if the temperature is low enough then the Gumbel-Softmax will produce something very close to one hot vector, where the probability of the predicted label will be 1 and other labels will have a probability of zero. So for example, if the Gumbel-Softmax gave the highest probability to $X_1$, you can do:

\begin{equation} 
\ X = \sum_{x_i} P_g(X_i)*X_i = 1*X_1 + 0*X_2 = X_1
\end{equation} 

Where $P_g$ is the Gumbel-Softmax operation. No argmax is needed! So with this cool trick we can sample from a discrete distribution in a differentiable way.