understanding linear algebra of a forget gate This blog covers the basics of LSTMs.

A forget gate is defined as :

$$f_t = \sigma(W_f \cdot [h_{t-1}, x_t]+ b_f)$$

At this point the linear algebra confuses me more than it should. The syntax of $W\cdot [h,x]$ is confusing in this context. I think a vector should go into the activation function since the output $f$ is a vector, but the syntax of the forget gate above implies that the input has $2$ columns because $[h,x]$ will be an $n\times 2$ matrix

For the sake of example  lets say ...

\begin{align} W &amp;= \begin{bmatrix} 0 &amp; 1 \\
          2 &amp;3 \end{bmatrix}\\
    h &amp;= \begin{bmatrix} -1 \\
          2 \end{bmatrix}\\
    x &amp;= \begin{bmatrix} 3 \\
          0 \end{bmatrix}\\
    b &amp;= \begin{bmatrix} 1 \\
          -2 \end{bmatrix}\end{align}

Can anyone give the final vector that goes into the sigmoid function ?

I think the math is 

$$  \begin{bmatrix} 0 &amp; 1 \\ 2 &amp; 3 \end{bmatrix}\begin{bmatrix} -3 &amp; 3 \\ 2 &amp; 0 \end{bmatrix} + \begin{bmatrix} 1 \\ -2\end{bmatrix} = \begin{bmatrix} 2 &amp; 0 \\ 4 &amp; 6\end{bmatrix}+ \begin{bmatrix} 1 \\ -2\end{bmatrix} = \text{ Something wrong}$$
Note that $$[h_{t-1}, x_t]$$
is the concatenation of two vectors.
In your example, it would be: $$[h_{t-1}, x_t] = [-1, 2 , 3, 0]$$
and then the dimensions of $W_f$ would be $2 \times 4$, where $2$ is the dimension of the output of the LSTM cell, i.e. the activation $h_t$, that you defined to be of dimension $2$.

Hence, $$W_f \cdot [h_{t-1}, x_t] $$ is a multiplication of a matrix of dimension $2\times4$ by a vector of $4$, which will return a vector of dimesion $2$. And then the sigmoid function will be applied point wise on each of the two elements of the result.

Hope it makes sense.