What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?
I had a very similar issue as you did with the dimensions. Here's the rundown:

Every node you see inside the LSTM cell has the exact same output dimensions, including the cell state. Otherwise, you'll see with the forget gate and output gate, how could you possible do an element wise multiplication with the cell state? They have to have the same dimensions in order for that to work.

Using an example where n_hiddenunits = 256:

Output of forget gate: 256
Input gate: 256
Activation gate: 256
Output gate: 256
Cell state: 256
Hidden state: 256


Now this can obviously be problematic if you want the LSTM to output, say, a one hot vector of size 5. So to do this, a softmax layer is slapped onto the end of the hidden state, to convert it to the correct dimension. So just a standard FFNN with normal weights (no bias', because softmax). Now, also imagining that we input a one hot vector of size 5:

input size: 5
total input size to all gates: 256+5 = 261 (the hidden state and input are appended)
Output of forget gate: 256
Input gate: 256
Activation gate: 256
Output gate: 256
Cell state: 256
Hidden state: 256
Final output size: 5


That is the final dimensions of the cell.