how can LSTM output ever be negative with 0 initialization?
Cannot find the equations in your reference, so I take them from wikipedia. At first time step the following equation holds: $$h_{i} = \sigma(A)\tanh((\sigma(B)(\tanh(C))$$
where $A, B$ and $C$ are values calculated from linear functions. So it can be negative if $C$ is so.