The model of LSTM with more than one unit
The last image that you draw, where (at a given timestep) each cell of the second layer receives input from all cells in the first layer, is the right one.  

You can think of recurrent layers as fully connected layers which receive sequences of input and apply their transformations at each timestep. They just receive other inputs depending on their previous inputs but the connectivity between layer is exactly the same as a fully connected : each unit of a given layer performs a weighted average of the activations of all the units in the previous layer (or of the inputs if it's the first layer).