Understanding the multidimensional-nature of the data being fed to a RNN and its output
I think you’re confused about what a “batch” is. A batch has a very specific definition in machine learning. 

In my experience:


Dimension 1 = number of bins or number of data points per each time step
Dimension 2 = window size, the number of time steps
Dimension 3 = batch size, the total number of examples which you’ll feed the network per training batch


Looks like this for [4,3,2]:

[[[0,1,4,3],[0,8,6,9],[9,6,7,4]],[[6,8,7,0],[1,7,7,9],[1,3,5,8]]]


4 values per time step, 3 total time steps per example, 2 examples in the batch. 

Where [0,0,0] returns 0, [3,1,1] returns 9.

Also, one friendly piece of advice - don’t think in terms of time stamps. You’re not dealing with time in an RNN. You’re dealing with steps in a sequence. The sequence can be related to time.

Edit: With respect to the tensorflow stuff, a tf.dataset object can contain both the input data and the labels for each example. The point is to make writing code easier, managing data easier etc. 

The process for training/testing is the same.