Keras stateful LSTM returns NaN for validation loss
No idea about the tensor board stuff, but the NAN for val loss could be being caused by an unexpected very large or very small number. As your training dataset is very small there is a high chance that the validation data has a very different underlying distribution.
If the latent space of your NN isn't continuous and validation samples fall too far away from the distribution of the training samples, the NN will not deal with them well. In a classification problem this would result in misclassification (most likely). However, as you are generating a prediction, this could mean that the NN creates a really large number or a really small one. Then, in your loss function might be getting something divided by zero or by infinity. 

I'd suggest looking at the the loss function you are using, if it has a divide, can the denominator be zero? One solution to this would be to use the clip function to make sure that your predicted tensor is between 0+epsilon and 1.

def clipped_squared_error_loss(y_true, y_pred):
    y_true = K.clip(y_true, K.epsilon(), 1)
    y_pred = K.clip(y_pred, K.epsilon(), 1)
    return K.sum(K.square(y_pred - y_true))


You may also want to look into Kullback Leibler Divergence as a means to make the latent space of your NN continuous.