How to update the posterior belief when we are observing a stream of correlated data from a fixed but unknown data source
I think all of the options you give will yield results for the problem you described. It seems you have a y like a die that yields results which are rolls of the die all unpredictable but still linked from the y and you want to infer the y based on the x values. Like a 
 Hidden Markov model.  

The part about wanting to accurately make an estimate of y at each x and the x's being correlated make me believe that LSTM technology may be of benefit. If you want to use a NN.

Optimal stopping and Sequential Hypothesis Testing and Sequential probability ratio test and HDI+ROPE will all work for the abstract problem you describe. Until you give us more details about your problem like what specifically you will be working with it is hard to give you concise direction. 

If you create a LSTM That takes in an X and makes a guess about which y and train it on x data you have then you would have the predictive model you seek.