title: Bias/Variance of Reinforcement Algorithms for Non-Markov States
body: <p>Hello StackOverflow Community!</p>
  
  <p>I have aquestion about Model-Free Prediction/Control algorithms in Reinforcement Learning.
  In the lectures of David Silver, the bias/variance tradeoff analysis is done for MC and TD (i.e. MC has no bias and high variance whereas TD(0) has some bias and low variance) but while making this comparisons the states in the environment have Markov property.<br>
  <br>
  Can you comment on what happens to bias and variance: <br>
  <strong>1.</strong> when we use MC in an environment with states,which do not have Markov property <br>
  <strong>2.</strong> same for the TD algorithm as well<br>
  compared to applied on states having Markov property?</p>