Is there HMM that looks two elements back?
I think the keyword you are looking for is Higher Order Hidden Markov Models. Your "ordinary" HMM would be considered a First Order HMM. So yes, there is literature these types of HMMs.