Can a convolutional network predict states for a RL Agent
It sounds like what you're suggesting is similar to what is done in methods that use a planner. These methods looks to learn the dynamics of the MDP to use to plan during training; that is they want to be able to learn the transition probabilities $p(s'| s, a)$.
In this paper that I read recently they note that learning to predict environment dynamics when the state/action space is high dimensional, as is the case with images, is difficult; so whilst it may be possible in theory it would be difficult to do and if you were predicting many steps into the future then the error would compound.
A way around this, as is done in the referenced paper, is to use predict environment dynamics in a latent space. This means that they use a latent variable to predict the next state using e.g. Variational Autoencoders.