Are there any connectionist models that integrate reinforcement and  fully supervised learning?
I'm not sure I fully understand your design; perhaps you can clarify what you want your network to learn, why TD-learning "isn't cutting it", and what you mean by 'reinforcement' and  'prediction' learning. In particular, TD-learning is a reinforcement learning model, and it does reward based on predicted (and not just observed) outcomes. However, you seem to be describing reinforcement and prediction learning as orthogonal models, so again, I'm not sure I understand correctly.

As a general suggestion, you might consider using an Elman/Jordan network (e.g. recurrent neural network/RNN). Rather than relying on knowledge of only the current state to make a prediction about the next state, RNN can learn to recognize sequences of events. This is especially useful for predicting future states in a task that unfolds over time (e.g. [1]). I suggest this mostly because you say your task is a 'real-time control' task, but without more knowledge of your task I don't really know if this is appropriate.

As for your suggestion of using two different learning mechanisms to modify a single set of weights-- I don't have the answer, but it seems counterintuitive to me. You're using two different optimization techniques on a single set of data. If the techniques disagree, your network will probably never learn its connection weights. If they agree (i.e., converge on the same answer) then I'm not certain you're adding any value by having two learning mechanisms.

[1] Elman (1990). Finding structure in time. Cognitive Science, 14, 179-211. Retrieved from http://synapse.cs.byu.edu/~dan/678/papers/Recurrent/Elman.pdf