Minimal requirements for a dedicated VNC client
I suspect you'll end up designing something very similar to the Raspberry Pi, with the trouble of manufacturing it yourself and the same cost issues (simply getting a PCB with ethernet, ps2, video and power connectors is probably going to exceed your $2 budget before you put any logic on it). I suspect with aggresive value engineering for your specific application you might be able to get the cost down to $10 by using a different core without the GPU and with less RAM.

I went looking on the Raspberry Pi blog for their own articles on the subject, but couldn't immediately find them; well worth reading that and Bunnie Huang's on the Chumby before setting out to do any manufacturing of your own.