Fully differential opamp with weird behavior
I don't know what you are expecting without using input resistors. At the moment you have a differential trans impedance amplifier and that will have virtually zero ohms input impedance. The AD8331 is a voltage output device so something looks very screwed up in your interfacing philosophy.

Try using input resistors in series with each input line and look at what the VGA output of the AD8331 is expecting to see - fig 57.