The influence of a potentiometer on photo-diode bias
You probably just have your variable resistance (potentiometer) installed in the upper portion of your voltage divider. You can play with a virtual one here and see the effect.

UPDATE (after schematic with photodiode was added):

It's impossible to say what's going on exactly because the "Amplifier Circuit" is drawn as a box, but in general...

when you decrease the bias resistance of the photodiode you are increasing the ratio of dark current to signal current which reduces the sensitivity and increases the offset (results in the behavior you describe with a higher voltage in zero light).

Sensitivity is the ratio: 

$$\frac{\mbox{change out}}{\mbox{change in}}$$

Getting less voltage difference out, for the same light difference in is a decrease in sensitivity.

UPDATE2: 

According to the datasheets here (click documents), Rs isn't accessible! Where are you connecting your potentiometer?