Buck converter LM3671 acting as a (weird) voltage follower
I believe you have the fixed 5 volt version of the chip (where feedback is directly fed from Vout to Vfb) but you think you have the adjustable version that requires an external potential divider.

If I am correct, the device will always be trying to make the output as high as it can in order to produce 5 volts at Vfb and this is, what you appear to be seeing.



See the red box I've added to the bottom of the above diagram.