Faulty circuit or faulty opamp?
I've had trouble with oscillation around the loop when using a p-fet like that. The gate is going up whilst the feedback is going down and vice-a-versa. Could you perhaps see if there is a small alternating voltage across the load perhaps with an oscilloscope. Measuring the voltages with your meter set to dc will not tell you if there is ac there.
I solved my problem by swapping the p-fet for n-fet (source to motor) in source follower mode and taking the feedback back to the op-amp's inverting input with the control voltage applied to the non-inverting input.

Depending on your motor current you will need a heat sink on the transistor.