Slight drop in voltage regulator circuit
That's behaving more like a 5.1 V Zener. At zero output current the voltage is a little over 5.1 V. At 3 mA it's about 5.1 V.

Once you draw more than 4 mA the voltage drop across the 1k resistor will be > 4 V so the output voltage will drop.

I recommend that you double-check the part number and specification. If you're sure it's a 4 V Zener (although your test measurements say that it's not) then add the part number and link to datasheet into your question.