How step-down voltage regulators behave when their input drops below their output voltage?
The LM2596 is probably a bit better than what you think. If you look at figure 8 in the data sheet: -



It tells you that for a 1A load (at 25ºC) the drop-out is about 0.9V. Also note that the output is now just slightly out of regulation (Vout = Vreg - 50mV).

If you are running less than 1A this figure will improve.

However, take note of figure 11: -



It is telling you that at 25ºC the minimum operating supply voltage is about 3.6V (and this id for producing a miserly 1.23V on the output).

There are better devices than this I suspect and you need to consider what your maximum load current is when selecting one.

The LP2950 is a linear regulator and if you look at figure 10 it tells you that for a 50 ohm load and 4V inputted you will get about 3.6V coming out. Also look at figure 12 because this gives a different viewpoint in terms of output current.