Well pump wont stop running
A typical well installation includes a pressure tank in the circuit. The tank contains a bladder which is inflated while the system is turned off and no water pressure is indicated. You use any inflator device such as a bicycle hand pump or tire pump to load the bladder to the recommended pressure.

When the pump is powered on, it will load the tank with water, compressing the bladder until the internal pressure reaches the cut-off switch setting. Let's say that's 50 psi as a reference level. Also, your cut-in setting should be lower, say 35 psi for this example.

The bladder pressure now matches the cut-off pressure and will continue to apply that level of pressure to the water in the system as it is used. When sufficient water has been used to expand the bladder to the original inflation level, the overall water pressure will drop during use until the cut-in figure is reached.

If your bladder has lost pressure or otherwise failed, you will have the circumstances you've described.