Replacing pressure gauge on water holding tank
There is no need for (and good reason not to have) a valve between the well and the pressure tank. Turn off the pump power, release pressure, change the gauge. There's no need to hurry. The well is below the tank, with no power to the pump the only way water will even want to flow is back to the well, though if the check valve on the pump is in good shape it won't. There should be a drain valve near the tank...drain all the water in the tank and take as long as you like to fix it.

Edit: either shut off the valve between the tank and the house (assuming there is one, there usually is), or shut off your water heater if there is no such valve, as draining or partially draining a water heater that's turned on is a bad thing...

If you add a valve where you suggest, you could have the valve shut off while the pump power is on, and your pressure switch would see low or no pressure (valve shut) and send power to the pump, which would duly pressurize up to the the shut valve, and eventually find the weak point in the system between the pump and the shut valve, blowing it open. Bad idea.

Likewise, if your pipes are not leaking, don't mess with them. Polyethylene pipe (what you probably have, not rubber) is not all that easy to work with, especially if you have no experience and it's old. If it's not leaking, save the experience for when you have a reason to get it. "Looking old" is a rather vague description - I have plenty of "old" hose clamps in service, they have been mostly made of stainless steel for quite long time, old and dirty (or even light surface rust, as even stainless can get) is not a problem (they are on the outside of the pipe.) 40 years old and just fine. If you have an oddball that's actually rusting to the point of being compromised, sure, replace it, but don't take the pipe apart.

When you are done, simply re-apply power to the pump, and run water until all the air is out of the pipes - this will often result in "alarming looking" but perfectly benign sediment being knocked loose, so don't be alarmed if that happens, just run the water until it clears up, and perhaps remove your faucet aerator screens and clean them. Very shortly (seconds) after you restore pump power you should hear water in the pipes, and see pressure build in the gauge you replaced. Sometimes you have to fiddle with a lever on the gray pressure switch box (if it has a low-pressure cut off.)