Why doesn't my pressure relief valve work as expected?
Sounds like you need an expansion tank for the hot water system. When you heat water, it expands. If there's no place for it to go, the pressure will rise until something gives. You might also need to adjust (or replace) the thermostat(s) on the water heater.

A PRV acts also as a check valve so 60 PSI into the house going cold into the water heater can then become a much higher pressure when the cold water is heated, if no valves are open using water. The new check valve (which you call a regulator, but I'm betting it's not) does the same thing closer to the water heater.