Noise in portable power supply to guitar effects pedals
Noise in audio devices is not always due to the power supply. Quite often one or more devices can introduce noise on the supply line due to its constantly changing demands and processing.

What you are looking for is isolation in terms of guitar pedal power supplies. This is covered thoroughly by guitar tech experts (such as Daniel Steinhardt of That Pedal Show, see this video).

If you use a power supply that isolates each pedal from the others, then noise introduced by one will not affect others. Note again that this is not (necessarily) due to a problem with your power supply but instead because one or more pedals actually introduce the noise to the common power bus.

Isolation circuits typically use a higher starting voltage (24V for example), and multiple step-down transformers or buck regulators, one for each 9V output. (Guitar pedals are usually 9VDC center negative, but not always. Always check polarity, voltage, and current requirements.)

Since this is an electrical engineering site, your question should pertain to technical aspects of designing or building such a power supply, rather than assembling something from USB/phone chargers. However, as both an electrical engineer and guitar nerd, I hope the above information is helpful in getting you closer to an engineering perspective.

Edit: Another possible cause of noise is that your power supply (battery or booster) cannot provide 600mA. Connecting a more demanding load than it can supply will cause one or more pedals to not operate correctly, and this can often result in buzzing, whining, clicking or ticking, etc.