How do I calculate the correct resistor, capacitor for a low pass circuit?
The use of decoupling caps is hugely dependant on the circuit in question. Any circuitry that generates a lot of high-speed switching is going to generate a noisy EMC profile. FPGAs, microcontrollers and high-speed interfaces all perform a lot of high speed switching and so require decoupling caps. 

Usually a device datasheet will specify recommended cap values, locations and even dialectrics. These values are usually chosen specifically for the device in question. Sometimes these caps are optional, to be added at the user's discretion based on surrounding circuitry. If a voltage regulator is going to be used in a noisy EMC environment, then the caps may be included, otherwise they may not be necessary (the datasheet will indicate this), however decoupling is generally good practise.

Often devices will have several caps in parallel of different values. Each cap filters up to a specific frequency based on the impedance (Z) of the circuit. The cut-off frequency is calculated as follows:

\$f_c = \dfrac{1}{2 \pi Z C}\$

However, the impedance of the circuit is the most difficult factor to calculate. It's based on your allowable noise margin and the amount of current that's going to be switched at once. This really useful TI application report gives details. 

If you're building a classic first order low-pass RC circuit then the Z turns out to be the R value and the maths is a lot easier.

This is the reason why most ICs give recommended cap values, they've done all the maths for you. If you'd like a practical example, the application note above gives a nice example on how to calculate a required cap value. 

In terms of testing, if you have a power supply on its own, you probably won't see much noise at all. It would be necessary to measure the power supply noise when you're using it to power a microcontroller or similar device and choose cap values based on what the application note for the device says. Also, give trial and error a try. Try different cap values and see what you get.