How do I know to choose a first order filter vs a second order filter?
Based on comments, you want 20 dB of attenuation.

A one pole low-pass filter rolls off at 20 dB per decade above its cut-off frequency. So you'd need to put your cut off one decade below the undesired signal frequency to achieve 20 dB attenuation with a one pole filter. That would be 12 Hz.

A two pole filter rolls of at 40 dB per decade. So you'd need to put your cut off half a decade below the undesired signal, which is \$120 / \sqrt{10}\$ or about 38 Hz, right about where you wanted to put it. I'd lower the cut-off frequency a bit further to be sure of reaching your goal of dropping the interference amplitude below 8 mV.

Or, consider using a notch filter instead, which can achieve higher attenuation in a narrow band around some target frequency for a similar design complexity.

Also, if your 120 Hz signal comes from your powerline, and you ever might want to use this circuit in a place with 50 Hz instead of 60 Hz power, you may want to adjust your filter design to cover 100-120 Hz (plus some margin) instead of just 120 Hz (hat tip to @TimWescott for suggesting this addition).