Resistor requirement for 3.3v
An LED has a point where the terminal voltage applied causes a significant rise in the current taken and if you increase that voltage by even as little as 0.1 volts, the current taken by the LED could be beyond its operational rating.

Because that voltage (for your LED) might be OK set at 3.3 volts, it doesn't mean you won't blow the back off it at 3.4 volts. This is why we put a current limit resistor in series. If your LED takes 100 mA at 3.3 volts (for instance) you would need to drop 1.7 volts from 5 volts with a resistor.

That means a resistor of value 1.7 volts / 100 mA = 17 ohm.

Now, if your 5 volt rose to 5.1 volts, the 100 mA would rise to no more than 1.8 volts / 17 ohm = 106 mA. If your LED is rated at a maximum current of 120 mA, you could in fact allow the 5 volts to rise to 5.34 volts before being on the cusp of exceeding its ratings.

Do you see how adding a resistor protects your LED from over-current?

For pull-down resistors on switches you need to ensure that if there is a wetting current requirement to be met, the resistor is low enough to allow that current to flow.