Will a switch that says 250VAC 3A work directly connected to the wall?
In terms of Ohm's law, power, and resistive circuits (such as a lamp), AC and DC are the same. The amperages you are referring to aren't "delivered" amps, they are "rated" amps. Connecting something that draws more than the rated value will cause them to either trip or be damaged.

First, standard household outlets. In the USA, normally these are actually part of a circuit rated for 15A (not 20A) but to be sure you would have to look at the circuit breaker feeding the circuit. This is the most amps all of the outlets on the circuit can deliver combined without tripping the breaker. This hypothetically could be one outlet delivering 15A and nothing connected to the rest, but usually that's not the case. Further, you would not normally run the circuit above 80% of the trip value continuously, in the example I'm using, 12A. The breaker is likely to wear out over time above this level.

Second, the value on the switch is how much the switch can deliver without damage. Since a standard outlet in the USA is 110-125V, if you are in the USA you would use the 125V value, which is 6A. You cannot connect a load that would exceed his value.

As your friend stated, the lamp will draw whatever it draws in accordance with Ohm's law and the power equation P = IV (same as DC). For example, a 120W lamp would draw 120 W / 120V = 1A. 

If it is just a lamp, you're probably OK as this switch could power 125V x 6A = 750 W (this power is the same regardless of voltage/country since 250x3A is also 750 W), and most lamps are well under that.

AC and DC differ most when applying highly inductive (or, less likely, capacitive) loads, such as an AC motor. In that case standard DC equations do not work and you cannot merely add currents, or divide power by voltage to get current. Since you are using a lamp, it is basically the same as DC so I'm not going to go into those AC calculations here. As pointed out in the comments, another difference is an AC rated switch may not be suitable for DC as it might arc or even get welded shut in a DC circuit.