Is wiring a motion sensor switch with 18 AWG wire on a 20 amp circuit innapprorpiate?
You are good to go. The NEC does not regulate manufactured equipment. In the US they are regulated primarily by UL, NEMA, and NFPA.  In you case the motion sensor is what will regulate your connected load. In other words, it will probably burn up before the wiring is damaged.

Good luck and stay safe.