Is it okay for an electrical switch to get warm?
Short answer, most dimmers do get warm when in use. If the dimmer does not have a mechanical method of turning completely off. Then there is a good chance that you are still getting some electrical flow through the switch and that would mean the dimmer could stay warm. In other words what @Harper said.

The biggest question I would have would be if it was getting excessively hot? I would say somewhere above 30 C (86 F) would seem to be excessive.