Hard wiring Undercabinet kitchen lights made to plug in
You're correct that running an extension cord through the wall would be against code. Some undercabinet lights are convertible between hardwired and plug-in use, but it doesn't look like yours are. The best options would be to replace them with units designed to be hardwired. Otherwise, you'll have to keep the plugs, but you can take some simple steps to hide the cord such as:


Attach the cord to your cabinets using surface-mount raceway so it does not hang down.
Reposition the lights so there's less slack in the cords.
If it helps reduce cord clutter, interlink multiple lights rather than plugging them into separate switched receptacles.
Move the electrical outlet they're plugged into to a spot where the wiring will be hidden (perhaps including placing it inside a cabinet).