How can I wipe my SD card on an unresponsive Nexus 4?
You can perform a factory reset with just using the physical buttons.

Step 1: Boot into recovery mode.

Step 2: Perform a reset from your recovery.

These steps don't require screen to sense touches. If your screen is completely gone so it shows nothing here's a video of it so you can follow from there.