What causes such a narrow slit in a cliff? (see photo)
If you see the other user (Jan Doggen's) google maps link, you can see apparently similar features in different states - this is a particularly neat example of a general phenomenon.

These notches form when headlands are undercut by caves carved out by the sea. They may be initiated where there is a fault or jointing in the rock (northwest Jersey is granite, according to google). 

In this particular case, the notch seems to have a rounded termination. Blowholes also form from undercutting of headlands producing caves whose roofs then collapse. It's possible this cave may have collapsed after a blowhole formed, so you get a notch where the inland end of the feature appears rounded. That bit is just speculation though.

I found this Jersey geology trail guide that might have more detail: http://jerseygeologytrail.net/Geomorph.shtml