Is there a way to measure the "sharpness" of a decision boundary of a CNN?
You might enjoy looking into the literature on "adversarial examples".  Given an instance $x$ with a label $y$, an adversarial example is a (typically carefully constructed) instance $x'$ that is very similar to $x$, but whose label differs from $y$.  The research literature suggests that it is often possible to find adversarial examples that are very close to the original $x$.  You could use the distance $d(x,x')$ as a measure of sharpness of the decision boundary near $x$; or you could average this over many $x$ to get a global measure of sharpness.

There are many methods for finding adversarial examples.  A standard simple one is the gradient sign method, originally described in the following paper:

Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy. Explaining and harnessing adversarial examples. arXiv:1412.6572, 2014.

Since then there have been numerous improvements that find even closer adversarial examples, e.g., iterative gradient sign (arxiv:1607.02533), Deepfool (arxiv:1511.04599), and others.  You might also be interested in  Cleverhans, a software library to assist with finding adversarial examples.