Aborting a program for invalid floating-point input
There are a couple of issues.


Why is the application pausing for 2 seconds before exiting?  Why not just exit straightaway?  (And if the answer is that you are running this via Windows 'cmd.exe' then you are better off handling the pausing at that level ...)
Given that you do need to pause, then you are better of writing a helper method to do something like "print a message, pause, and exit with a non-zero exit code".  Then use that in multiple places, as required.
You should be setting a non-zero exit code when you exit due to a failure.  This allows the script or whatever that launched the java app to detect the failure and (maybe) do something about it; e.g.  pause so that the user can see the error message ...
The InterruptedException is not a program error, so printing a stacktrace is probably not appropriate.  


Figuring out what to do with an InterruptedException exception is not straightforward.  The exception indicates that something (either the user or the application) has interrupted the application, either before or during the sleep.  


In this use-case you are just about to exit anyway, so the best strategy is to ignore it an exit anyway.
You need to decide what is a sensible thing to do.  Some possibilities are:


Ignore the interrupt and carry on regardless.
Take steps to cause the application to stop what it was doing; e.g. throw a custom unchecked "stop now" exception.  (You could also call System.exit() directly, but that can also be a bad thing to do, depending on the nature of the code.  For instance, calling System.exit() in a library ... or a webapp ... is a big non-no.)
Punt: reinstate the "interrupted" state by calling Thread.currentThread().interrupt() in the exception handler.  For instance, you might do this if some enclosing code is checking the "interrupted" flag.