how much overhead an error in RDBMS has?
If the unique field is properly indexed, a UNIQUE violation should be fairly quick.

Honestly though, I think doing a "two inner select" (I think I know what you mean here) is a better solution.  Even with trapping for an exception (at the application level), that's just not a very clean way to handle programatic logic.  Exception handling should be reserved for those rare occasions where you have an "exception to the rule."

And (without knowing which language you're coding in) if you're just trapping for a "SQLException", it's important to remember that those can be thrown for a variety of reasons. Your application might produce unexpected results if you incorrectly assume why an exception happened.

Bottom-line, I would say that the best approach is to do the SELECT to check for duplication.