How to add a JS confirmation popup when clicking an #ajax button
Drupal is a PHP framework, and it's amazing how much fancy AJAX stuff you can get using that and the FAPI. But it has it limits and for a use case like this, I would suggest that you use custom JavaScript. Also instead of using the usually JavaScript dialog, you could use jQuery UI to create a themable dialog instead.

Anyways the problem that you're facing is probably caused because you use AJAX on the submit button. Since you are using AJAX for the actual delete call, preventDefault and the like won't work.

What you would have to do is something like this. (This is not widely tested but should work.)

Drupal.behaviors.module = {
  attach: function() {

    var events = $('.confirm').data('events'); // Get the jQuery events.
    $('.confirm').unbind('click'); // Remove the click events.
    $('.confirm').click(function () {
      if (confirm('Are you sure you want to delete that?')) {
        $.each(events.click, function() {
          this.handler(); // Invoke the click handlers that was removed.
        });
      }
      // Prevent default action.
      return false;
    });
  }
}