prevent script to run twice after 2 ajax loads
I figured it  out. The wrong code came out of misunderstsatnding .once function. "nav" means a class that the selector #colorbox #block-book-navigation would be marked, so code is not fired multiple times on that selector.
http://codekarate.com/blog/drupal-7-prevent-duplicating-javascript-behaviors
Correct code should be:

(function ($) {
  Drupal.behaviors.myCustomJS = {
    attach: function(context, settings) {
      $('#colorbox #block-book-navigation').once('nav', function() {

        $("#colorbox #block-book-navigation a").each(function(){         
            $(this).addClass("colorbox-node");

       });


      });
    }
  };
})(jQuery);