Drupal Commerce Hook - Extra validation before adding to cart
Seems like hook_commerce_cart_product_add is what you're after. I couldn't find any examples for preventing the line item being added to the cart but at the very least you should be able to use drupal_goto to interupt the process. You might also want to look at the Rules events\actions provided by Commerce.