Reset the cart if same product is added
Here you can find the solution for restricting duplicated items to the cart.

Solution is 


Create rule, name it; Event: before adding a product to the cart
You can add there conditions if you wish. I am chcecking here is product is digital or not.
In Actions add Loop through commerce-order:commerce-line-items
Create component, single rule, name it
Create following variables of the component and name them as you wish:


Commerce line item (line item of the order from the loop of the main rule)
Commerce product (product added to the cart)

Add condition to the component: Entity has field, Entity: [line-item-in-order], Field: commerce_product. this will bring us the product from actual line item in current order (shopping cart).
Add condition to the component: Data comparison. Data to compare: line-item-in-order:commerce-product with product-added-to-cart (variable brought from the main rule)
Add action to the component: Remove an item from a list. List: [order:commerce-line-items], Item to remove: [line-item-in-order]. This will completely remove duplicated product from the shopping cart and from the order as well.
Eventually add another actions. At least Display some message to the user to inform him/her that the item has been removed sounds sensible.
Come back to your main rule created in step 1. Add action to the loop and select the component you have just created in steps 4. - 9. and set the variables.
You are done. Chcek it, it should work