How to disable shipping for ONLY virtual products and not if combined with physical
Commerce Shipping has a couple of ways you can approach this. The simplest is to set the weight of your virtual products to zero, they won't be assessed shipping charges, and they won't affect any weight based calculations. 

EDIT: Alternatively, you can use Commerce Rules Extra to add conditions that exclude your virtual products in the conditions for your Flat Rate.