How can I order each product variant in my templates?
product.variants just returns an actual array of the variants, not an ElementCriteriaModel. So you can’t chain on additional params like order.

To get variants using custom parameters, you can do this:

{% set variants = craft.commerce.variants({
    product: product,
    order: 'price asc'
}) %}

{% for purchasable in variants %}
    // ...
{% endfor %}