How to set Drupal Commerce shipping rules by the longest of the shortest dimensions of a product?
The best way to do this will be to create a custom rules action that sets this the way you want it. Writing rules actions isn't very scary. It's mostly a wrapper around a PHP function. Then you can get the logic just right. You could:


Write an action that takes the order as a parameter, returns the dimension you want, and then calculate the shipping in Rules, or
write a function that applies a shipping line item directly to the order.


Option 1 is easier to write and would be my initial tack on the problem. Option 2 is a bit more holistic but requires that you look at the shipping modules to see how they properly add a shipping line item to the order.