What's easier to change: largest or smallest tire for rims that support multiple sizes?
I actually run two tyres identical except for the width, and when I first fitted them it made no difference at all to the difficulty. My situation is a little different: they're marathon plus 28 and 32 mm (700c).

There's much more variation between brands, types, and maybe even batches, with anti puncture tyres generally harder to get on/off than cheap multi purpose tyres and some coming up big, some small (which makes a big difference) 

To be honest though, if the time to take a tyre off and change the tube is significant, something else is wrong. I've just had my first puncture in over 3000 miles on fairly rubbish tarmac, for example.